Shiryaev a., Peskir G. Optimal Stopping and Free-Boundary Problems s

Lectures in MathematicsETH ZürichDepartment of MathematicsResearch Institute of Mathematics

Managing Editor:Michael Struwe

Chapter I.

Optimal stopping: General facts

The aim of the present chapter is to exhibit basic results of general theory ofoptimal stopping. Both martingale and Markovian approaches are studied first indiscrete time and then in continuous time. The discrete time case, being directand intuitively clear, provides a number of important insights into the continuoustime case.

1. Discrete time

The aim of the present section is to exhibit basic results of optimal stopping inthe case of discrete time. We first consider a martingale approach. This is thenfollowed by a Markovian approach.

1.1. Martingale approach

1. Let G = (Gn)n≥0 be a sequence of random variables defined on a filteredprobability space (Ω,F , (Fn)n≥0, P) . We interpret Gn as the gain obtained ifthe observation of G is stopped at time n . It is assumed that G is adaptedto the filtration (Fn)n≥0 in the sense that each Gn is Fn -measurable. Recallthat each Fn is a σ -algebra of subsets of Ω such that F0 ⊂ F1 ⊂ · · · ⊂ F .Typically (Fn)n≥0 coincides with the natural filtration (FG

n )n≥0 but generallymay also be larger. We interpret Fn as the information available up to time n .All our decisions in regard to optimal stopping at time n must be based on thisinformation only (no anticipation is allowed).

The following definition formalizes the previous requirement and plays a keyrole in the study of optimal stopping.

Definition 1.1. A random variable τ : Ω → 0, 1, . . . ,∞ is called a Markov timeif τ ≤ n ∈ Fn for all n ≥ 0 . A Markov time is called a stopping time if τ < ∞P-a.s.

2 Chapter I. Optimal stopping: General facts

The family of all stopping times will be denoted by M , and the family ofall Markov times will be denoted by M . The following subfamilies of M will beused in the present chapter:

MNn = τ ∈ M : n ≤ τ ≤ N (1.1.1)

where 0 ≤ n ≤ N . For simplicity we will set MN = MN0 and Mn = M∞

n .

The optimal stopping problem to be studied seeks to solve

V∗ = supτ

EGτ (1.1.2)

where the supremum is taken over a family of stopping times. Note that (1.1.2)involves two tasks: (i) to compute the value function V∗ as explicitly as possible;(ii) to exhibit an optimal stopping time τ∗ at which the supremum is attained.

To ensure the existence of EGτ in (1.1.2) we need to impose additionalconditions on G and τ . If the following condition is satisfied (with GN ≡ 0when N = ∞ ):

E(

supn≤k≤N

|Gk|)

< ∞ (1.1.3)

then EGτ is well defined for all τ ∈ MNn . Although for many results below it is

possible to go beyond this condition and replace |Gk| above by G−k or G+

k (oreven consider only those τ for which EGτ is well defined) we will for simplicityassume throughout that (1.1.3) is satisfied. A more careful inspection of the proofswill easily reveal how the condition (1.1.3) can be relaxed.

With the subfamilies of stopping times MNn introduced in (1.1.1) above we

will associate the following value functions:

V Nn = sup

τ∈MNn

EGτ (1.1.4)

where 0 ≤ n ≤ N . Again, for simplicity, we will set V N = V N0 and Vn = V ∞

n .Likewise, we will set V = V ∞

0 when the supremum is taken over all τ in M .The main purpose of the present subsection is to study the optimal stoppingproblem (1.1.4) using a martingale approach.

Sometimes it is also of interest to admit that τ in (1.1.2) takes the value ∞with positive probability, so that τ belongs to M . In such a case we need to makean agreement about the value of Gτ on τ = ∞ . Clearly, if limn→∞ Gn exists,then G∞ is naturally set to take this value. Another possibility is to let G∞ takean arbitrary but fixed value. Finally, for certain reasons of convenience, it is usefulto set G∞ = lim supn→∞ Gn . In general, however, none of these choices is betterthan the others, and a preferred choice should always be governed by the meaningof a specific problem studied.

Section 1. Discrete time 3

2. The method of backward induction. The first method for solving the prob-lem (1.1.4) when N <∞ uses backward induction first to construct a sequenceof random variables (SN

n )0≤n≤N that solves the problem in a stochastic sense.Taking expectation then solves the problem in the original mean-valued sense.

Consider the optimal stopping problem (1.1.4) when N < ∞ . Recall that(1.1.4) reads more explicitly as follows:

V Nn = sup

n≤τ≤NEGτ (1.1.5)

where τ is a stopping time and 0 ≤ n ≤ N . To solve the problem we can let thetime go backward and proceed recursively as follows.

For n = N we have to stop immediately and our gain SNN equals GN .

For n = N − 1 we can either stop or continue. If we stop our gain SNN−1 will

be equal to GN−1 , and if we continue optimally our gain SNN−1 will be equal

to E (SNN | FN−1) . The latter conclusion reflects the fact that our decision about

stopping or continuation at time n = N − 1 must be based on the informationcontained in FN−1 only. It follows that if GN−1 ≥ E (SN

N | FN−1) then we needto stop at time n = N −1 , and if GN−1 < E (SN

N | FN−1) then we need to con-tinue at time n = N −1 . For n = N −2, . . . , 0 the considerations are continuedanalogously.

The method of backward induction just explained leads to a sequence ofrandom variables (SN

n )0≤n≤N defined recursively as follows:

SNn = GN for n = N, (1.1.6)

SNn = max

(Gn, E (SN

n+1 | Fn))

for n = N −1, . . . , 0. (1.1.7)

The method also suggests that we consider the following stopping time:

τNn = inf n ≤ k ≤ N : SN

k = Gk (1.1.8)

for 0 ≤ n ≤ N . Note that the infimum in (1.1.8) is always attained.

The first part of the following theorem shows that SNn and τN

n solve theproblem in a stochastic sense. The second part of the theorem shows that thisleads to a solution of the initial problem (1.1.5). The third part of the theoremprovides a supermartingale characterization of the solution. The method of back-ward induction and the results presented in the theorem play a central role in thetheory of optimal stopping.

Theorem 1.2. (Finite horizon) Consider the optimal stopping problem (1.1.5) uponassuming that the condition (1.1.3) holds. Then for all 0 ≤ n ≤ N we have:

SNn ≥ E (Gτ | Fn) for each τ ∈ MN

n , (1.1.9)

SNn = E (GτN

n| Fn). (1.1.10)


Moreover, if 0 ≤ n ≤ N is given and fixed, then we have:

The stopping time τNn is optimal in (1.1.5). (1.1.11)

If τ∗ is an optimal stopping time in (1.1.5) then τNn ≤ τ∗ P-a.s. (1.1.12)

The sequence (SNk )n≤k≤N is the smallest supermartingale which dom-

inates (Gk)n≤k≤N .(1.1.13)

The stopped sequence (SNk∧τN

n)n≤k≤N is a martingale. (1.1.14)

Proof. (1.1.9)–(1.1.10): The proof will be carried out by induction over n = N,N − 1, . . . , 0 . Note that both relations are trivially satisfied when n = N dueto (1.1.6) above. Let us thus assume that (1.1.9) and (1.1.10) hold for n =N, N − 1, . . . , k where k ≥ 1 , and let us show that (1.1.9) and (1.1.10) mustthen also hold for n = k − 1 .

(1.1.9): Take τ ∈ MNk−1 and set τ = τ ∨ k . Then τ ∈ MN

k and sinceτ ≥k ∈ Fk−1 it follows that

E (Gτ | Fk−1) = E(I(τ = k−1) Gk−1 | Fk−1

)+ E

(I(τ ≥k) Gτ | Fk−1

)(1.1.15)

= I(τ = k−1) Gk−1 + I(τ ≥k) E(E (Gτ | Fk) | Fk−1

).

By the induction hypothesis the inequality (1.1.9) holds for n = k . Since τ ∈MN

k this implies that E (Gτ | Fk) ≤ SNk . On the other hand, from (1.1.7) we

see that Gk−1 ≤ SNk−1 and E (SN

k | Fk−1) ≤ SNk−1 . Applying the preceding three

inequalities to the right-hand side of (1.1.15) we get

E (Gτ | Fk−1) ≤ I(τ = k−1)SNk−1 + I(τ ≥k) E (SN

k | Fk−1) (1.1.16)

≤ I(τ = k−1) SNk−1 + I(τ ≥k) SN

k−1 = SNk−1.

This shows that (1.1.9) holds for n = k − 1 as claimed.

(1.1.10): To prove that (1.1.10) holds for n = k − 1 it is enough to checkthat all inequalities in (1.1.15) and (1.1.16) remain equalities when τ = τN

k−1 . Forthis, note from (1.1.8) that τN

k−1 = τNk on τN

k−1 ≥ k , so that from (1.1.15) withτ = τN

k−1 and the induction hypothesis (1.1.10) for n = k , we get

E (GτNk−1

| Fk−1) = I(τNk−1 = k − 1) Gk−1 (1.1.17)

+ I(τNk−1≥k) E

(E (GτN

k| Fk) | Fk−1

)= I(τN

k−1 = k−1) Gk−1 + I(τNk−1≥k) E (SN

k | Fk−1)

= I(τNk−1 = k−1) SN

k−1 + I(τNk−1≥k) SN

k−1 = SNk−1

where in the second last equality we use that Gk−1 = SNk−1 on τN

k−1 = k− 1 by (1.1.8) as well as that E (SN

k | Fk−1) = SNk−1 on τN

k−1 ≥ k by (1.1.8) and(1.1.7). This shows that (1.1.10) holds for n = k − 1 as claimed.


(1.1.11): Taking E in (1.1.9) we find that ESNn ≥ EGτ for all τ ∈ MN

n andhence by taking the supremum over all τ ∈ MN

n we see that ESNn ≥ V N

n . On theother hand, taking the expectation in (1.1.10) we get ESN

n = EGτNn

which showsthat ESN

n ≤ V Nn . The two inequalities give the equality V N

n = ESNn , and since

ESNn = EGτN

n, we see that V N

n = EGτNn

implying the claim.

(1.1.12): We claim that the optimality of τ∗ implies that SNτ∗ = Gτ∗ P-a.s.

Indeed, if this would not be the case, then using that SNk ≥ Gk for all n ≤ k ≤ N

by (1.1.6)–(1.1.7), we see that SNτ∗ ≥ Gτ∗ with P(SN

τ∗ > Gτ∗) > 0 . It thusfollows that EGτ∗ < ESN

τ∗ ≤ ESNn = V N

n where the second inequality followsby the optional sampling theorem (page 60) and the supermartingale property of(SN

k )n≤k≤N established in (1.1.13) below, while the final equality follows from theproof of (1.1.11) above. The strict inequality, however, contradicts the fact thatτ∗ is optimal. Hence SN

τ∗ = Gτ∗ P-a.s. as claimed and the fact that τNn ≤ τ∗

P-a.s. follows from the definition (1.1.8).

(1.1.13): From (1.1.7) it follows that

SNk ≥ E (SN

k+1 | Fk) (1.1.18)

for all n≤k≤N −1 showing that (SNk )n≤k≤N is a supermartingale. From (1.1.6)

and (1.1.7) it follows that SNk ≥ Gk P-a.s. for all n ≤ k ≤ N meaning that

(SNk )n≤k≤N dominates (Gk)n≤k≤N . Moreover, if (Sk)n≤k≤N is another super-

martingale which dominates (Gk)n≤k≤N , then the claim that Sk ≥ SNk P-a.s.

can be verified by induction over k = N, N −1, . . . , l . Indeed, if k = N then theclaim follows by (1.1.6). Assuming that Sk ≥ SN

k P-a.s. for k = N, N −1, . . . , lwith l ≥ n + 1 it follows by (1.1.7) that SN

l−1 = max(Gl−1, E (SNl | Fl−1)) ≤

max(Gl−1, E (Sl | Fl−1)) ≤ Sl−1 P-a.s. using the supermartingale property of(Sk)n≤k≤N and proving the claim.

(1.1.14): To verify the martingale property

E(SN

(k+1)∧τNn

| Fk

)= SN

k∧τNn

(1.1.19)

with n ≤ k ≤ N − 1 given and fixed, note that

E(SN

(k+1)∧τNn

| Fk

)(1.1.20)

= E(I(τN

n ≤ k) SNk∧τN

n| Fk

)+ E

(I(τN

n ≥ k+1) SNk+1 | Fk

)= I(τN

n ≤ k) SNk∧τN

n+ I(τN

n ≥ k+1) E (SNk+1 | Fk)

= I(τNn ≤ k) SN

k∧τNn

+ I(τNn ≥ k+1) SN

k = SNk∧τN

n

where the second last equality follows from the fact that SNk = E (SN

k+1 | Fk) on τN

n ≥ k + 1 , while τNn ≥ k + 1 ∈ Fk since τN

n is a stopping time. Thisestablishes (1.1.19) and the proof of the theorem is complete.


Note that (1.1.9) can also be derived from the supermartingale property(1.1.13), and that (1.1.10) can also be derived from the martingale property(1.1.14), both by means of the optional sampling theorem (page 60).

It follows from Theorem 1.2 that the optimal stopping problem V N0 is solved

inductively by solving the problems V Nn for n = N, N − 1, . . . , 0 . Moreover, the

optimal stopping rule τNn for V N

n satisfies τNn = τN

k on τNn ≥k for 0 ≤ n ≤

k ≤ N where τNk is the optimal stopping rule for V N

k . This, in other words,means that if it was not optimal to stop within the time set n, n+1, . . . , k− 1then the same optimality rule applies in the time set k, k+1, . . . , N . In parti-cular, when specialized to the problem V N

0 , the following general principle isobtained: If the stopping rule τN

0 is optimal for V N0 and it was not optimal to

stop within the time set 0, 1, . . . , n− 1 , then starting the observation at timen and being based on the information Fn , the same stopping rule is still optimalfor the problem V N

n . This principle of solution for optimal stopping problemshas led to the general principle of dynamic programming in the theory of optimalstochastic control (often referred to as Bellman’s principle).

3. The method of essential supremum. The method of backward inductionby its nature requires that the horizon N be finite so that the case of infinitehorizon N remains uncovered. It turns out, however, that the random variablesSN

n defined by the recurrent relations (1.1.6)–(1.1.7) above admit a different char-acterization which can be directly extended to the case of infinite horizon N . Thischaracterization forms the basis for the second method that will now be presented.

With this aim note that (1.1.9) and (1.1.10) in Theorem 1.2 above suggestthat the following identity should hold:

SNn = sup

τ∈MNn

E (Gτ | Fn

). (1.1.21)

A difficulty arises, however, from the fact that both (1.1.9) and (1.1.10) hold onlyP-a.s. so that the exceptional P -null set may depend on the given τ ∈ MN

n . Thus,if the supremum in (1.1.21) is taken over uncountably many τ , then the right-hand side need not define a measurable function, and the identity (1.1.21) mayfail as well. To overcome this difficulty it turns out that the concept of essentialsupremum proves useful.

Lemma 1.3. (Essential supremum) Let Zα : α ∈ I be a family of randomvariables defined on (Ω,G, P) where the index set I can be arbitrary. Then thereexists a countable subset J of I such that the random variable Z∗ : Ω → R

defined by

Z∗ = supα∈J

Zα (1.1.22)


satisfies the following two properties :

P(Zα≤Z∗) = 1 for each α ∈ I. (1.1.23)

If Z : Ω → R is another random variable satisfying (1.1.23) inplace of Z∗, then P(Z∗≤ Z) = 1.

(1.1.24)

The random variable Z∗ is called the essential supremum of Zα : α∈ I relative to P and is denoted by Z∗ = esssupα∈I Zα . It is determined by theproperties (1.1.23) and (1.1.24) uniquely up to a P -null set.

Moreover, if the family Zα : α∈I is upwards directed in the sense that

For any α and β in I there exists γ in I such thatZα ∨ Zβ ≤ Zγ P-a.s.

(1.1.25)

then the countable set J = αn : n≥1 can be chosen so that

Z∗ = limn→∞Zαn P-a.s. (1.1.26)

where Zα1 ≤ Zα2 ≤ · · · P-a.s.

Proof. Since x → (2/π) arctan(x) is a strictly increasing function from R to[−1, 1] , it is no restriction to assume that |Zα| ≤ 1 for all α ∈ I . Otherwise,replace Zα by (2/π) arctan(Zα) for α ∈ I and proceed as in the rest of theproof.

Let C denote the family of all countable subsets C of I . Choose an increas-ing sequence Cn : n ≥ 1 in C such that

a = supC∈C

E(

supα∈C

Zα

)= sup

n≥1E(

supα∈Cn

Zα

). (1.1.27)

Then J :=⋃∞

n=1 Cn is a countable subset of I and we claim that Z∗ definedby (1.1.22) satisfies (1.1.23) and (1.1.24).

To verify these claims take α ∈ I arbitrarily and note the following. If α ∈ Jthen Zα ≤ Z∗ so that (1.1.23) holds. On the other hand, if α /∈ J and we assumethat P(Zα > Z∗) > 0 , then a < E (Z∗ ∨ Zα) ≤ a since a = EZ∗ ∈ [−1, 1](by the monotone convergence theorem) and J ∪ α belongs to C . As the strictinequality is clearly impossible, we see that (1.1.23) holds for all α ∈ I as claimed.Moreover, it is obvious that (1.1.24) follows from (1.1.22) and (1.1.23) since J iscountable.

Finally, if (1.1.25) is satisfied then the initial countable set J = α01, α

02, . . .

can be replaced by a new countable set J = α1, α2, . . . if we initially setα1 = α0

1 , and then inductively choose αn+1 ≥ αn ∨ α0n+1 for n ≥ 1 , where

γ ≥ α ∨ β corresponds to Zα , Zβ and Zγ such that Zγ ≥ Zα ∨ Zβ P-a.s.


The concluding claim in (1.1.26) is then obvious, and the proof of the lemma iscomplete.

With the concept of essential supremum we may now rewrite (1.1.9) and(1.1.10) in Theorem 1.2 above as follows:

SNn = esssup

n≤τ≤NE (Gτ | Fn) (1.1.28)

for all 0 ≤ n ≤ N . This identity provides an additional characterization of thesequence of random variables (SN

n )0≤n≤N introduced initially by means of therecurrent relations (1.1.6)–(1.1.7). Its advantage in comparison with the recurrentrelations lies in the fact that the identity (1.1.28) can naturally be extended tothe case of infinite horizon N . This programme will now be described.

Consider the optimal stopping problem (1.1.4) when N = ∞ . Recall that(1.1.4) reads more explicitly as follows:

Vn = supτ≥n

EGτ (1.1.29)

where τ is a stopping time and n ≥ 0 . To solve the problem we will consider thesequence of random variables (Sn)n≥0 defined as follows:

Sn = esssupτ≥n

E (Gτ | Fn) (1.1.30)

as well as the following stopping time:

τn = inf k ≥ n : Sk = Gk (1.1.31)

for n ≥ 0 where inf ∅ = ∞ by definition. The sequence (Sn)n≥0 is often referredto as the Snell envelope of G .

The first part of the following theorem shows that (Sn)n≥0 satisfies the samerecurrent relations as (SN

n )0≤n≤N . The second part of the theorem shows thatSn and τn solve the problem in a stochastic sense. The third part of the theoremshows that this leads to a solution of the initial problem (1.1.29). The fourth partof the theorem provides a supermartingale characterization of the solution.

Theorem 1.4. (Infinite horizon) Consider the optimal stopping problem (1.1.29)upon assuming that the condition (1.1.3) holds. Then the following recurrent rela-tions hold:

Sn = max(Gn, E (Sn+1 | Fn)

)(1.1.32)

for all n ≥ 0 . Assume moreover when required below that

P(τn <∞) = 1 (1.1.33)


where n ≥ 0 . Then for all n ≥ 0 we have:

Sn ≥ E (Gτ | Fn) for each τ ∈ Mn, (1.1.34)Sn = E (Gτn | Fn). (1.1.35)

Moreover, if n ≥ 0 is given and fixed, then we have:

The stopping time τn is optimal in (1.1.29). (1.1.36)If τ∗ is an optimal stopping time in (1.1.29) then τn ≤ τ∗ P-a.s. (1.1.37)The sequence (Sk)k≥n is the smallest supermartingale whichdominates (Gk)k≥n.

(1.1.38)

The stopped sequence (Sk∧τn)k≥n is a martingale. (1.1.39)

Finally, if the condition (1.1.33) fails so that P(τn = ∞) > 0 , then there is nooptimal stopping time (with probability 1) in (1.1.29).

Proof. (1.1.32): Let us first show that the left-hand side is smaller than the right-hand side when n ≥ 0 is given and fixed.

For this, take τ ∈ Mn and set τ = τ ∨ (n + 1) . Then τ ∈ Mn+1 and since τ ≥ n+1 ∈ Fn we have

E (Gτ | Fn) = E(I(τ = n)Gn | Fn

)+ E

(I(τ ≥ n+1)Gτ | Fn

)(1.1.40)

= I(τ = n) Gn + I(τ ≥ n+1) E (Gτ | Fn)

= I(τ = n) Gn + I(τ ≥ n+1) E(E (Gτ | Fn+1) | Fn)

≤ I(τ = n) Gn + I(τ ≥ n+1) E (Sn+1 | Fn)

≤ max(Gn, E (Sn+1 | Fn)

).

From this inequality it follows that

esssupτ≥n

E (Gτ | Fn) ≤ max(Gn, E (Sn+1 | Fn)

)(1.1.41)

which is the desired inequality.

To prove the reverse inequality, let us first note that Sn ≥ Gn P-a.s. by thedefinition of Sn so that it is enough to show that

Sn ≥ E (Sn+1 | Fn) (1.1.42)

which is the supermartingale property of (Sn)n≥0 . To verify this inequality, let usfirst show that the family E (Gτ | Fn+1) : τ ∈ Mn+1 is upwards directed in thesense that (1.1.25) is satisfied. For this, note that if σ1 and σ2 are from Mn+1

and we set σ3 = σ1IA + σ2IAc where A = E (Gσ1 | Fn+1) ≥ E (Gσ2 | Fn+1) ,then σ3 belongs to Mn+1 and we have

E (Gσ3 | Fn+1) = E (Gσ1IA+ Gσ2IAc | Fn+1) (1.1.43)

= IA E (Gσ1 | Fn+1) + IAc E (Gσ2 | Fn+1)

= E (Gσ1 | Fn+1) ∨ E (Gσ2 | Fn+1)


implying (1.1.25) as claimed. Hence by (1.1.26) there exists a sequence σk : k ≥ 1in Mn+1 such that

esssupτ≥n+1

E (Gτ | Fn+1) = limk→∞

E (Gσk| Fn+1) (1.1.44)

where E (Gσ1 | Fn+1) ≤ E (Gσ2 | Fn+1) ≤ · · · P-a.s. Since the left-hand sidein (1.1.44) equals Sn+1 , by the conditional monotone convergence theorem weget

E (Sn+1 | Fn) = E(

limk→∞

E (Gσk| Fn+1) | Fn

)(1.1.45)

= limk→∞

E(E (Gσk

| Fn+1) | Fn

)= lim

k→∞E (Gσk

| Fn) ≤ Sn

where the final inequality follows from the definition of Sn . This establishes(1.1.42) and the proof of (1.1.32) is complete.

(1.1.34): This inequality follows directly from the definition (1.1.30).

(1.1.35): The proof of (1.1.39) below shows that the stopped sequence(Sk∧τn)k≥n is a martingale. Moreover, setting G∗

n = supk≥n |Gk| we have

|Sk| ≤ esssupτ≥k

E( |Gτ | | Fk

) ≤ E (G∗n | Fk) (1.1.46)

for all k ≥ n . Since G∗n is integrable due to (1.1.3), it follows from (1.1.46) that

(Sk)k≥n is uniformly integrable. Thus the optional sampling theorem (page 60)can be applied to the martingale (Mk)k≥n = (Sk∧τn)k≥n and the stopping timeτn yielding

Mn = E (Mτn | Fn). (1.1.47)

Since Mn = Sn and Mτn = Sτn we see that (1.1.47) is the same as (1.1.35).

(1.1.36): This is proved using (1.1.34) and (1.1.35) in exactly the same wayas (1.1.11) above using (1.1.9) and (1.1.10).

(1.1.37): This is proved in exactly the same way as (1.1.12) above.

(1.1.38): It was shown in (1.1.42) that (Sk)k≥n is a supermartingale. More-over, it follows from (1.1.30) that Sk ≥ Gk P-a.s. for all k ≥ n meaningthat (Sk)k≥n dominates (Gk)k≥n . Finally, if (Sk)k≥n is another supermartingalewhich dominates (Gk)k≥n , then by (1.1.35) we find

Sk = E (Gτk| Fk) ≤ E (Sτk

| Fk) ≤ Sk (1.1.48)

for all k ≥ n where the final inequality follows by the optional sampling theo-rem (page 60) being applicable since S−

k ≤ G−k ≤ G∗

n for all k ≥ n with G∗n

integrable.


(1.1.39): This is proved in exactly the same way as (1.1.14) above.

Finally, note that the final claim follows directly from (1.1.37). This completesthe proof of the theorem.

4. In the last part of this subsection we will briefly explore a connectionbetween the two methods above when the horizon N tends to infinity in theformer.

For this, note from (1.1.28) that N → SNn and N → τN

n are increasing, sothat

S∞n = lim

N→∞SN

n and τ∞n = lim

N→∞τNn (1.1.49)

exist P-a.s. for each n ≥ 0 . Note also from (1.1.5) that N → V Nn is increasing,

so thatV ∞

n = limN→∞

V Nn (1.1.50)

exists for each n ≥ 0 . From (1.1.28) and (1.1.30) we see that

S∞n ≤ Sn and τ∞

n ≤ τn (1.1.51)

P-a.s. for each n ≥ 0 . Similarly, from (1.1.10) and (1.1.35) we find that

V ∞n ≤ Vn (1.1.52)

for each n ≥ 0 . The following simple example shows that in the absence of thecondition (1.1.3) above the inequalities in (1.1.51) and (1.1.52) can be strict.

Example 1.5. Let Gn =∑n

k=0 εk for n ≥ 0 where (εk)k≥0 is a sequence ofindependent and identically distributed random variables with P(εk = −1) =P(εk = 1) = 1/2 for k ≥ 0 . Setting Fn = σ(ε1, . . . , εn) for n ≥ 0 it followsthat (Gn)n≥0 is a martingale with respect to (Fn)n≥0 . From (1.1.28) using theoptional sampling theorem (page 60) one sees that SN

n = Gn and hence τNn = n

as well as V Nn = 0 for all 0 ≤ n ≤ N . On the other hand, if we make use of the

stopping times σm = inf k ≥ n : Gk = m upon recalling that P(σm < ∞) = 1whenever m ≥ 1 , it follows by (1.1.30) that Sn ≥ m P-a.s. for all m ≥ 1 . Fromthis one sees that Sn = ∞ P-a.s. and hence τn = ∞ P-a.s. as well as Vn = ∞for all n ≥ 0 . Thus, in this case, all inequalities in (1.1.51) and (1.1.52) are strict.

Theorem 1.6. (From finite to infinite horizon) Consider the optimal stopping prob-lems (1.1.5) and (1.1.29) upon assuming that the condition (1.1.3) holds. Thenequalities in (1.1.51) and (1.1.52) hold for all n ≥ 0 .

Proof. Letting N → ∞ in (1.1.7) and using the conditional monotone convergencetheorem one finds that the following recurrent relations hold:

S∞n = max

(Gn, E (S∞

n+1 | Fn))

(1.1.53)


for all n ≥ 0 . In particular, it follows that (S∞n )n≥0 is a supermartingale. Since

S∞n ≥ Gn P-a.s. we see that (S∞

n )− ≤ G−n ≤ supn≥0 G−

n P-a.s. for all n ≥ 0from where by means of (1.1.3) we see that ((S∞

n )−)n≥0 is uniformly integrable.Thus by the optional sampling theorem (page 60) we get

S∞n ≥ E (S∞

τ | Fn) (1.1.54)

for all τ ∈ Mn . Moreover, since S∞k ≥ Gk P-a.s. for all k ≥ n , it follows that

S∞τ ≥ Gτ P-a.s. for all τ ∈ Mn , and hence

E (S∞τ | Fn) ≥ E (Gτ | Fn) (1.1.55)

for all τ ∈ Mn . Combining (1.1.54) and (1.1.55) we see by (1.1.30) that S∞n ≥

Sn P-a.s. for all n ≥ 0 . Since the reverse inequality holds in general as shownin (1.1.51) above, this establishes that S∞

n = Sn P-a.s. for all n ≥ 0 . Fromthis it also follows that τ∞

n = τn P-a.s. for all n ≥ 0 . Finally, the third identityV ∞

n = Vn follows by the monotone convergence theorem. The proof of the theoremis complete.

1.2. Markovian approach

In this subsection we will present basic results of optimal stopping when the timeis discrete and the process is Markovian. (Basic definitions and properties of suchprocesses are given in Subsections 4.1 and 4.2.)

1. Throughout we consider a time-homogeneous Markov chain X = (Xn)n≥0

defined on a filtered probability space (Ω,F , (Fn)n≥0, Px) and taking values ina measurable space (E,B) where for simplicity we assume that E = Rd forsome d ≥ 1 and B = B(Rd) is the Borel σ -algebra on Rd . It is assumedthat the chain X starts at x under Px for x ∈ E . It is also assumed thatthe mapping x → Px(F ) is measurable for each F ∈ F . It follows that themapping x → Ex(Z) is measurable for each random variable Z . Finally, withoutloss of generality we assume that (Ω,F) equals the canonical space (EN0 ,BN0)so that the shift operator θn : Ω → Ω is well defined by θn(ω)(k) = ω(n+k) forω = (ω(k))k≥0 ∈ Ω and n, k ≥ 0 . (Recall that N0 stands for N ∪ 0 .)

2. Given a measurable function G : E → R satisfying the following condition(with G(XN ) = 0 if N = ∞ ):

Ex

(sup

0≤n≤N|G(Xn)|

)< ∞ (1.2.1)

for all x ∈ E , we consider the optimal stopping problem

V N (x) = sup0≤τ≤N

ExG(Xτ ) (1.2.2)


where x ∈ E and the supremum is taken over all stopping times τ of X . Thelatter means that τ is a stopping time with respect to the natural filtration of Xgiven by FX

n = σ(Xk : 0 ≤ k ≤ n) for n ≥ 0 . Since the same results remainvalid if we take the supremum in (1.2.2) over stopping times τ with respect to(Fn)n≥0 , and this assumption makes final conclusions more powerful (at leastformally), we will assume in the sequel that the supremum in (1.2.2) is taken overthis larger class of stopping times. Note also that in (1.2.2) we admit that N canbe +∞ as well. In this case, however, we still assume that the supremum is takenover stopping times τ , i.e. over Markov times τ satisfying τ <∞ P-a.s. In thisway any specification of G(X∞) becomes irrelevant for the problem (1.2.2).

3. To solve the problem (1.2.2) in the case when N < ∞ we may note thatby setting

Gn = G(Xn) (1.2.3)

for n ≥ 0 the problem (1.2.2) reduces to the problem (1.1.5) where instead of Pand E we have Px and Ex for x ∈ E . Introducing the expectation in (1.2.2)with respect to Px under which X0 = x and studying the resulting problem bymeans of the mapping x → V N (x) for x ∈ E constitutes a profound step whichmost directly aims to exploit the Markovian structure of the problem. (The sameremark applies in the theory of optimal stochastic control in contrast to classicalmethods developed in calculus of variations.)

Having identified the problem (1.2.2) as the problem (1.1.5) we can applythe method of backward induction (1.1.6)–(1.1.7) which leads to a sequence ofrandom variables (SN

n )0≤n≤N and a stopping time τNn defined in (1.1.8). The

key identity isSN

n = V N−n(Xn) (1.2.4)

for 0 ≤ n ≤ N . This will be established in the proof of the next theorem. Once(1.2.4) is known to hold, the results of Theorem 1.2 translate immediately intothe present setting and get a more transparent form as follows.

In the sequel we set

Cn = x ∈ E : V N−n(x) > G(x) , (1.2.5)

Dn = x ∈ E : V N−n(x) = G(x) (1.2.6)

for 0 ≤ n ≤ N . We define

τD = inf 0 ≤ n ≤ N : Xn ∈ Dn . (1.2.7)

Finally, the transition operator T of X is defined by

TF (x) = ExF (X1) (1.2.8)

for x ∈ E whenever F : E → R is a measurable function so that F (X1) isintegrable with respect to Px for all x ∈ E .


Theorem 1.7. (Finite horizon: The time-homogeneous case) Consider the optimalstopping problem (1.2.2) upon assuming that the condition (1.2.1) holds. Then thevalue function V n satisfies the Wald–Bellman equations

V n(x) = max(G(x), TV n−1(x)) (x ∈ E) (1.2.9)

for n = 1, . . . , N where V 0 = G . Moreover, we have:

The stopping time τD is optimal in (1.2.2). (1.2.10)If τ∗ is an optimal stopping time in (1.2.2) then τD ≤ τ∗ Px-a.s.for every x ∈ E.

(1.2.11)

The sequence (V N−n(Xn))0≤n≤N is the smallest supermartingalewhich dominates (G(Xn))0≤n≤N under Px for x ∈ E given and fixed.

(1.2.12)

The stopped sequence (V N−n∧τD(Xn∧τD ))0≤n≤N is a martingaleunder Px for every x ∈ E.

(1.2.13)

Proof. To verify (1.2.4) recall from (1.1.10) that

SNn = Ex

(G(XτN

n) | Fn

)(1.2.14)

for 0 ≤ n ≤ N . Since SN−nk θn = SN

n+k we get that τNn satisfies

τNn = inf n ≤ k ≤ N : SN

k = G(Xk) = n + τN−n0 θn (1.2.15)

for 0 ≤ n ≤ N . Inserting (1.2.15) into (1.2.14) and using the Markov property weobtain

SNn = Ex

(G(Xn+τN−n

0 θn) | Fn

)= Ex

(G(XτN−n

0) θn | Fn

)(1.2.16)

= EXnG(XτN−n0

) = V N−n(Xn)

where the final equality follows by (1.1.9)–(1.1.10) which imply

ExSN−n0 = ExG(XτN−n

0) = sup

0≤τ≤N−nExG(Xτ ) = V N−n(x) (1.2.17)

for 0 ≤ n ≤ N and x ∈ E . Thus (1.2.4) holds as claimed.

To verify (1.2.9) note that (1.1.7) using (1.2.4) and the Markov propertyreads as follows:

V N−n(Xn) = max(G(Xn), Ex

(V N−n−1(Xn+1) | Fn

))(1.2.18)

= max(G(Xn), Ex

(V N−n−1(X1) θn | Fn

))= max

(G(Xn), EXn

(V N−n−1(X1)

))= max

(G(Xn), TV N−n−1(Xn)

)


for all 0 ≤ n ≤ N . Letting n = 0 and using that X0 = x under Px we see that(1.2.18) yields (1.2.9).

The remaining statements of the theorem follow directly from Theorem 1.2above. The proof is complete.

4. The Wald–Bellman equations (1.2.9) can be written in a more compactform as follows. Introduce the operator Q by setting

QF (x) = max(G(x), TF (x)) (1.2.19)

for x ∈ E where F : E → R is a measurable function for which F (X1) ∈ L1(Px)for x ∈ E . Then (1.2.9) reads as follows:

V n(x) = QnG(x) (1.2.20)

for 1 ≤ n ≤ N where Qn denotes the n -th power of Q . The recursive relations(1.2.20) form a constructive method for finding V N when Law(X1 | Px) is knownfor x ∈ E .

5. Let us now discuss the case when X is a time-inhomogeneous Markovchain. Setting Zn = (n, Xn) for n ≥ 0 one knows that Z = (Zn)n≥0 is a time-homogeneous Markov chain. Given a measurable function G : 0, 1, . . . , N×E →R satisfying the following condition:

En,x

(sup

0≤k≤N−n|G(n+k, Xn+k)|

)< ∞ (1.2.1′)

for all 0 ≤ n ≤ N and x ∈ E , the optimal stopping problem (1.2.2) thereforenaturally extends as follows:

V N (n, x) = sup0≤τ≤N−n

En,xG(n+τ, Xn+τ) (1.2.2′)

where the supremum is taken over stopping times τ of X and Xn = x underPn,x with 0 ≤ n ≤ N and x ∈ E given and fixed.

As above one verifies that

SNn+k = V N (n+k, Xn+k) (1.2.21)

under Pn,x for 0 ≤ n ≤ N − n . Moreover, inserting this into (1.1.7) and usingthe Markov property one finds

V N (n+k, Xn+k) (1.2.22)

= max(G(n+k, Xn+k), En,x

(V N (n+k+1, Xn+k+1) | Fn+k

))= max

(G(Zn+k), Ez

(V N (Zn+k+1) | Fn+k

))= max

(G(Zn+k), Ez

(V N (Z1) θn+k | Fn+k

))= max

(G(Zn+k), EZn+k

(V N(Z1)

))


for 0 ≤ k ≤ N − n − 1 where z = (n, x) with 0 ≤ n ≤ N and x ∈ E . Lettingk = 0 and using that Zn = z = (n, x) under Pz , one gets

V N (n, x) = max(G(n, x), TV N (n, x)

)(1.2.23)

for n = N−1, . . . , 1, 0 where TV N (N−1, x) = EN−1,xG(N, XN ) and T is thetransition operator of Z given by

TF (n, x) = En,xF (n+1, Xn+1) (1.2.24)

for 0 ≤ n ≤ N and x ∈ E whenever the right-hand side in (1.2.24) is well defined(finite).

The recursive relations (1.2.23) are the Wald–Bellman equations correspond-ing to the time-inhomogeneous problem (1.2.2′) . Note that when X is time-homogeneous (and G = G(x) only) we have V N (n, x) = V N−n(x) and (1.2.23)reduces to (1.2.9). In order to present a reformulation of the property (1.2.12) inTheorem 1.7 above we will proceed as follows.

6. The following definition plays a fundamental role in finding a solution tothe optimal stopping problem (1.2.2′) .

Definition 1.8. A measurable function F : 0, 1, . . . , N × E → R is said to besuperharmonic if

TF (n, x) ≤ F (n, x) (1.2.25)

for all (n, x) ∈ 0, 1, . . . , N × E .

It is assumed in (1.2.25) that TF (n, x) is well defined i.e. that F (n +1, Xn+1) ∈ L1(Pn,x) for all (n, x) as above. Moreover, if F (n+k, Xn+k) ∈ L1(Pn,x)for all 0 ≤ k ≤ N − n and all (n, x) as above, then one verifies directly by theMarkov property that the following stochastic characterization of superharmonicfunctions holds:

F is superharmonic if and only if (F (n+k, Xn+k))0≤k≤N−n isa supermartingale under Pn,x for all (n, x) ∈ 0, 1, . . . , N − 1×E .

(1.2.26)

The proof of this fact is simple and will be given in a more general case following(1.2.40) below.

Introduce the continuation set

C =(n, x) ∈ 0, 1, . . . , N × E : V (n, x) > G(n, x)

(1.2.27)

and the stopping set

D =(n, x) ∈ 0, 1, . . . , N × E : V (n, x) = G(n, x)

. (1.2.28)

Introduce the first entry time τD into D by setting

τD = inf n ≤ k ≤ N : (n+k, Xn+k) ∈ D (1.2.29)

under Pn,x where (n, x) ∈ 0, 1, . . . , N × E .


The preceding considerations may now be summarized in the following ex-tension of Theorem 1.7.

Theorem 1.9. (Finite horizon: The time-inhomogeneous case) Consider the opti-mal stopping problem (1.2.2′) upon assuming that the condition (1.2.1′) holds.Then the value function V N satisfies the Wald–Bellman equations

V N (n, x) = max(G(n, x), TV N (n, x)

)(1.2.30)

for n = N −1, . . . , 1, 0 where TV N (N −1, x) = EN−1,xG(N, XN ) and x ∈ E .Moreover, we have:

The stopping time τD is optimal in (1.2.2′). (1.2.31)If τ∗ is an optimal stopping time in (1.2.2′) then τD ≤ τ∗ Pn,x-a.s.for every (n, x) ∈ 0, 1, . . . , N×E.

(1.2.32)

The value function V N is the smallest superharmonic function whichdominates the gain function G on 0, 1, . . . , N×E.

(1.2.33)

The stopped sequence(V N ((n+k) ∧ τD, X(n+k)∧τD

))0≤k≤N−n

isa martingale under Pn,x for every (n, x) ∈ 0, 1, . . . , N×E.

(1.2.34)

Proof. The proof is carried out in exactly the same way as the proof of Theorem 1.7above. The key identity linking the problem (1.2.2′) to the problem (1.1.5) is(1.2.21). This yields (1.2.23) i.e. (1.2.30) as shown above. Note that (1.2.33) refines(1.2.12) and follows by (1.2.26). The proof is complete.

7. Consider the optimal stopping problem (1.2.2) when N = ∞ . Recall that(1.2.2) reads as follows:

V (x) = supτ

ExG(Xτ ) (1.2.35)

where τ is a stopping time of X and Px(X0 = x) = 1 for x ∈ E .

Introduce the continuation set

C = x ∈ E : V (x) > G(x) (1.2.36)

and the stopping setD = x ∈ E : V (x) = G(x) . (1.2.37)

Introduce the first entry time τD into D by setting

τD = inf t ≥ 0 : Xt ∈ D . (1.2.38)

8. The following definition plays a fundamental role in finding a solution tothe optimal stopping problem (1.2.35). Note that Definition 1.8 above may beviewed as a particular case of this definition.

Definition 1.10. A measurable function F : E → R is said to be superharmonic if

TF (x) ≤ F (x) (1.2.39)

for all x ∈ E .


It is assumed in (1.2.39) that TF (x) is well defined by (1.2.8) above i.e.that F (X1) ∈ L1(Px) for all x ∈ E . Moreover, if F (Xn) ∈ L1(Px) for all n ≥ 0and all x ∈ E , then the following stochastic characterization of superharmonicfunctions holds (recall (1.2.26) above):

F is superharmonic if and only if (F (Xn))n≥0 is a supermartingaleunder Px for every x ∈ E .

(1.2.40)

The proof of this equivalence relation is simple. Suppose first that F issuperharmonic. Then (1.2.39) holds for all x ∈ E and therefore by the Markovproperty we get

TF (Xn) = EXn(F (X1)) = Ex(F (X1) θn | Fn) (1.2.41)

= Ex(F (Xn+1) | Fn) ≤ F (Xn)

for all n ≥ 0 proving the supermartingale property of (F (Xn))n≥0 under Px forevery x ∈ E . Conversely, if (F (Xn))n≥0 is a supermartingale under Px for everyx ∈ E , then the final inequality in (1.2.41) holds for all n ≥ 0 . Letting n = 0and taking Ex on both sides gives (1.2.39). Thus F is superharmonic as claimed.

9. In the case of infinite horizon (i.e. when N = ∞ in (1.2.2) above) it isnot necessary to treat the time-inhomogeneous case separately from the time-homogeneous case as we did it for clarity in the case of finite horizon (i.e. whenN < ∞ in (1.2.2) above). This is due to the fact that the state space E may begeneral anyway (two-dimensional) and the passage from the time-inhomogeneousprocess (Xn)n≥0 to the time-homogeneous process (n, Xn)n≥0 does not affect thetime set in which the stopping times of X take values (by altering the remainingtime).

Theorem 1.11. (Infinite horizon) Consider the optimal stopping problem (1.2.35)upon assuming that the condition (1.2.1) holds. Then the value function V satis-fies the Wald–Bellman equation

V (x) = max(G(x), TV (x)) (1.2.42)

for x ∈ E . Assume moreover when required below that

Px(τD < ∞) = 1 (1.2.43)

for all x ∈ E . Then we have:

The stopping time τD is optimal in (1.2.35). (1.2.44)If τ∗ is an optimal stopping time in (1.2.35) then τD ≤ τ∗ Px-a.s. forevery x ∈ E.

(1.2.45)

The value function V is the smallest superharmonic function whichdominates the gain function G on E.

(1.2.46)

The stopped sequence (V (Xn∧τD))n≥0 is a martingale under Px forevery x ∈ E.

(1.2.47)


Finally, if the condition (1.2.43) fails so that Px(τD = ∞) > 0 for some x ∈ E ,then there is no optimal stopping time (with probability 1 ) in (1.2.35).

Proof. The key identity in reducing the problem (1.2.35) to the problem (1.1.29)is

Sn = V (Xn) (1.2.48)

for n ≥ 0 . This can be proved by passing to the limit for N → ∞ in (1.2.4) andusing the result of Theorem 1.6 above. In exactly the same way one derives (1.2.42)from (1.2.9). The remaining statements follow from Theorem 1.4 above. Note alsothat (1.2.46) refines (1.1.38) and follows by (1.2.40). The proof is complete.

Corollary 1.12. (Iterative method) Under the initial hypothesis of Theorem 1.11we have

V (x) = limn→∞ QnG(x) (1.2.49)

for all x ∈ E .

Proof. It follows from (1.2.9) and Theorem 1.6 above. The relation (1.2.49) offers a constructive method for finding the value func-

tion V . (Note that n → QnG(x) is increasing on 0, 1, 2, . . . for every x ∈ E .)

10. We have seen in Theorem 1.7 and Theorem 1.9 that the Wald–Bellmanequations (1.2.9) and (1.2.30) characterize the value function V N when the hori-zon N is finite (i.e. these equations cannot have other solutions). This is dueto the fact that V N equals G in the “end of time” N . When the horizon Nis infinite, however, this characterization is no longer true for the Wald–Bellmanequation (1.2.42). For example, if G is identically equal to a constant c thenany other constant C larger than c will define a function solving (1.2.42). Onthe other hand, it is evident from (1.2.42) that every solution of this equation issuperharmonic and dominates G . By (1.2.46) we thus see that a minimal solutionof (1.2.42) will coincide with the value function. This “minimality condition” (overall points) can be replaced by a single condition as the following theorem shows.From the standpoint of finite horizon such a “boundary condition at infinity” isnatural.

Theorem 1.13. (Uniqueness in the Wald–Bellman equation)Under the hypothesis of Theorem 1.11 suppose that F : E → R is a functionsolving the Wald–Bellman equation

F (x) = max(G(x), TF (x)) (1.2.50)

for x ∈ E . (It is assumed that F is measurable and F (X1) ∈ L1(Px) for allx ∈ E .) Suppose moreover that F satisfies

E(

supn≥0

F (Xn))

< ∞. (1.2.51)


Then F equals the value function V if and only if the following “boundary con-dition at infinity” holds:

lim supn→∞

F (Xn) = lim supn→∞

G(Xn) Px -a.s. (1.2.52)

for every x ∈ E . ( In this case the lim sup on the left-hand side of (1.2.52) equalsthe lim inf , i.e. the sequence (F (Xn))n≥0 is convergent Px -a.s. for every x ∈ E .)

Proof. If F = V then by (1.2.46) we know that F is the smallest superharmonicfunction which dominates G on E . Let us show (the fact of independent interest)that any such function F must satisfy (1.2.52). Note that the condition (1.2.51)is not needed for this implication.

Since F ≥ G we see that the left-hand side in (1.2.52) is evidently largerthan the right-hand side. To prove the reverse inequality, consider the functionH : E → R defined by

H(x) = Ex

(supn≥0

G(Xn))

(1.2.53)

for x ∈ E . Then the key property of H stating that

H is superharmonic (1.2.54)

can be verified as follows. By the Markov property we have

TH(x) = ExH(X1) = Ex

(EX1

(supn≥0

G(Xn)))

(1.2.55)

= Ex

(Ex

(supn≥0

G(Xn) θ1

∣∣ F1

))= Ex

(supn≥0

G(Xn+1))

≤ H(x)

for all x ∈ E proving (1.2.54). Moreover, since X0 = x under Px we see thatH(x) ≥ G(x) for all x ∈ E . Hence F (x) ≤ H(x) for all x ∈ E by assumption.By the Markov property it thus follows that

F (Xn) ≤ H(Xn) = EXn

(supk≥0

G(Xk))

= Ex

(supk≥0

G(Xk) θn

∣∣ Fn

)(1.2.56)

= Ex

(supk≥0

G(Xk+n)∣∣ Fn

)≤ Ex

(supl≥m

G(Xl)∣∣ Fn

)for any m ≤ n given and fixed where x ∈ E . The final expression in (1.2.56)defines a (generalized) martingale for n ≥ 1 under Px which is known to convergePx -a. s. as n → ∞ for every x ∈ E with the limit satisfying the followinginequality:

limn→∞Ex

(supl≥m

G(Xl)∣∣ Fn

)≤ Ex

(supl≥m

G(Xl)∣∣ F∞

)= sup

l≥mG(Xl) (1.2.57)


where the final identity follows from the fact that supl≥m G(Xl) is F∞ -measur-able. Letting n → ∞ in (1.2.56) and using (1.2.57) we find

lim supn→∞

F (Xn) ≤ supl≥m

G(Xl) Px -a.s. (1.2.58)

for all m ≥ 0 and x ∈ E . Letting finally m → ∞ in (1.2.58) we end up with(1.2.52). This ends the first part of the proof.

Conversely, suppose that F satisfies (1.2.50)–(1.2.52) and let us show thatF must then be equal to V . For this, first note that (1.2.50) implies that F issuperharmonic and that F ≥ G . Hence by (1.2.46) we see that V ≤ F . To showthat V ≥ F consider the stopping time

τDε = inf n ≥ 0 : F (Xn) ≤ G(Xn)+ε (1.2.59)

where ε > 0 is given and fixed. Then by (1.2.52) we see that τDε < ∞ Px -a.s.for x ∈ E . Moreover, we claim that

(F (XτDε∧n)

)n≥0

is a martingale under Px

for x ∈ E . For this, note that the Markov property and (1.2.50) imply

Ex

(F (XτDε∧n) | Fn−1

)(1.2.60)

= Ex

(F (Xn)I(τDε≥ n) | Fn−1

)+ Ex

(F (XτDε

)I(τDε< n) | Fn−1

)= Ex

(F (Xn) | Fn−1

)I(τDε≥ n) + Ex

(∑n−1k=0F (Xk)I(τDε= k) | Fn−1

)= EXn−1

(F (X1)

)I(τDε ≥ n) +

∑n−1k=0F (Xk) I(τDε = k)

= TF (Xn−1) I(τDε ≥ n) + F (XτDε) I(τDε< n)

= F (Xn−1) I(τDε≥ n) + F (XτDε) I(τDε< n)

= F (XτDε∧(n−1)) I(τDε≥ n) + F (XτDε∧(n−1)) I(τDε< n)

= F (XτDε∧(n−1))

for all n ≥ 1 and x ∈ E proving the claim. Hence

Ex

(F (XτDε∧n)

)= F (x) (1.2.61)

for all n ≥ 0 and x ∈ E . Next note that

Ex

(F (XτDε∧n)

)= Ex

(F (XτDε

) I(τDε ≤ n))

+ Ex

(F (Xn) I(τDε > n)

)(1.2.62)

for all n ≥ 0 . Letting n → ∞ , using (1.2.51) and (1.2.1) with F ≥ G , we get

Ex

(F (XτDε

))

= F (x) (1.2.63)

for all x ∈ E . This fact is of independent interest.


Finally, since V is superharmonic, we find using (1.2.63) that

V (x) ≥ ExV (XτDε) ≥ ExG(XτDε

) ≥ ExF (XτDε) − ε = F (x) − ε (1.2.64)

for all ε > 0 and x ∈ E . Letting ε ↓ 0 we get V ≥ F as needed and the proofis complete.

11. Given α ∈ (0, 1] and (bounded) measurable functions g : E → R andc : E → R+ , consider the optimal stopping problem

V (x) = supτ

Ex

(ατg(Xτ ) −

τ∑k=1

αk−1c(Xk−1))

(1.2.65)

where τ is a stopping time of X and Px(X0 = x) = 1 .

The value c(x) is interpreted as the cost of making the next observation ofX when X equals x . The sum in (1.2.65) by definition equals 0 when τ equals0.

The problem formulation (1.2.65) goes back to a problem formulation dueto Bolza in classic calculus of variation (a more detailed discussion will be givenin Chapter III below). Let us briefly indicate how the problem (1.2.65) can bereduced to the setting of Theorem 1.11 above.

For this, let X = (Xn)n≥0 denote the Markov chain X killed at rate α . Itmeans that the transition operator of X is given by

TF (x) = α TF (x) (1.2.66)

for x ∈ E whenever F (X1) ∈ L1(Px) . The problem (1.2.65) then reads

V (x) = supτ

Ex

(g(Xτ ) −

τ∑k=1

c(Xk−1))

(1.2.65′)

where τ is a stopping time of X and Px(X0 = x) = 1 .

Introduce the sequence

In = a +n∑

k=1

c(Xk−1) (1.2.67)

for n ≥ 1 with I0 = a in R . Then Zn = (Xn, In) defines a Markov chain forn ≥ 0 with Z0 = (X0, I0) = (x, a) under Px so that we may write Px,a instead ofPx . (The latter can be justified rigorously by passage to the canonical probabilityspace.) The transition operator of Z = (X, I ) equals

T eZ F (x, a) = Ex,aF (X1, I1) (1.2.68)

for (x, a) ∈ E × R whenever F (X1, I1) ∈ L1(Px,a) .


The problem (1.2.65′) may now be rewritten as follows:

W (x, a) = supτ

Ex,aG(Zτ ) (1.2.65′′)

where we setG(z) = g(x) − a (1.2.69)

for z = (x, a) ∈ E×R . Obviously by subtracting a on both sides of (1.2.65′) weset that

W (x, a) = V (x) − a (1.2.70)

for all (x, a) ∈ E × R .

The problem (1.2.65′′) is of the same type as the problem (1.2.35) aboveand thus Theorem 1.11 is applicable. To write down (1.2.42) more explicitly notethat

T eZ W (x, a) = Ex,aW (X1, I1) = Ex,a

(V (X1) − I1

)(1.2.71)

= ExV (X1) − a − c(x) = αTV (x) − a − c(x)

so that (1.2.42) reads

V (x) − a = max(g(x)− a , αTV (x)− a− c(x)

)(1.2.72)

where we used (1.2.70), (1.2.69) and (1.2.71). Clearly a can be removed from(1.2.72) showing finally that the Wald–Bellman equation (1.2.42) takes the follow-ing form:

V (x) = max(g(x) , αTV (x) − c(x)

)(1.2.73)

for x ∈ E . Note also that (1.2.39) takes the following form:

αTF (x) − c(x) ≤ F (x) (1.2.74)

for x ∈ E . Thus F satisfies (1.2.74) if and only if (x, a) → F (x) − a is super-harmonic relative to the Markov chain Z = (X, I) . Having (1.2.73) and (1.2.74)set out explicitly the remaining statements of Theorem 1.11 and Corollary 1.12are directly applicable and we shall omit further details. It may be noted abovethat L = T − I is the generator of the Markov chain X . More general problemsof this type (involving also the maximum functional) will be discussed in ChapterIII below. We will conclude this section by giving an illustrative example.

12. The following example illustrates general results of optimal stopping the-ory for Markov chains when applied to a nontrivial problem in order to determinethe value function and an optimal Markov time (in the class M ).

Example 1.14. Let ξ, ξ1, ξ2, . . . be independent and identically distributed randomvariables, defined on a probability space (Ω,F , P) , with Eξ < 0 . Put S0 = 0 ,


Sn = ξ1 + · · · + ξn for n ≥ 1 ; X0 = x , Xn = x + Sn for n ≥ 1 , and M =supn≥0 Sn . Let Px be the probability distribution of the sequence (Xn)n≥0 withX0 = x from R . It is clear that the sequence (Xn)n≥0 is a Markov chain startedat x .

For any n ≥ 1 define the gain function Gn(x) = (x+)n where x+ =max(x, 0) for x ∈ R , and let

Vn(x) = supτ∈M

ExGn(Xτ ) (1.2.75)

where the supremum is taken over the class M of all Markov (stopping) times τsatisfying Px(τ < ∞) = 1 for all x ∈ R . Let us also denote

Vn(x) = supτ∈M

ExGn(Xτ )I(τ < ∞) (1.2.76)

where the supremum is taken over the class M of all Markov times.

The problem of finding the value functions Vn(x) and Vn(x) is of interestfor the theory of American options because these functions represent arbitrage-free(fair, rational) prices of “Power options” under the assumption that any exercisetime τ belongs to the class M or M respectively. In the present case we haveVn(x) = Vn(x) for n ≥ 1 and x ∈ R , and it will be clear from what follows belowthat an optimal Markov time exists in the class M (but does not belong to theclass M of stopping times).

We follow [144] where the authors solved the formulated problems (see also[119]). First of all let us introduce the notion of the Appell polynomial which willbe used in the formulation of the basic results.

Let η = η(ω) be a random variable with Eeλ|η| < ∞ for some λ > 0 .Consider the Esscher transform

x eux

Eeuη|u| ≤ λ, x ∈ R, (1.2.77)

and the decompositioneux

Eeuη=

∞∑k=0

uk

k!Q

(η)k (x). (1.2.78)

Polynomials Q(η)k (x) are called the Appell polynomials for the random variable

η . (If E|η|n < ∞ for some n ≥ 1 then the polynomials Q(η)k (x) are uniquely

defined for all k ≤ n .)

The polynomials Q(η)k (x) can be expressed through the semi-invariants κ1 ,

κ2, . . . of the random variable η . For example,Q

(η)0 (x) = 1, Q

(η)2 (x) = (x−κ1)2 − κ2,

Q(η)1 (x) = x − κ1, Q

(η)3 (x) = (x−κ1)3 − 3κ2(x−κ1) − κ3,

. . . (1.2.79)


where (as is well known) the semi-invariants κ1, κ2, . . . are expressed via themoments µ1, µ2, . . . of η :

κ1 = µ1, κ2 = µ2 − µ21, κ3 = 2µ3

1 − 3µ1µ2 + µ3, . . . . (1.2.80)

Let us also mention the following property of the Appell polynomials: ifE |η|n < ∞ then for k ≤ n we have

d

dxQ

(η)k (x) = kQ

(η)k−1(x), (1.2.81)

EQ(η)k (x+η) = xk. (1.2.82)

For simplicity of notation we will use Qk(s) to denote the polynomialsQ

(M)k (x) for the random variable M = supn≥0 Sn . Every polynomial Qk(x) has

a unique positive root a∗k . Moreover, Qk(x) ≤ 0 for 0 ≤ x < a∗

k and Qk(x)increases for x ≥ a∗

k .

In accordance with the characteristic property (1.2.46) recall that the valuefunction Vn(x) is the smallest superharmonic (excessive) function which domi-nates the gain function Gn(x) on R . Thus, one method to find Vn(x) is to searchfor the smallest excessive majorant of the function Gn(x) . In [144] this methodis realized as follows.

For every a ≥ 0 introduce the Markov time

τa = infn ≥ 0 : Xn ≥ a (1.2.83)

and for each n ≥ 1 consider the new optimal stopping problem:

V (x) = supa≥0

ExGn(Xτa)I(τa < ∞). (1.2.84)

It is clear that Gn(Xτa) = (X+τa

)n = Xnτa

(on the set τa < ∞ ). Hence

V (x) = supa≥0

ExXnτa

I(τa < ∞). (1.2.85)

The identity (1.2.82) prompts that the following property should be valid: ifE |M |n < ∞ then

EQn(x+M)I(x+M ≥ a) = ExXnτa

I(τa < ∞). (1.2.86)

This formula and properties of the Appell polynomials imply that

V (x) = supa≥0

EQn(x+M)I(x+M ≥ a) = EQn(x+M)I(x+M ≥ a∗n). (1.2.87)

From this we see that τa∗n

is an optimal Markov time for the problem (1.2.84).


It is clear that Vn(x) ≥ Vn(x) . From (1.2.87) and properties of the Appellpolynomials we obtain that Vn(x) is an excessive majorant of the gain function( Vn(x) ≥ ExVn(X1) and Vn(x) ≥ Gn(x) for x ∈ R ). But Vn(x) is the smallestexcessive majorant of Gn(x) . Thus Vn(x) ≤ Vn(x) .

On the whole we obtain the following result (for further details see [144]):

Suppose that E (ξ+)n+1 < ∞ and a∗n is the largest root of the equation

Qn(x) = 0 for n ≥ 1 fixed. Denote τ∗n = inf k ≥ 0 : Xk ≥ a∗

n . Then theMarkov time τ∗

n is optimal:

Vn(x) = supτ∈M

Ex(X+τ )nI(τ < ∞) = Ex

(X+

τ∗n

)nI(τ < ∞). (1.2.88)

Moreover,Vn(x) = E Qn(x+M)I(x+M ≥ a∗

n). (1.2.89)

Remark 1.15. In the cases n = 1 and n = 2 we have

a∗1 = EM and a∗

2 = EM +√

DM. (1.2.90)

Remark 1.16. If P(ξ = 1) = p , P(ξ = −1) = q and p < q , then M := supn≥0 Sn

(with S0 = 0 and Sn = ξ1 + · · · + ξn ) has geometric distribution:

P(M ≥ k) =(p

q

)k(1.2.91)

for k ≥ 0 . HenceEM =

q

q − p. (1.2.92)

2. Continuous time

The aim of the present section is to exhibit basic results of optimal stopping in thecase of continuous time. We first consider a martingale approach (cf. Subsection 1.1above). This is then followed by a Markovian approach (cf. Subsection 1.2 above).

2.1. Martingale approach

1. Let G = (Gt)t≥0 be a stochastic process defined on a filtered probability space(Ω,F , (Ft)t≥0, P) . We interpret Gt as the gain if the observation of G is stoppedat time t . It is assumed that G is adapted to the filtration (Ft)t≥0 in the sensethat each Gt is Ft -measurable. Recall that each Ft is a σ -algebra of subsetsof Ω such that Fs ⊆ Ft ⊆ F for s ≤ t . Typically (Ft)t≥0 coincides with thenatural filtration (FG

t )t≥0 but generally may also be larger. We interpret Ft asthe information available up to time t . All our decisions in regard to optimal

Section 2. Continuous time 27

stopping at time t must be based on this information only (no anticipation isallowed).

The following definition formalizes the previous requirement and plays a keyrole in the study of optimal stopping (cf. Definition 1.1).

Definition 2.1. A random variable τ : Ω → [0,∞] is called a Markov time ifτ ≤ t ∈ Ft for all t ≥ 0 . A Markov time is called a stopping time if τ < ∞P-a.s.

In the sequel we will only consider stopping times. We refer to Subsection1.1 above for other similar comments which translate to the present setting ofcontinuous time without major changes.

2. We will assume that the process G is right-continuous and left-continuousover stopping times (if τn and τ are stopping times such that τn ↑ τ as n → ∞then Gτn → Gτ P-a.s. as n → ∞ ). We will also assume that the followingcondition is satisfied (with GT = 0 when T = ∞ ):

E(

sup0≤t≤T

|Gt|)

< ∞. (2.1.1)

Just as in the case of discrete time (Subsection 1.1) here too it is possible togo beyond this condition in both theory and applications of optimal stopping,however, none of the conclusions will essentially be different and we thus workwith (2.1.1) throughout.

In order to invoke a theorem on the existence of a right-continuous modifi-cation of a given supermartingale, we will assume in the sequel that the filtration(Ft)t≥0 is right-continuous and that each Ft contains all P -null sets from F .This is a technical requirement and its enforcement has no significant impact on in-terpretations of the optimal stopping problem under consideration and its solutionto be presented.

3. We consider the optimal stopping problem

V Tt = sup

t≤τ≤TEGτ (2.1.2)

where τ is a stopping time and 0 ≤ t ≤ T . In (2.1.2) we admit that T can be+∞ as well. In this case, however, we assume that the supremum is still takenover stopping times τ , i.e. over Markov times τ satisfying t ≤ τ < ∞ . In thiscase we will set Vt = V ∞

t for t ≥ 0 . Moreover, for certain reasons of conveniencewe will also drop T from V T

t in (2.1.1) even if the horizon T is finite.

4. By analogy with the results of Subsection 1.1 above (discrete time case)there are two possible ways to tackle the problem (2.1.2). The first method consistsof replacing the time interval [0, T ] by sets Dn = tn0 , tn1 , . . . , tnn where Dn ↑ D


as n → ∞ and D is a (countable) dense subset of [0, T ] , applying the results ofSubsection 1.1 (the method of backward induction) to each Gn = (Gtn

i)0≤i≤n , and

then passing to the limit as n → ∞ . In this context it is useful to know that eachstopping time τ can be obtained as a decreasing limit of the discrete stoppingtimes τn =

∑ni=1 tni I(tni−1 ≤ τ < tni ) as n → ∞ . The methodology described

becomes useful for getting numerical approximations for the solution but we willomit further details in this direction. The second method aims directly to extendthe method of essential supremum in Subsection 1.1 above from the discrete timecase to the continuous time case. This programme will now be addressed.

5. Since there is no essential difference in the treatment of either finite orinfinite horizon T , we will treat both cases at the same time by setting

Vt = V Tt (2.1.3)

for simplicity of notation.

To solve the problem (2.1.2) we will (by analogy with the results of Subsection1.1) consider the process S = (St)t≥0 defined as follows:

St = esssupτ≥t

E (Gτ |Ft) (2.1.4)

where τ is a stopping time. In the case of a finite horizon T we also requirein (2.1.4) that τ is smaller than or equal to T . We will see in the proof ofTheorem 2.2 below that there is no restriction to assume that the process S isright-continuous. The process S is often referred to as the Snell envelope of G .

For the same reasons we will consider the following stopping time:

τt = inf s ≥ t : Ss = Gs (2.1.5)

for t ≥ 0 where inf ∅ = ∞ by definition. In the case of a finite horizon T wealso require in (2.1.5) that s is smaller than or equal to T .

Regarding the initial part of Theorems 1.2 and 1.4 (the Wald–Bellman equa-tion) one should observe that Theorem 2.2 below implies that

St ≥ max(Gt, E (Ss | Ft)

)(2.1.6)

for s ≥ t . The reverse inequality, however, is not true in general. The reasonroughly speaking lies in the fact that, unlike in discrete time, in continuous timethere is no smallest unit of time, so that no matter how close s to t is (whenstrictly larger) the values Su can still wander far away from St when u ∈ (t, s) .Note however that Theorem 2.2 below implies that the following refinement of theWald–Bellman equation still holds:

St = max(Gt, E (Sσ∧τt | Ft)

)(2.1.7)


for every stopping time σ larger than or equal to t (note that σ can also beidentically equal to any s ≥ t ) where τt is given in (2.1.5) above.

The other three parts of Theorems 1.2 and 1.4 (pages 3 and 8) extend tothe present case with no significant change. Thus the first part of the followingtheorem shows that (Ss)s≥t and τt solve the problem in a stochastic sense. Thesecond part of the theorem shows that this leads to a solution of the initial problem(2.1.2). The third part of the theorem provides a supermartingale characterizationof the solution.

Theorem 2.2. Consider the optimal stopping problem (2.1.2) upon assuming thatthe condition (2.1.1) holds. Assume moreover when required below that

P(τt < ∞) = 1 (2.1.8)

where t ≥ 0 . (Note that this condition is automatically satisfied when the horizonT is finite.) Then for all t ≥ 0 we have:

St ≥ E (Gτ | Ft) for each τ ∈ Mt, (2.1.9)St = E (Gτt | Ft) (2.1.10)

where Mt denotes the family of all stopping times τ satisfying τ ≥ t (being alsosmaller than or equal to T when the latter is finite). Moreover, if t ≥ 0 is givenand fixed, then we have:

The stopping time τt is optimal in (2.1.2). (2.1.11)If τ∗ is an optimal stopping time in (2.1.2) then τt ≤ τ∗ P-a.s. (2.1.12)The process (Ss)s≥t is the smallest right-continuous supermartingalewhich dominates (Gs)s≥t.

(2.1.13)

The stopped process (Ss∧τt)s≥t is a right-continuous martingale. (2.1.14)

Finally, if the condition (2.1.8) fails so that P(τt = ∞) > 0 , then there is nooptimal stopping time (with probability 1) in (2.1.2).

Proof. 1. Let us first show that S = (St)t≥0 defined by (2.1.4) above is asupermartingale. For this, fix t ≥ 0 and let us show that the family

E (Gτ | Ft) :

τ ∈ Mt

is upwards directed in the sense that (1.1.25) is satisfied. Indeed, note

that if σ1 and σ2 are from Mt and we set σ3 = σ1IA + σ2IAc where A =E (Gσ1 | Ft) ≥ E (Gσ2 | Ft)

, then σ3 belongs to Mt and we have

E (Gσ3 | Ft) = E (Gσ1IA + Gσ2IAc | Ft) (2.1.15)= IAE (Gσ1 | Ft) + IAcE (Gσ2 | Ft)= E (Gσ1 | Ft) ∨ E (Gσ2 | Ft)

implying (1.1.25) as claimed. Hence by (1.1.26) there exists a sequence σk : k ≥1 in Mt such that

esssupτ∈Mt

E (Gτ | Ft) = limk→∞

E (Gσk| Ft) (2.1.16)


where E (Gσ1 | Ft) ≤ E (Gσ2 | Ft) ≤ · · · P-a.s. Since the left-hand side in (2.1.16)equals St , by the conditional monotone convergence theorem using (2.1.1) above,we find for any s ∈ [0, t] that

E (St | Fs) = E(

limk→∞

E (Gσk| Ft) | Fs

)(2.1.17)

= limk→∞

E(E (Gσk

| Ft) | Fs

)= lim

k→∞E (Gσk

| Fs) ≤ Ss

where the final inequality follows by the definition of Ss given in (2.1.4) above.This shows that (St)t≥0 is a supermartingale as claimed. Note also that (2.1.4)and (2.1.16) using the monotone convergence theorem and (2.1.1) imply that

ESt = supτ≥t

EGτ (2.1.18)

where τ is a stopping time and t ≥ 0 .

2. Let us next show that the supermartingale S admits a right-continuousmodification S = (St)t≥0 . A well-known result in martingale theory (see e.g.[134]) states that the latter is possible to achieve if and only if

t → ESt is right-continuous on R+ . (2.1.19)

To verify (2.1.19) note that by the supermartingale property of S we haveESt ≥ · · · ≥ ESt2 ≥ ESt1 so that L := limn→∞ EStn exists and ESt ≥ Lwhenever tn ↓ t as n → ∞ is given and fixed. To prove the reverse inequality, fixε > 0 and by means of (2.1.18) choose σ ∈ Mt such that

EGσ ≥ ESt − ε. (2.1.20)

Fix δ > 0 and note that there is no restriction to assume that tn ∈ [t, t + δ] forall n ≥ 1 . Define a stopping time σn by setting

σn =

σ if σ > tn,

t + δ if σ ≤ tn(2.1.21)

for n ≥ 1 . Then for all n ≥ 1 we have

EGσn = EGσI(σ > tn) + EGt+δI(σ ≤ tn) ≤ EStn (2.1.22)

since σn ∈ Mtn and (2.1.18) holds. Letting n → ∞ in (2.1.22) and using (2.1.1)we get

EGσI(σ > t) + EGt+δI(σ = t) ≤ L (2.1.23)

for all δ > 0 . Letting now δ ↓ 0 and using that G is right-continuous we finallyobtain

EGσI(σ > t) + EGtI(σ = t) = EGσ ≤ L. (2.1.24)


From (2.1.20) and (2.1.24) we see that L ≥ ESt−ε for all ε > 0 . Hence L ≥ ESt

and thus L = ESt showing that (2.1.19) holds. It follows that S admits a right-continuous modification S = (St)t≥0 which we also denote by S throughout.

3. Let us show that (2.1.13) holds. For this, let S = (Ss)s≥t be anotherright-continuous supermartingale which dominates G = (Gs)s≥t . Then by theoptional sampling theorem (page 60) using (2.1.1) above we have

Ss ≥ E (Sτ | Fs) ≥ E (Gτ | Fs) (2.1.25)

for all τ ∈ Ms when s ≥ t . Hence by the definition of Ss given in (2.1.4) abovewe find that Ss ≤ Ss P-a.s. for all s ≥ t . By the right-continuity of S and Sthis further implies that P(Ss ≤ Ss for all s ≥ t) = 1 as claimed.

4. Noticing that (2.1.9) follows at once from (2.1.4) above, let us now showthat (2.1.10) holds. For this, let us first consider the case when Gt ≥ 0 for allt ≥ 0 .

For each λ ∈ (0, 1) introduce the stopping time

τλt = inf s ≥ t : λSs ≤ Gs (2.1.26)

where t ≥ 0 is given and fixed. For further reference note that by the right-continuity of S and G we have:

λSτλt≤ Gτλ

t, (2.1.27)

τλt+ = τλ

t (2.1.28)

for all λ ∈ (0, 1) . In exactly the same way we find:

Sτt = Gτt , (2.1.29)τt+ = τt (2.1.30)

for τt defined in (2.1.5) above.

Next note that the optional sampling theorem (page 60) using (2.1.1) aboveimplies

St ≥ E (Sτλt| Ft) (2.1.31)

since τλt is a stopping time greater than or equal to t . To prove the reverse

inequalitySt ≤ E (Sτλ

t| Ft) (2.1.32)

consider the processRt = E (Sτλ

t| Ft) (2.1.33)

for t ≥ 0 . We claim that R = (Rt)t≥0 is a supermartingale. Indeed, for s < t wehave

E (Rt | Fs) = E(E (Sτλ

t| Ft) | Fs

)= E (Sτλ

t| Fs) ≤ E (Sτλ

s| Fs) = Rs (2.1.34)


where the inequality follows by the optional sampling theorem (page 60) using(2.1.1) above since τλ

t ≥ τλs when s < t . This shows that R is a supermartingale

as claimed. Hence ERt+h increases when h decreases and limh↓0 ERt+h ≤ ERt .On the other hand, note by Fatou’s lemma using (2.1.1) above that

limh↓0

ERt+h = limh↓0

ESτλt+h

≥ ESτλt

= ERt (2.1.35)

where we also use (2.1.28) above together with the facts that τλt+h decreases

when h decreases and S is right-continuous. This shows that t → ERt is right-continuous on R+ and hence R admits a right-continuous modification which wealso denote by R in the sequel. It follows that there is no restriction to assumethat the supermartingale R is right-continuous.

To prove (2.1.32) i.e. that St ≤ Rt P-a.s. consider the right-continuoussupermartingale defined as follows:

Lt = λSt + (1 − λ)Rt (2.1.36)

for t ≥ 0 . We then claim that

Lt ≥ Gt P-a.s. (2.1.37)

for all t ≥ 0 . Indeed, we have

Lt = λSt + (1−λ)Rt = λSt + (1−λ)RtI(τλt = t) (2.1.38)

+ (1−λ)RtI(τλt > t)

= λSt + (1 − λ)E(StI(τλ

t = t) | Ft

)+ (1−λ)RtI(τλ

t > t)

= λStI(τλt = t) + (1 − λ)StI(τλ

t = t) + λStI(τλt > t)

+ (1−λ)RtI(τλt > t)

≥ StI(τλt = t) + λStI(τλ

t > t) ≥ GtI(τλt = t) + GtI(τλ

t > t) = Gt

where in the second last inequality we used that Rt ≥ 0 and in the last inequal-ity we used the definition of τλ

t given in (2.1.26) above. Thus (2.1.37) holds asclaimed. Finally, since S is the smallest right-continuous supermartingale whichdominates G , we see that (2.1.37) implies that

St ≤ Lt P-a.s. (2.1.39)

from where by (2.1.36) we conclude that St ≤ Rt P-a.s. Thus (2.1.32) holds asclaimed. Combining (2.1.31) and (2.1.32) we get

St = E (Sτλt| Ft) (2.1.40)

for all λ ∈ (0, 1) . From (2.1.40) and (2.1.27) we find

St ≤ 1λ

E (Gτλt| Ft) (2.1.41)


for all λ ∈ (0, 1) . Letting λ ↑ 1 , using the conditional Fatou’s lemma and (2.1.1)above together with the fact that G is left-continuous over stopping times, weobtain

St ≤ E (Gτ1t| Ft) (2.1.42)

where τ1t is a stopping time given by

τ1t = lim

λ↑1τλt . (2.1.43)

(Note that τλt increases when λ increases.) Since by (2.1.4) we know that the

reverse inequality in (2.1.42) is always fulfilled, we may conclude that

St = E (Gτ1t| Ft) (2.1.44)

for all t ≥ 0 . Thus to complete the proof of (2.1.10) it is enough to verify that

τ1t = τt (2.1.45)

where τt is defined in (2.1.5) above. For this, note first that τλt ≤ τt for all

λ ∈ (0, 1) so that τ1t ≤ τt . On the other hand, if τt(ω) > t (the case τt(ω) = t

being obvious) then there exists ε > 0 such that τt(ω) − ε > t and Sτt(ω)−ε >Gτt(ω)−ε ≥ 0 . Hence one can find λ ∈ (0, 1) (close enough to 1 ) such thatλSτt(ω)−ε > Gτt(ω)−ε showing that τλ

t (ω) ≥ τt(ω) − ε . Letting first λ ↑ 1 andthen ε ↓ 0 we conclude that τ1

t ≥ τt . Hence (2.1.45) holds as claimed and theproof of (2.1.10) is complete in the case when Gt ≥ 0 for all t ≥ 0 .

5. In the case of general G satisfying (2.1.1) we can set

H = inft≥0

Gt (2.1.46)

and introduce the right-continuous martingale

Mt = E (H | Ft) (2.1.47)

for t ≥ 0 so as to replace the initial gain process G by a new gain processG = (Gt)t≥0 defined by

Gt = Gt − Mt (2.1.48)

for t ≥ 0 . Note that G need not satisfy (2.1.1) due to the existence of M , but M

itself is uniformly integrable since H ∈ L1(P) . Similarly, G is right-continuousand not necessarily left-continuous over stopping times due to the existence of M ,but M itself is a (uniformly integrable) martingale so that the optional samplingtheorem (page 60) is applicable. Finally, it is clear that Gt ≥ 0 and the optionalsampling theorem implies that

St = esssupτ∈Mt

E(Gτ | Ft

)= esssup

τ∈Mt

E (Gτ − Mτ | Ft) = St − Mt (2.1.49)


for all t ≥ 0 . A closer inspection based on the new properties of G displayedabove instead of the old ones imposed on G when Gt ≥ 0 for all t ≥ 0 showsthat the proof above can be applied to G and S to yield the same conclusionsimplying (2.1.10) in the general case.

6. Noticing that (2.1.11) follows by taking expectation in (2.1.10) and using(2.1.18), let us now show that (2.1.12) holds. We claim that the optimality of τ∗implies that Sτ∗ = Gτ∗ P-a.s. Indeed, if this would not be the case then wewould have Sτ∗ ≥ Gτ∗ P-a.s. with P(Sτ∗ > Gτ∗) > 0 . It would then followthat EGτ∗ < ESτ∗ ≤ ESt = Vt where the second inequality follows by theoptional sampling theorem (page 60) and the supermartingale property of (Ss)s≥t

using (2.1.1) above, while the final equality is stated in (2.1.18) above. The strictinequality, however, contradicts the fact that τ∗ is optimal. Hence Sτ∗ = Gτ∗P-a.s. as claimed and the fact that τt ≤ τ∗ P-a.s. follows from the definition(2.1.5) above.

7. To verify the martingale property (2.1.14) it is enough to prove that

ESσ∧τt = ESt (2.1.50)

for all (bounded) stopping times σ greater than or equal to t . For this, note firstthat the optional sampling theorem (page 60) using (2.1.1) above implies

ESσ∧τt ≤ ESt. (2.1.51)

On the other hand, from (2.1.10) and (2.1.29) we likewise see that

ESt = EGτt = ESτt ≤ ESσ∧τt . (2.1.52)

Combining (2.1.51) and (2.1.52) we see that (2.1.50) holds and thus (Ss∧τt)s≥t

is a martingale (right-continuous by (2.1.13) above). This completes the proof of(2.1.14).

Finally, note that the final claim follows directly from (2.1.12). This completesthe proof of the theorem.

2.2. Markovian approach

In this subsection we will present basic results of optimal stopping when the timeis continuous and the process is Markovian. (Basic definitions and properties ofsuch processes are given in Subsection 4.3.)

1. Throughout we will consider a strong Markov process X = (Xt)t≥0 de-fined on a filtered probability space (Ω,F , (Ft)t≥0, Px) and taking values in ameasurable space (E,B) where for simplicity we will assume that E = Rd forsome d ≥ 1 and B = B(Rd) is the Borel σ -algebra on Rd . It is assumed that theprocess X starts at x under Px for x ∈ E and that the sample paths of X are


right-continuous and left-continuous over stopping times (if τn ↑ τ are stoppingtimes, then Xτn → Xτ Px -a.s. as n → ∞ ). It is also assumed that the filtration(Ft)t≥0 is right-continuous (implying that the first entry times to open and closedsets are stopping times). In addition, it is assumed that the mapping x → Px(F ) ismeasurable for each F ∈ F . It follows that the mapping x → Ex(Z) is measurablefor each (bounded or non-negative) random variable Z . Finally, without loss ofgenerality we will assume that (Ω,F) equals the canonical space (E[0,∞),B[0,∞))so that the shift operator θt : Ω → Ω is well defined by θt(ω)(s) = ω(t+s) forω = (ω(s))s≥0 ∈ Ω and t, s ≥ 0 .

2. Given a measurable function G : E → R satisfying the following condition(with G(XT ) = 0 if T = ∞ ):

Ex

(sup

0≤t≤T|G(Xt)|

)< ∞ (2.2.1)

for all x ∈ E , we consider the optimal stopping problem

V (x) = sup0≤τ≤T

ExG(Xτ ) (2.2.2)

where x ∈ E and the supremum is taken over stopping times τ of X . Thelatter means that τ is a stopping time with respect to the natural filtration ofX given by FX

t = σ(Xs : 0 ≤ s ≤ t) for t ≥ 0 . Since the same results remainvalid if we take the supremum in (2.2.2) over stopping times τ with respect to(Ft)t≥0 , and this assumption makes certain conclusions more elegant (the optimalstopping time will be attained), we will assume in the sequel that the supremumin (2.2.2) is taken over this larger class of stopping times. Note also that in (2.2.2)we admit that T can be ∞ as well (infinite horizon). In this case, however, westill assume that the supremum is taken over stopping times τ , i.e. over Markovtimes satisfying 0 ≤ τ < ∞ . In this way any specification of G(X∞) becomesirrelevant for the problem (2.2.2).

3. Recall that V is called the value function and G is called the gain func-tion. To solve the optimal stopping problem (2.2.2) means two things. Firstly, weneed to exhibit an optimal stopping time, i.e. a stopping time τ∗ at which thesupremum is attained. Secondly, we need to compute the value V (x) for x ∈ Eas explicitly as possible.

Let us briefly comment on what one expects to be a solution to the problem(2.2.2) (recall also Subsection 1.2 above). For this note that being Markovianmeans that the process X always starts afresh. Thus following the sample patht → Xt(ω) for ω ∈ Ω given and fixed and evaluating G(Xt(ω)) it is naturallyexpected that at each time t we shall be able optimally to decide either to continuewith the observation or to stop it. In this way the state space E naturally splitsinto the continuation set C and the stopping set D = E \ C . It follows that assoon as the observed value Xt(ω) enters D , the observation should be stopped


and an optimal stopping time is obtained. The central question thus arises as howto determine the sets C and D . (Note that the same arguments also hold in thediscrete-time case of Subsection 1.2 above.)

In comparison with the general optimal stopping problem of Subsection 2.1above, it may be noted that the description of the optimal stopping time justgiven does not involve any probabilistic construction (of a new stochastic processS = (St)t≥0 ) but is purely deterministic (obtained by splitting E into two disjointsubsets defined by the deterministic functions G and V ).

4. In the sequel we will treat the finite horizon formulation ( T < ∞ ) andthe infinite horizon formulation ( T = ∞ ) of the optimal stopping problem (2.2.2)at the same time. It should be noted that in the former case ( T < ∞ ) we need toreplace the process Xt by the process Zt = (t, Xt) for t ≥ 0 so that the problemreads

V (t, x) = sup0≤τ≤T−t

Et,xG(t+τ, Xt+τ ) (2.2.2′)

where the “rest of time” T − t changes when the initial state (t, x) ∈ [0, T ] × Echanges in its first argument. It turns out, however, that no argument below is moreseriously affected by this change, and the results obtained for the problem (2.2.2)with T = ∞ will automatically hold for the problem (2.2.2′) if we simply thinkof X to be Z (with a new “two-dimensional” state space E equal to R+ × E ).Moreover, it may be noted in (2.2.2′) that at time T we have the “terminal”condition V (T, x) = G(T, x) for all x ∈ E so that the first entry time of Z tothe stopping set D , denoted below by τD , will always be smaller than or equalto T and thus finite. This works to a technical advantage of the finite horizonformulation (2.2.2′) over the infinite horizon formulation (2.2.2) (where insteadof the condition V (T, x) = G(T, x) for all x ∈ E another “boundary conditionat infinity” such as (2.2.52) may hold).

5. Consider the optimal stopping problem (2.2.2) when T = ∞ . Recall that(2.2.2) reads as follows:

V (x) = supτ

ExG(Xτ ) (2.2.3)

where τ is a stopping time (with respect to (Ft)t≥0 ) and Px(X0 = x) = 1 forx ∈ E . Introduce the continuation set

C = x ∈ E : V (x)>G(x) (2.2.4)

and the stopping setD = x ∈ E : V (x) = G(x) (2.2.5)

Note that if V is lsc (lower semicontinuous) and G usc (upper semicontinuous)then C is open and D is closed. Introduce the first entry time τD of X into Dby setting

τD = inf t ≥ 0 : Xt ∈ D . (2.2.6)


Note that τD is a stopping (Markov) time with respect to (Ft)t≥0 when D isclosed since both X and (Ft)t≥0 are right-continuous.

6. The following definition plays a fundamental role in solving the optimalstopping problem (2.2.3).

Definition 2.3. A measurable function F : E → R is said to be superharmonic if

ExF (Xσ) ≤ F (x) (2.2.7)

for all stopping times σ and all x ∈ E .

It is assumed in (2.2.7) that the left-hand side is well defined (and finite) i.e.that F (Xσ) ∈ L1(Px) for all x ∈ E whenever σ is a stopping time. Moreover,it will be verified in the proof of Theorem 2.4 below that the following stochasticcharacterization of superharmonic functions holds (recall also (1.2.40)):

F is superharmonic if and only if (F (Xt))t≥0 is a right-continuous supermartingale under Px for every x ∈ E

(2.2.8)

whenever F is lsc and (F (Xt))t≥0 is uniformly integrable.

7. The following theorem presents necessary conditions for the existence ofan optimal stopping time.

Theorem 2.4. Let us assume that there exists an optimal stopping time τ∗ in(2.2.3), i.e. let

V (x) = ExG(Xτ∗) (2.2.9)

for all x ∈ E . Then we have:

The value function V is the smallest superharmonic functionwhich dominates the gain function G on E .

(2.2.10)

Let us in addition to (2.2.9) assume that V is lsc and G is usc. Then we have:

The stopping time τD satisfies τD ≤ τ∗ Px-a.s. for all x ∈ E andis optimal in (2.2.3).

(2.2.11)

The stopped process (V (Xt∧τD))t≥0 is a right-continuous martin-gale under Px for every x ∈ E.

(2.2.12)

Proof. (2.2.10): To show that V is superharmonic note that by the strong Markovproperty we have:

ExV (Xσ) = ExEXσG(Xτ∗) = ExEx

(G(Xτ∗) θσ | Fσ) (2.2.13)

= ExG(Xσ+τ∗θσ) ≤ supτ

ExG(Xτ ) = V (x)


for each stopping time σ and all x ∈ E . This establishes (2.2.7) and proves theinitial claim.

Let F be a superharmonic function which dominates G on E . Then wehave

ExG(Xτ ) ≤ ExF (Xτ ) ≤ F (x) (2.2.14)

for each stopping time τ and all x ∈ E . Taking the supremum over all τ in(2.2.14) we find that V (x) ≤ F (x) for all x ∈ E . Since V is superharmonicitself, this proves the final claim.

(2.2.11): We claim that V (Xτ∗) = G(Xτ∗) Px -a.s. for all x ∈ E . Indeed, ifPx

(V (Xτ∗) > G(Xτ∗)

)> 0 for some x ∈ E , then ExG(Xτ∗) < ExV (Xτ∗) ≤ V (x)

since V is superharmonic, leading to a contradiction with the fact that τ∗ isoptimal. From the identity just verified it follows that τD ≤ τ∗ Px -a.s. for allx ∈ E as claimed.

To make use of the previous inequality we may note that setting σ ≡ s in(2.2.7) and using the Markov property we get

V (Xt) ≥ EXtV (Xs) = Ex

(V (Xt+s) | Ft

)(2.2.15)

for all t, s ≥ 0 and all x ∈ E . This shows:

The process (V (Xt))t≥0 is a supermartingale under Px for each x ∈ E .(2.2.16)

Moreover, to indicate the argument as clearly as possible, let us for the momentassume that V is continuous. Then obviously it follows that (V (Xt))t≥0 is right-continuous. Thus, by the optional sampling theorem (page 60) using (2.2.1) above,we see that (2.2.7) extends as follows:

ExV (Xτ ) ≤ ExV (Xσ) (2.2.17)

for stopping times σ and τ such that σ ≤ τ Px -a.s. with x ∈ E . In particular,since τD ≤ τ∗ Px -a.s. by (2.2.17) we get

V (x) = ExG(Xτ∗) = ExV (Xτ∗) ≤ ExV (XτD ) = ExG(XτD ) ≤ V (x) (2.2.18)

for x ∈ E upon using that V (XτD ) = G(XτD ) since V is lsc and G is usc. Thisshows that τD is optimal if V is continuous. Finally, if V is only known to belsc, then by Proposition 2.5 below we know that (V (Xt))t≥0 is right-continuousPx -a.s. for each x ∈ E , and the proof can be completed as above. This shows thatτD is optimal if V is lsc as claimed.

(2.2.12): By the strong Markov property we have


Ex

(V (Xt∧τD) | Fs∧τD

)= Ex

(EXt∧τD

G(XτD ) | Fs∧τD

)(2.2.19)

= Ex

(Ex

(G(XτD ) θt∧τD | Ft∧τD

) | Fs∧τD

)= Ex

(Ex

(G(XτD ) | Ft∧τD

) | Fs∧τD

)= Ex

(G(XτD ) | Fs∧τD

)= EXs∧τD

(G(XτD )

)= V (Xs∧τD)

for all 0 ≤ s ≤ t and all x ∈ E proving the martingale property. The right-continuity of

(V (Xt∧τD)

)t≥0

follows from the right-continuity of (V (Xt))t≥0 andthe proof is complete.

The following fact was needed in the proof above to extend the result fromcontinuous to lsc V .

Proposition 2.5. If a superharmonic function F : E → R is lsc (lower semicontin-uous), then the supermartingale (F (Xt))t≥0 is right-continuous Px -a.s. for everyx ∈ E .

Proof. Firstly, we will show that

F (Xτ ) = limh↓0

F (Xτ+h) Px -a.s. (2.2.20)

for any given stopping time τ and x ∈ E . For this, note that the right-continuityof X and the ls-continuity of F , we get

F (Xτ ) ≤ lim infh↓0

F (Xτ+h) Px -a.s. (2.2.21)

To prove the reverse inequality we will first derive it for τ ≡ 0 , i.e. we have

lim suph↓0

F (Xh) ≤ F (x) Px -a.s. (2.2.22)

For this, note by Blumenthal’s 0-1 law (cf. page 97) that lim suph↓0 F (Xh) isequal Px -a.s. to a constant c ∈ R . Let us assume that c > F (x) . Then thereis ε > 0 such that c > F (x)+ε . Set Aε = y ∈ E : F (y) > F (x)+ε andconsider the stopping time τε = inf h ≥ 0 : Xh ∈ Aε . By definition of cand Aε we see that τε = 0 Px -a.s. Note however that Aε is open (since F islsc) and that we cannot claim a priori that Xτε , i.e. x , belongs to Aε as onewould like to reach a contradiction. For this reason choose an increasing sequenceof closed sets Kn for n ≥ 1 such that

⋃∞n=1 Kn = Aε . Consider the stopping

time τn = inf h ≥ 0 : Xh ∈ Kn for n ≥ 1 . Then τn ↓ τε as n → ∞ andsince Kn is closed we see that Xτn ∈ Kn for all n ≥ 1 . Hence Xτn ∈ Aε i.e.F (Xτn) > F (x) + ε for all n ≥ 1 . Using that F is superharmonic this implies

F (x) ≥ ExF (Xτn∧1) = ExF (Xτn)I(τn ≤ 1) + ExF (X1)I(τn > 1) (2.2.23)

≥ (F (x) + ε)P(τn ≤ 1) + ExF (X1)I(τn > 1) → F (x) + ε


as n → ∞ since τn ↓ 0 Px -a.s. as n → ∞ and F (X1) ∈ L1(Px) . As clearly(2.2.23) is impossible, we may conclude that (2.2.22) holds as claimed.

To treat the case of a general stopping time τ , take Ex on both sides of(2.2.22) and insert x = Xτ . This by the strong Markov property gives

F (Xτ ) ≥ EXτ

(lim sup

h↓0F (Xh)

)= Ex

(lim sup

h↓0F (Xh) θτ | Fτ

)(2.2.24)

= Ex

(lim sup

h↓0F (Xτ+h) | Fτ

)= lim sup

h↓0F (Xτ+h) Px-a.s.

since lim suph↓0 F (Xτ+h) is Fτ+ -measurable and Fτ = Fτ+ by the right-conti-nuity of (Ft)t≥0 . Combining (2.2.21) and (2.2.24) we get (2.2.20). In particular,taking τ ≡ t we see that

limh↓0

F (Xt+h) = F (Xt) Px -a.s. (2.2.25)

for all t ≥ 0 . Note that the Px -null set in (2.2.25) does depend on the given t .

Secondly, by means of (2.2.20) we will now show that a single Px -null set canbe selected so that the convergence relation in (2.2.25) holds on its complementsimultaneously for all t ≥ 0 . For this, set τ0 = 0 and define the stopping time

τn = inf t ≥ τn−1 : |F (Xt) − F (Xτn−1)| > ε/2 (2.2.26)

for n = 1, 2, . . . where ε > 0 is given and fixed. By (2.2.20) we see that foreach n ≥ 1 there is a Px -null set Nn such that τn > τn−1 on Ω \ Nn .Continuing the procedure (2.2.26) by transfinite induction over countable ordi-nals (there can be at most countably many disjoint intervals in R+ ) and callingthe union of the countably many Px -null set by Nε , it follows that for eachω ∈ Ω \Nε and each t ≥ 0 there is a countable ordinal α such that τα(ω) ≤ t <τα+1(ω) . Hence for every s ∈ [τα(ω), τα+1(ω)) we have |F (Xt(ω))−F (Xs(ω))| ≤|F (Xt(ω))−F (Xτα(ω))| + |F (Xs(ω))−F (Xα(ω))| ≤ ε/2 + ε/2 = ε . This showsthat lim sups↓t |F (Xt)−F (Xs)| ≤ ε on Ω \ Nε . Setting N =

⋃∞n=1 N1/n we see

that Px(N) = 0 and lims↓t F (Xs) = F (Xt) on Ω\N completing the proof.

Remark 2.6. The result and proof of Theorem 2.4 above extend in exactly thesame form (by slightly changing the notation only) to the finite horizon problem(2.2.2′) . We will omit further details in this direction.

8. The following theorem provides sufficient condition for the existence of anoptimal stopping time.

Theorem 2.7. Consider the optimal stopping problem (2.2.3) upon assuming thatthe condition (2.2.1) is satisfied. Let us assume that there exists the smallest su-perharmonic function V which dominates the gain function G on E . Let us in


addition assume that V is lsc and G is usc. Set D = x ∈ E : V (x) = G(x)and let τD be defined by (2.2.6) above. We then have:

If Px(τD < ∞) = 1 for all x ∈ E, then V = V and τD is optimalin (2.2.3).

(2.2.27)

If Px(τD < ∞) < 1 for some x ∈ E, then there is no optimalstopping time (with probability 1) in (2.2.3).

(2.2.28)

Proof. Since V is superharmonic, we have

ExG(Xτ ) ≤ ExV (Xτ ) ≤ V (x) (2.2.29)

for all stopping times τ and all x ∈ E . Taking the supremum in (2.2.17) over allτ we find that

G(x) ≤ V (x) ≤ V (x) (2.2.30)

for all x ∈ E . Assuming that Px(τD < ∞) = 1 for all x ∈ E , we will now presenttwo different proofs of the fact that V = V implying also that τD is optimal in(2.2.3).

First proof. Let us assume that G is bounded. With ε > 0 given and fixed,consider the sets:

Cε = x ∈ E : V (x) > G(x)+ε , (2.2.31)

Dε = x ∈ E : V (x) ≤ G(x)+ε . (2.2.32)

Since V is lsc and G is usc we see that Cε is open and Dε is closed. Moreover,it is clear that Cε ↑ C and Dε ↓ D as ε ↓ 0 where C and D are defined by(2.2.4) and (2.2.5) above respectively.

Define the stopping time

τDε = inf t ≥ 0 : Xt ∈ Dε. (2.2.33)

Since D ⊆ Dε and Px(τD < ∞) = 1 for all x ∈ E , we see that Px(τDε < ∞) = 1for all x ∈ E . The latter fact can also be derived directly (without assuming theformer fact) by showing that lim supt→∞ V (Xt) = lim supt→∞ G(Xt) Px -a.s. forall x ∈ E . This can be done in exactly the same way as in the first part of theproof of Theorem 1.13.

In order to show that

ExV(XτDε

)= V (x) (2.2.34)

for all x ∈ E , we will first show that

G(x) ≤ ExV(XτDε

)(2.2.35)


for all x ∈ E . For this, set

c = supx∈E

(G(x) − ExV (XτDε

))

(2.2.36)

and note thatG(x) ≤ c + ExV

(XτDε

)(2.2.37)

for all x ∈ E . (Observe that c is finite since G is bounded implying also thatV is bounded.)

Next by the strong Markov property we find

ExEXσ V(XτDε

)= ExEx

(V(XτDε

) θσ | Fσ

)(2.2.38)

= ExEx

(V(Xσ+τDεθσ

) | Fσ

)= ExV

(Xσ+τDεθσ

) ≤ ExV(XτDε

)

using that V is superharmonic and lsc (recall Proposition 2.5 above) and σ +τDε θσ ≥ τDε since τDε is the first entry time to a set. This shows that thefunction

x → ExV(XτDε

) is superharmonic (2.2.39)

from E to R . Hence the function of the right-hand side of (2.2.37) is also super-harmonic so that by the definition of V we can conclude that

V (x) ≤ c + ExV(XτDε

) (2.2.40)

for all x ∈ E .

Given 0 < δ ≤ ε choose xδ ∈ E such that

G(xδ) − ExδV(XτDε

) ≥ c − δ. (2.2.41)

Then by (2.2.40) and (2.2.41) we get

V (xδ) ≤ c + ExδV(XτDε

) ≤ G(xδ) + δ ≤ G(xδ) + ε. (2.2.42)

This shows that xδ ∈ Dε and thus τDε ≡ 0 under Pxδ. Inserting the latter

conclusion into (2.2.41) we get

c − δ ≤ G(xδ) − V (xδ) ≤ 0. (2.2.43)

Letting δ ↓ 0 we see that c ≤ 0 thus proving (2.2.35) as claimed. Using thedefinition of V and (2.2.39) we see that (2.2.34) follows directly from (2.2.35).

Finally, from (2.2.34) we get

V (x) = ExV(XτDε

) ≤ ExG(XτDε

)+ ε ≤ V (x) + ε (2.2.44)


for all x ∈ E upon using that V(XτDε

) ≤ G(XτDε

)+ ε since V is lsc and G

is usc. Letting ε ↓ 0 in (2.2.44) we see that V ≤ V and thus by (2.2.30) we canconclude that V = V . From (2.2.44) we thus also see that

V (x) ≤ ExG(XτDε

)+ ε (2.2.45)

for all x ∈ E .

Letting ε ↓ 0 and using that Dε ↓ D we see that τDε ↑ τ0 where τ0 is astopping time satisfying τ0 ≤ τD . Since V is lsc and G is usc it is easily seenfrom the definition of τDε that V

(XτDε

) ≤ G(XτDε

)+ ε for all ε > 0 . Letting

ε ↓ 0 and using that X is left-continuous over stopping times it follows thatV (Xτ0) ≤ G(Xτ0) since V is lsc and G is usc. This shows that V (Xτ0) = G(Xτ0)and therefore τD ≤ τ0 showing that τ0 = τD . Thus τDε ↑ τD as ε ↓ 0 .

Making use of the latter fact in (2.2.34) upon letting ε ↓ 0 and applyingFatou’s lemma, we get

V (x) ≤ lim supε↓0

ExG(XτDε

) ≤ Ex lim supε↓0

G(XτDε

)(2.2.46)

≤ ExG(

lim supε↓0

XτDε

)= ExG(XτD)

using that G is usc. This shows that τD is optimal in the case when G isbounded.

Second proof. We will divide the second proof in two parts depending on ifG is bounded (from below) or not.

1. Let us assume that G is bounded from below. It means that c :=infx∈E G(x) > −∞ . Replacing G by G− c and V by V − c when c < 0 we seethat there is no restriction to assume that G(x) ≥ 0 for all x ∈ E .

By analogy with (2.2.31) and (2.2.32), with 0 < λ < 1 given and fixed,consider the sets

Cλ = x ∈ E : λV (x) > G(x) , (2.2.47)

Dλ = x ∈ E : λV (x) ≤ G(x) . (2.2.48)

Since V is lsc and G is usc we see that Cλ is open and D is closed. Moreover,it is clear that Cλ ↑ C and Dλ ↓ D as λ ↑ 1 where C and D are defined by(2.2.4) and (2.2.5) above respectively.


τDλ= inf t ≥ 0 : Xt ∈ Dλ. (2.2.49)

Since D ⊆ Dλ and Px(τD < ∞) = 1 for all x ∈ E , we see that Px(τDλ< ∞) = 1

for all x ∈ E . (The latter fact can also be derived directly as in the remarkfollowing (2.2.33) above.)


In order to show that

ExV(XτDλ

)= V (x) (2.2.50)

for all x ∈ E , we will first note that

G(x) ≤ λV (x) + (1−λ)ExV(XτDλ

)(2.2.51)

for all x ∈ E . Indeed, if x ∈ Cλ then G(x) < λV (x) ≤ λV (x)+(1−λ)ExV(XτDλ

)since V ≥ G ≥ 0 on E . On the other hand, if x ∈ Dλ then τDλ

≡ 0 and (2.2.51)follows since G ≤ V on E .

Next in exactly the same way as in (2.2.38) above one verifies that the func-tion

x → ExV(XτDλ

)is superharmonic (2.2.52)

from E to R . Hence the function on the right-hand side of(2.2.51) is superhar-monic so that by the definition of V we can conclude that

V (x) ≤ λV (x) + (1−λ)ExV(XτDλ

)(2.2.53)

for all x ∈ E . This proves (2.2.50) as claimed.

From (2.2.50) we get

V (x) = ExV(XτDλ

) ≤ 1λ

ExG(XτDλ

) ≤ 1λ

V (x) (2.2.54)

for all x ∈ E upon using that V(XτDλ

) ≤ (1/λ)G(XτDλ

)since V is lsc and G

is usc. Letting λ ↑ 1 in (2.2.54) we see that V ≤ V and thus by (2.2.30) we canconclude that V = V . From (2.2.54) we thus see that

V (x) ≤ 1λ

ExG(XτDλ

)(2.2.55)

for all x ∈ E and all 0 ≤ λ < 1 .

Letting λ ↑ 1 and using that Dλ ↓ D we see that τDλ↑ τ1 where τ1 is a

stopping time satisfying τ1 ≤ τD . Since V is lsc and G is usc it is easily seen fromthe definition of τDλ

that V (τDλ) ≤ (1/λ)G(τDλ

) for all 0 < λ < 1 . Lettingλ ↑ 1 and using that X is left-continuous over stopping times it follows thatV (Xτ1) ≤ G(Xτ1) since V is lsc and G is usc. This shows that V (Xτ1) = G(Xτ1)and therefore τD ≤ τ1 showing that τ1 = τD . Thus τDλ

↑ τ1 as λ ↑ 1 .

Making use of the latter fact in (2.2.55) upon letting λ ↑ 1 and applyingFatou’s lemma, we get

V (x) ≤ lim supλ↑1

ExG(XτDλ

) ≤ Ex lim supλ↑1

G(XτDλ

)(2.2.56)

≤ ExG(

lim supλ↑1

XτDλ

)= ExG(XτD)


using that G is usc. This shows that τD is optimal in the case when G is boundedfrom below.

2. Let us assume that G is a (general) measurable function satisfying(2.2.1) (i.e. not necessarily bounded or bounded from below). Then Part 1 ofthe proof can be extended by means of the function h : E → R defined by

h(x) = Ex

(inft≥0

G(Xt))

(2.2.57)

for x ∈ E . The key observation is that −h is superharmonic which is seen asfollows (recall (2.2.57)):

Ex(−h(Xσ)) = ExEXσ supt≥0

(−G(Xt)) = ExEx

(supt≥0

(−G(Xt)) θσ | Fσ

)(2.2.58)

= ExEx

(supt≥0

(−G(Xσ+t)))≤ −h(x)

for all x ∈ E proving the claim. Moreover, it is obvious that V − h ≥ G − h ≥ 0on E . Knowing this we can define sets Cλ and Dλ by extending (2.2.47) and(2.2.48) as follows:

Cλ =

x ∈ E : λ(V (x)− h(x)

)> G(x)− h(x)

(2.2.59)

Dλ =

x ∈ E : λ(V (x)− h(x)

) ≤ G(x)− h(x)

(2.2.60)

for 0 < λ < 1 .

We then claim that

G(x) − h(x) ≤ λ(V (x) − h(x)

)+ (1−λ)Ex

(V (XτDλ

) − h(XτDλ))

(2.2.61)

for all x ∈ E . Indeed, if x ∈ Cλ then (2.2.61) follows by the fact that V ≥ hon E . On the other hand, if x ∈ Dλ then τDλ

= 0 and the inequality (2.2.61)reduces to the trivial inequality that G ≤ V . Thus (2.2.61) holds as claimed.

Since −h is superharmonic we have

−h(x) ≥ −λh(x) + (1−λ)Ex

(− h(XτDλ))

(2.2.62)

for all x ∈ E . From (2.2.61) and (2.2.62) we see that

G(x) ≤ λV (x) + (1−λ)ExV(XτDλ

)(2.2.63)

for all x ∈ E . Upon noting that Dλ ↓ D as λ ↑ 1 the rest of the proof can becarried out in exactly the same way as in Part 1 above. (If h does not happento be lsc, then Cλ and Dλ are still measurable sets and thus τDλ

is a stoppingtime (with respect to the completion of (FX

t )t≥0 by the family of all Px -null


sets from FX∞ for x ∈ E ). Moreover, it is easily verified using the strong Markovproperty of X and the conditional Fatou lemma that

h(XτD) ≤ lim supλ↓0

h(XτDλ

)Px -a.s. (2.2.64)

for all x ∈ E , which is sufficient for the proof.)

The final claim of the theorem follows from (2.2.11) in Theorem 2.4 above.The proof is complete.

Remark 2.8. The result and proof of Theorem 2.7 above extend in exactly thesame form (by slightly changing the notation only) to the finite horizon problem(2.2.2′) . Note moreover in this case that τD ≤ T < ∞ (since V (T, x) = G(T, x)and thus (T, x) ∈ D for all x ∈ E ) so that the condition Px(τD < ∞) = 1 isautomatically satisfied for all x ∈ E and need not be assumed.

9. The following corollary is an elegant tool for tackling the optimal stoppingproblem in the case when one can prove directly from definition of V that V islsc. Note that the result is particularly useful in the case of finite horizon sinceit provides the existence of an optimal stopping time τ∗ by simply identifying itwith τD from (2.2.6) above.

Corollary 2.9. (The existence of an optimal stopping time)

Infinite horizon. Consider the optimal stopping problem (2.2.3) upon assumingthat the condition (2.2.1) is satisfied. Suppose that V is lsc and G is usc. IfPx(τD < ∞) = 1 for all x ∈ E , then τD is optimal in (2.2.3). If Px(τD < ∞) < 1for some x ∈ E , then there is no optimal stopping time (with probability 1) in(2.2.3).

Finite horizon. Consider the optimal stopping problem (2.2.2′) upon assumingthat the corresponding condition (2.2.1) is satisfied. Suppose that V is lsc and Gis usc. Then τD is optimal in (2.2.2′) .

Proof. The case of finite horizon can be proved in exactly the same way as thecase of infinite horizon if the process (Xt) is replaced by the process (t, Xt) fort ≥ 0 . A proof in the case of infinite horizon can be given as follows.

The key is to show that V is superharmonic. For this, note that V ismeasurable (since it is lsc) and thus so is the mapping

V (Xσ) = supτ

EXσG(Xτ ) (2.2.65)

for any stopping time σ which is given and fixed. On the other hand, by thestrong Markov property we have

EXσG(Xτ ) = Ex

(G(Xσ+τθσ ) | Fσ

)(2.2.66)


for every stopping time τ and x ∈ E . From (2.2.65) and (2.2.66) we see that

V (Xσ) = esssupτ

Ex

(G(Xσ+τθσ) | Fσ

)(2.2.67)

under Px where x ∈ E is given and fixed.

Next we will show that the familyEx(Xσ+τθσ | Fσ

): τ is a stopping time

(2.2.68)

is upwards directed in the sense of (1.1.25). Indeed, if τ1 and τ2 are stoppingtimes given and fixed, set ρ1 = σ + τ1 θσ and ρ2 = σ + τ2 θσ , and define

B =

Ex(Xρ1 | Fσ) ≥ Ex(Xρ2 | Fσ). (2.2.69)

Then B ∈ Fσ and the mapping

ρ = ρ1 IB + ρ2 IBc (2.2.70)

is a stopping time. To verify this let us note that ρ ≤ t = (ρ1 ≤ t ∩ B) ∪(ρ2 ≤ t∩Bc) = (ρ1 ≤ t∩B ∩ σ ≤ t)∪ (ρ2 ≤ t∩Bc ∩ σ ≤ t) ∈ Ft sinceB and Bc belong to Fσ proving the claim. Moreover, the stopping time ρ canbe written as

ρ = σ + τ θσ (2.2.71)

for some stopping time τ . Indeed, setting

A =

EX0G(Xτ1) ≥ EX0G(Xτ2)

(2.2.72)

we see that A ∈ F0 and B = θ−1σ (A) upon recalling (2.2.66). Hence from (2.2.70)

we getρ = (σ + τ1 θσ)IB + (σ + τ2 θσ)IBc (2.2.73)

= σ +((τ1 θσ)(IA θσ) + (τ2 θσ)(IAc θσ)

)= σ + (τ1IA + τ2IAc) θσ

which implies that (2.2.71) holds with the stopping time τ = τ1 IA + τ2 IAc . (Thelatter is a stopping time since τ ≤ t = (τ1 ≤ t∩A)∪ (τ2 ≤ t∩Ac) ∈ Ft forall t ≥ 0 due to the fact that A ∈ F0 ⊆ Ft for all t ≥ 0 .) Finally, we have

E(Xρ | Fσ) = E(Xρ1 | Fσ)IB + E(Xρ2 | Fσ)IBc (2.2.74)

= E(Xρ1 | Fσ) ∨ E(Xρ2 | Fσ)

proving that the family (2.2.68) is upwards directed as claimed.

From the latter using (1.1.25) and (1.1.26) we can conclude that there existsa sequence of stopping times τn : n ≥ 1 such that

V (Xσ) = limn→∞Ex

(G(Xσ+τnθσ) | Fσ

)(2.2.75)


where the sequence

Ex

(G(Xσ+τnθσ) | Fσ

): n ≥ 1

is increasing Px -a.s. By

the monotone convergence theorem using (2.2.1) above we can therefore conclude

ExV (Xσ) = limn→∞ExG(Xσ+τnθσ) ≤ V (x) (2.2.76)

for all stopping times σ and all x ∈ E . This proves that V is superharmonic.(Note that the only a priori assumption on V used so far is that V is measurable.)As evidently V is the smallest superharmonic function which dominates G onE (recall (2.2.14) above) we see that the remaining claims of the corollary followdirectly from Theorem 2.7 above. This completes the proof. Remark 2.10. Note that the assumption of lsc on V and usc on G is natural,since the supremum of lsc functions defines an lsc function, and since every uscfunction attains its supremum on a compact set. To illustrate the former claimnote that if the function

x → ExG(Xτ ) (2.2.77)

is continuous (or lsc) for every stopping time τ , then x → V (x) is lsc and theresults of Corollary 2.9 are applicable. This yields a powerful existence result bysimple means (both in finite and infinite horizon). We will exploit the latter inour study of finite horizon problems in Chapters VI–VIII below. On the otherhand, if X is a one-dimensional diffusion, then V is continuous whenever G ismeasurable (see Subsection 9.3 below). Note finally that if Xt converges to X∞as t → ∞ then there is no essential difference between infinite and finite horizon,and the second half of Corollary 2.9 above (Finite horizon) applies in this case aswell, no matter if τD is finite or not. In the latter case one sees that τD is anoptimal Markov time (recall Example 1.14 above).

Remark 2.11. Theorems 2.4 and 2.7 above have shown that the optimal stoppingproblem (2.2.2) is equivalent to the problem of finding the smallest superharmonicfunction V which dominates G on E . Once V is found it follows that V = Vand τD from (2.2.6) is optimal (if no obvious contradiction arises).

There are two traditional ways for finding V :

(i) Iterative procedure (constructive but non-explicit),

(ii) Free-boundary problem (explicit or non-explicit).

Note that Corollary 2.9 and Remark 2.10 present yet another way for finding Vsimply by identifying it with V when the latter is known to be sufficiently regular(lsc).

The book [196, Ch. 3] provides numerous examples of (i) under various condi-tions on G and X . For example, it is known that if G is lsc and Ex inft≥0 G(Xt)> −∞ for all x ∈ E , then V can be computed as follows:

QnG(x) := G(x) ∨ ExG(X1/2n), (2.2.78)

V (x) = limn→∞ lim

N→∞QN

n G(x) (2.2.79)


for x ∈ E where QNn is the N -th power of Qn . The method of proof relies

upon discretization of the time set R+ and making use of discrete-time results ofoptimal stopping reviewed in Subsection 1.2 above. It follows that

If G is continuous and X is a Feller process, then V is lsc. (2.2.80)

The present book studies various examples of (ii). The basic idea (following fromthe results of Theorems 2.4 and 2.7) is that V and C ( or D ) should solve thefree-boundary problem:

LX V ≤ 0 (V minimal), (2.2.81)

V ≥ G (V > G on C & V = G on D) (2.2.82)

where LX is the characteristic (infinitesimal) operator of X (cf. Chapter II be-low).

Assuming that G is smooth in a neighborhood of ∂C the following “rule ofthumb” is valid. If X after starting at ∂C enters immediately into int (D) (e.g.when X is a diffusion process and ∂C is sufficiently nice) then the condition(2.2.81) (under (2.2.82) above) splits into the two conditions:

LX V = 0 in C, (2.2.83)

∂V

∂x

∣∣∣∂C

=∂G

∂x

∣∣∣∂C

(smooth fit). (2.2.84)

On the other hand, if X after starting at ∂C does not enter immediately intoint (D) (e.g. when X has jumps and no diffusion component while ∂C may stillbe sufficiently nice) then the condition (2.2.81) (under (2.2.82) above) splits intothe two conditions:

LX V = 0 in C, (2.2.85)

V∣∣∂C

= G∣∣∂C

(continuous fit). (2.2.86)

A more precise meaning of these conditions will be discussed in Chapter IV below(and through numerous examples throughout).

Remark 2.12. (Linear programming) A linear programming problem may be de-fined as the problem of maximizing or minimizing a linear function subject tolinear constraints.

Optimal stopping problems may be viewed as linear programming problems(cf. [55, p. 107]). Indeed, we have seen in Theorems 2.4 and 2.7 that the op-timal stopping problem (2.2.2) is equivalent to finding the smallest superhar-monic function V which dominates G on E . Letting L denote the linearspace of all superharmonic functions, letting the constrained set be defined byLG = V ∈ L : V ≥ G , and letting the objective function be defined by


F (V ) = V for V ∈ L , the optimal stopping problem (2.2.2) is equivalent to thelinear programming problem

V = infV ∈LG

F (V ) . (2.2.87)

Clearly, this formulation/interpretation extends to the martingale setting of Sec-tion 2.1 (where instead of superharmonic functions we need to deal with super-martingales) as well as to discrete time of both martingale and Markovian settings(Sections 1.1 and 1.2). Likewise, the free-boundary problem (2.2.81)–(2.2.82) maybe viewed as a linear programming problem.

A dual problem to the primal problem (2.2.87) can be obtained using the factthat the first hitting time τ∗ of St = V (Xt) to Gt = G(Xt) is optimal, so that

supt

(Gt−St) = 0 (2.2.88)

since St ≥ Gt for all t . It follows that

infS

E supt

(Gt−St) = 0 (2.2.89)

where the infimum is taken over all supermartingales S satisfying St ≥ Gt for allt . (Note that (2.2.89) holds without the expectation sign as well.) Moreover, theinfimum in (2.2.89) can equivalently be taken over all supermartingales S suchthat ES0 = ES0 (where we recall that ES0 = supτ EGτ ). Indeed, this followssince by the supermartingale property we have ESτ∗ ≤ ES0 so that

E supt

(Gt−St) ≥ E(Gτ∗−Sτ∗) ≥ EGτ∗−ES0 = EGτ∗−ES0 = 0 . (2.2.90)

Finally, since (St∧τ∗)t≥0 is a martingale, we see that (2.2.89) can also be writtenas

infM

E supt

(Gt−Mt) = 0 (2.2.91)

where the infimum is taken over all martingales M satisfying EM0 = ES0 . Inparticular, the latter claim can be rewritten as

supτ

EGτ = infM

E supt

(Gt−Mt) (2.2.92)

where the infimum is taken over all martingales M satisfying EM0 = 0 .

Notes. Optimal stopping problems originated in Wald’s sequential analysis[216] representing a method of statistical inference (sequential probability ratiotest) where the number of observations is not determined in advance of the ex-periment (see pp. 1–4 in the book for a historical account). Snell [206] formu-lated a general optimal stopping problem for discrete-time stochastic processes


(sequences), and using the methods suggested in the papers of Wald & Wolfowitz[219] and Arrow, Blackwell & Girshick [5], he characterized the solution by meansof the smallest supermartingale (called Snell’s envelope) dominating the gain se-quence. Studies in this direction (often referred to as martingale methods) aresummarized in [31].

The key equation V (x) = max(G(x), ExV (X1)) was first stated explicitly in[5, p. 219] (see also the footnote on page 214 in [5] and the book [18, p. 253]) butwas already characterized implicitly by Wald [216]. It is the simplest equation of“dynamic programming” developed by Bellman (cf. [15], [16]). This equation isoften referred to as the Wald–Bellman equation (the term which we use too) and itwas derived in the text above by a dynamic programming principle of “backwardinduction”. For more details on optimal stopping problems in the discrete-timecase see [196, pp. 111–112].

Following initial findings by Wald, Wolfowitz, Arrow, Blackwell and Girshickin discrete time, studies of sequential testing problems for continuous-time pro-cesses (including Wiener and Poisson processes) was initiated by Dvoretzky, Kiefer& Wolfowitz [51], however, with no advance to optimal stopping theory.

A transparent connection between optimal stopping and free-boundary prob-lems first appeared in the papers by Mikhalevich [135] and [136] where he usedthe “principle of smooth fit” in an ad hoc manner. In the beginning of the 1960’sseveral authors independently (from each other and from Mikhalevich) also con-sidered free-boundary problems (with “smooth-fit” conditions) for solving variousproblems in sequential analysis, optimal stopping, and optimal stochastic control.Among them we mention Chernoff [29], Lindley [126], Shiryaev [187], [188], [190],Bather [10], Whittle [222], Breakwell & Chernoff [22] and McKean [133]. Whilein the papers from the 1940’s and 50’s the ‘stopped’ processes were either sumsof independent random variables or processes with independent increments, the‘stopped’ processes in these papers had a more general Markovian structure.

Dynkin [52] formulated a general optimal stopping problem for Markov pro-cesses and characterized the solution by means of the smallest superharmonicfunction dominating the gain function. Dynkin treated the case of discrete timein detail and indicated that the analogous results also hold in the case of con-tinuous time. (For a connection of these results with Snell’s results [206] see thecorresponding remark in [52].)

The 1960’s and 70’s were years of an intensive development of the generaltheory of optimal stopping both in the “Markovian” and “martingale” setting aswell as both in the discrete and continuous time. Among references dealing mainlywith continuous time we mention [191], [88], [87], [193], [202], [210], [194], [184],[117], [62], [63], [211], [59], [60], [61], [141]. The book by Shiryaev [196] (see also[195]) provides a detailed presentation of the general theory of optimal stopping inthe “Markovian” setting both for discrete and continuous time. The book by Chow,Robbins & Siegmund [31] gives a detailed treatment of optimal stopping problemsfor general stochastic processes in discrete time using the “martingale” approach.The present Chapter I is largely based on results exposed in these books and


other papers quoted above. Further developments of optimal stopping followingthe 1970’s and extending to more recent times will be addressed in the presentmonograph. Among those not mentioned explicitly below we refer to [105] and[153] for optimal stopping of diffusions, [171] and [139] for diffusions with jumps,[120] and [41] for passage from discrete to continuous time, and [147] for optimalstopping with delayed information. The facts of dual problem (2.2.88)–(2.2.92)were used by a number of authors in a more or less disguised form (see [36], [13],[14], [176], [91], [95]).

Remark on terminology. In general theory of Markov processes the term ‘stop-ping time’ is less common and one usually prefers the term ‘Markov time’ (see e.g.[53]) originating from the fact that the strong Markov property remains preservedfor such times. Nevertheless in general theory of stochastic processes, where thestrong Markov property is not primary, one mostly uses the term ‘stopping’ (or‘optional’) time allowing it to take either finite or infinite values. In the presentmonograph we deal with both Markov processes and processes of general struc-ture, and we are mainly interested in optimal stopping problems for which thefinite stopping times are of central interest. This led us to use the “combined”terminology reserving the term ‘Markov’ for all and ‘stopping’ for finite times (thelatter corresponding to “real stopping” before the “end of time”).

Chapter II.

Stochastic processes: A brief review

From the table of contents of the monograph one sees that the basic processes wedeal with are

Martingales

(and related processes — supermartingales, submartingales, local martingales,semimartingales, etc.) and

Markov Processes.

We will mainly be interested in the case of continuous time. The case of discretetime can be considered as its particular case (by embedding). However, we shallconsider the case of discrete time separately because of its simplicity in comparisonwith the continuous-time case where there are many “technical” difficulties of themeasure-theoretic character (for example, the existence of “good” modifications,and similar).

3. Martingales

3.1. Basic definitions and properties

1. At the basis of all our probability considerations a crucial role belongs to thenotion of stochastic basis

(Ω,F , (Ft)t≥0, P) (3.1.1)

which is a probability space (Ω,F , P) equipped with an additional structure(Ft)t≥0 called ‘filtration’. A filtration is a nondecreasing (conforming to tradi-tion we say “increasing”) and right-continuous family of sub- σ -algebras of F (inother words Fs ⊆ Ft for all 0 ≤ s ≤ t and Ft =

⋂s>t Fs for all t ≥ 0 ). We

interpret Ft as the “information” (a family of events) obtained during the timeinterval [0, t] .

54 Chapter II. Stochastic processes: A brief review

Without further mention we assume that the stochastic basis (Ω,F , (Ft)t≥0,P) satisfies the usual conditions i.e. the σ -algebra F is P -complete and everyFt contains all P -null sets from F .

Instead of the term ‘stochastic basis’ one often uses the term ‘filtered prob-ability space’.

If X = (Xt)t≥0 is a family of random variables defined on (Ω,F) takingvalues in some measurable space (E, E) (i.e. such that E -valued variables Xt =Xt(ω) are F/E -measurable for each t ≥ 0 ) then one says that X = (Xt)t≥0 isa stochastic (random) process with values in E .

Very often it is convenient to consider the process X as a random elementwith values in ET where T = [0,∞) . From such a point of view a trajectoryt Xt(ω) is an element i.e. “point” in ET for each ω ∈ Ω .

All processes X considered in the monograph will be assumed to have theirtrajectories continuous ( X ∈ C , the space of continuous functions) or right-continuous for t ≥ 0 with left-hand limits for t > 0 ( X ∈ D , the space of cadlagfunctions; the French abbreviation cadlag means continu a droite avec des l imitesa gauche).

As usual we assume that for each t ≥ 0 the random variable Xt = Xt(ω)is Ft -measurable. To emphasize this property we often use the notation X =(Xt,Ft)t≥0 or X = (Xt,Ft) and say that the process X is adapted to thefiltration (Ft)t≥0 (or simply adapted).

2. Throughout the monograph a key role belongs to the notion of a Markovtime, i.e. a random variable, say τ = τ(ω) , with values in [0,∞] such that

ω : τ(ω) ≤ t ∈ Ft (3.1.2)

for all t ≥ 0 .

If τ(ω) < ∞ for all ω ∈ Ω or P -almost everywhere, then the Markov timeτ is said to be finite. Usually such Markov times are called stopping times.

The property (3.1.2) has clear meaning: for each t ≥ 0 a decision “to stopor not to stop” depends only on the “past and present information” Ft obtainedon the interval [0, t] and not depending on the “future”.

With the given process X = (Xt)t≥0 and a Markov time τ we associate a“stopped” process

Xτ = (Xt∧τ )t≥0 (3.1.3)

where Xt∧τ = Xt∧τ(ω)(ω) . It is clear that Xτ = X on the set ω : τ(ω) = ∞ .

If trajectories of the process X = (Xt,Ft) belong to the space D , thenvariables Xτ I(τ < ∞) are Fτ -measurable where by definition the σ -algebra

Fτ =A ∈ F : τ ≤ t ∩ A ∈ F for all t ≥ 0

. (3.1.4)

Section 3. Martingales 55

The notion of the stopped process plays a crucial role in defining the notions of“local classes” and “localization procedure”.

Namely, let X be some class of processes. We say that a process X belongsto the “localized” class Xloc if there exists an increasing sequence (τn)n≥1 ofMarkov times (depending on X ) such that limn τn = ∞ P-a.s. and each stoppedprocess Xτn belongs to X . The sequence (τn)n≥1 is called a localizing sequencefor X (relative to X ).

3. The process X = (Xt,Ft)t≥0 is called a martingale [respectively super-martingale or submartingale] if X ∈ D and

E |Xt| < ∞ for t ≥ 0; (3.1.5)E (Xt | Fs) = [ ≤ or ≥ ] Xt for s ≤ t (a martingale [respectivelysupermartingale or submartingale] property).

(3.1.6)

We denote the classes of martingales, supermartingales and submartingalesby M , supM and subM , respectively. It is clear that if X ∈ supM then−X ∈ subM . The processes X = (Xt,Ft)t≥0 which belong to the classes Mloc ,(supM)loc and (subM)loc are called local martingales, local supermartingalesand local submartingales, respectively.

The notion of a local martingale is important for definition of a semimartin-gale.

We say that the process X = (Xt,Ft)t≥0 with cadlag trajectories is asemimartingale ( X ∈ SemiM ) if this process admits a representation (gener-ally speaking, not unique) of the form

X = X0 + M + A (3.1.7)

where X0 is a finite-valued and F0 -measurable random variable, M = (Mt,Ft)is a local martingale ( M ∈ Mloc ) and A = (At,Ft) is a process of boundedvariation ( A ∈ V ), i.e.

∫ t

0|dAs(ω)| < ∞ for t > 0 and ω ∈ Ω .

4. In the whole semimartingale theory the notion of predictability plays also(together with notions of Markov time, martingales, etc.) an essential role, beingsome kind of stochastic “determinancy”.

Consider a space Ω × R+ = (ω, t) : ω ∈ Ω, t ≥ 0 and a process Y =(Yt(ω),Ft)t≥0 with left-continuous (cag = continuite a gauche) trajectories.

The predictable σ -algebra is the algebra P on Ω×R+ that is generated byall cag adapted processes Y considered as mappings (ω, t) Yt(ω) on Ω×R+ .

One may define P in the following equivalent way: the σ -algebra P isgenerated by


(i) the system of sets

A × 0 where A ∈ F0 andA × (s, t] where A ∈ Fs for 0 ≤ s ≤ t

(3.1.8)

or

(ii) the system of

sets A × 0 where A ∈ F0 andstochastic intervals 0, τ = (ω, t) : 0 ≤ t ≤ τ(ω) (3.1.9)

where τ are Markov times.

Every adapted process X = (Xt(ω))t≥0 , ω ∈ Ω , which is P -measurable is calleda predictable process.

5. The following theorem plays a fundamental role in stochastic calculus.In particular, it provides a famous example of a semimartingale. Recall that aprocess X belongs to the Dirichlet class (D) if the family of random variablesXτ : τ is a finite stopping time is uniformly integrable.

Theorem 3.1. (Doob–Meyer decomposition)(a) Every submartingale X = (Xt,Ft)t≥0 admits the decomposition

Xt = X0 + Mt + At, t ≥ 0 (3.1.10)

where M ∈ Mloc and A = (At,Ft)t≥0 is an increasing predictable locally inte-grable process (A ∈ P ∩ A+

loc) .

(b) Every submartingale X = (Xt,Ft)t≥0 of the Dirichlet class (D) admits thedecomposition Xt = X0 +Mt +At, t ≥ 0, with a uniformly integrable martingaleM and an increasing predictable integrable process A (∈ P ∩ A+) .

Given decompositions are unique up to stochastic indistinguishability (i.e. ifXt = X0 + M ′

t + A′t is another decomposition of the same type, then the sets

ω : ∃t with Mt(ω) = M ′t(ω) and ω : ∃t with At(ω) = A′

t(ω) are P -null.

Note two important corollaries of the Doob–Meyer decomposition.

(A) Every predictable local martingale M = (Mt,Ft)t≥0 with M0 = 0which at the same time belongs to the class V ( M ∈ Mloc ∩ V ) is equal to zero(up to stochastic indistinguishability).

(B) Suppose that a process A ∈ A+loc . Then there exists a process A ∈

P ∩A+loc (unique up to stochastic indistinguishability) such that A − A ∈ Mloc .

The process A is called a compensator of the process A . This process may alsobe characterized as a predictable increasing process A such that for any (finite)stopping time τ one has

EAτ = E Aτ (3.1.11)


or as a predictable increasing process A such that

E (H · A)∞ = E (H · A)∞ (3.1.12)

for any non-negative increasing predictable process H =(Ht,Ft)t≥0 . (Here (H·A)t

is the Lebesgue–Stieltjes integral∫ t

0 Hs(ω) dAs(ω) for ω ∈ Ω .)

6. The notions introduced above for the case of continuous time admit thecorresponding analogues also for the case of discrete time.

Here the stochastic basis is a filtered probability space (Ω,F , (Fn)n≥0, P)with a family of σ -algebras (Fn)n≥0 such that F0 ⊆ F1 ⊆ · · · ⊆ F . (The notionof right-continuity loses its meaning in the case of discrete time.) The notions ofmartingales and related processes are introduced in a similar way.

With every “discrete” stochastic basis (Ω,F , (Fn)n≥0, P) one may associatea “continuous” stochastic basis (Ω,F , (Ft)t≥0, P) by setting Ft = F[t] for t ≥ 0 .In particular Fn = Fn , Fn− = Fn−1 = Fn−1 for n ≥ 1 .

In a similar way with any process X = (Xn,Fn)n≥0 we may associate thecorresponding process X = (Xt, Ft)t≥0 in continuous time by setting Xt = X[t]

for t ≥ 0 .

Observe also that the notion of predictability of a process X = (Xn)n≥0 onthe stochastic basis (Ω,F , (Fn)n≥0, P) takes a very simple form: Xn is Fn−1 -measurable for each n ≥ 1 .

7. Along with the σ -algebra P of predictable sets on Ω×R+ an importantrole in the general theory of stochastic processes belongs to the notion of optionalσ -algebra O that is a minimal σ -algebra generated by all adapted processesY = (Yt(ω))t≥0 , ω ∈ Ω (considered as a mapping (ω, t) Yt(ω) ) with right-continuous (for t ≥ 0 ) trajectories which have left-hand limits (for t > 0 ).

It is clear that P ⊆ O . If a process X = (Xt,Ft) is O -measurable thenwe say that this process is optional. With an agreement that all our processes(martingales, supermartingales, etc.) have cadlag trajectories, we see that theyare optional. The important property of such processes is the following: for anyopen set B from B(R) the random variable

τ = inf t ≥ 0 : Xt ∈ B (3.1.13)

is a Markov time (we put inf ∅ = ∞ as usual). Note that this property also holdsfor adapted cad processes.

Another important property of optional processes X is the following: any“stopped” process Xτ = (Xt∧τ ,Ft) where τ is a Markov time is optional againand the random variable XτI(τ < ∞) is Fτ -measurable.


All adapted cad processes X = (Xt,Ft) are such that for each t > 0 theset

(ω, s) : ω ∈ Ω, s ≤ t and Xs(ω) ∈ B ∈ Ft ⊗ B([0, t]) (3.1.14)

where B ∈ B(R) . This property is called the “property of progressive measura-bility”. The importance of such processes may be demonstrated by the fact thatthen the process

It(ω) =∫ t

0

f(Xs(ω)) ds, t ≥ 0 (3.1.15)

for a measurable bounded function f will be adapted, i.e. It(ω) is Ft -measurablefor each t ≥ 0 .

8. Although in the monograph we deal mainly with continuous processes itwill be useful to consider the structure of general local martingales from the stand-point of its decomposition into “continuous” and “discontinuous” components.

We say that two local martingales M and N are orthogonal if their productMN is a local martingale. A local martingale M is called purely discontinuousif M0 = 0 and M is orthogonal to all continuous local martingales.

First decomposition of a local martingale M = (Mt,Ft)t≥0 states that thereexists a unique (up to indistinguishability) decomposition

M = M0 + M c + Md (3.1.16)

where M c0 = Md

0 = 0 , M c is a continuous local martingale, and Md is a purelydiscontinuous one.

Second decomposition of a local martingale M = (Mt,Ft)t≥0 states that Madmits a (non-unique) decomposition

M = M0 + M ′ + M ′′ (3.1.17)

where M ′ and M ′′ are local martingales with M ′0 = M ′′

0 = 0 , M ′ has finitevariation and |∆M ′′| ≤ a (i.e. |∆M ′′

t | ≤ a for all t > 0 where ∆M ′′t = M ′′

t −M ′′

t− ).

From this decomposition we conclude that every local martingale can bewritten as a sum of a local martingale of bounded variation and a locally square-integrable martingale (because M ′′ is a process of such a type).

With each pair M and N of locally square-integrable martingales ( M, N ∈M2

loc ) one can associate (by the Doob–Meyer decomposition) a predictable process〈M, N〉 of bounded variation such that

MN − 〈M, N〉 ∈ Mloc. (3.1.18)

If N = M then we get M2 − 〈M〉 ∈ Mloc where 〈M〉 stands for the increasingpredictable process 〈M, M〉 .


The process 〈M, N〉 is called the predictable quadratic covariation and the“angle bracket” process 〈M〉 is called the predictable quadratic variation or qua-dratic characteristic.

9. Let X = (Xt,Ft) be a semimartingale ( X ∈ SemiM ) with a decompo-sition

Xt = X0 + Mt + At, t ≥ 0 (3.1.19)

where M = (Mt,Ft) ∈ Mloc and A = (At,Ft) ∈ V .

In the class SemiM of semimartingales a special role is played by the classSp-SemiM of special semimartingales i.e. semimartingales X for which one canfind a decomposition X = X0 + M + A with predictable process A of boundedvariation. If we have also another decomposition X = X0 + M ′ + A′ with pre-dictable A′ then A = A′ and M = M ′ . So the “predictable” decomposition ofa semimartingale is unique.

The typical example of a special semimartingale is a semimartingale withbounded jumps ( |∆X | ≤ a ). For such semimartingales one has |∆A| ≤ a and|∆M | ≤ 2a . In particular, if X is a continuous semimartingale then A and Mare also continuous.

Every local martingale M has the following property: for all t > 0 ,∑s≤t

|∆Ms|2 < ∞ P-a.s. (3.1.20)

Because in a semimartingale decomposition X = X0 +M +A the process A hasbounded variation, we have for any t > 0 ,∑

s≤t

|∆Xs|2 < ∞ P-a.s. (3.1.21)

10. From the first decomposition of a local martingale M = M c + Md withM0 = 0 we conclude that if a semimartingale X has a decomposition X =X0 + M + A then

X = X0 + M c + Md + A. (3.1.22)

If moreover X = X0 + M c + Md + A is another decomposition then

(M c − M c) + (Md − Md) = A − A. (3.1.23)

Since the process A−A ∈ V it follows that the process (M c−M c)+(Md−Md) ∈V .

But every local martingale, which at the same time is a process from theclass V , is purely discontinuous. So the continuous local martingale M c − M c

is at the same time purely discontinuous and therefore P-a.s. equal to zero, i.e.M c = M c .


In other words the continuous martingale component of a semimartingale isdefined uniquely. It explains why this component of X is usually denoted by Xc .

3.2. Fundamental theorems

There are three fundamental results in the martingale theory (“three pillars ofmartingale theory”):

A. The optional sampling theorem;B. Martingale convergence theorem;C. Maximal inequalities.

The basic statements here belong to J. Doob and there are many different modi-fications of his results.

Let us state basic results from A, B and C.

A. The optional sampling theorem

(A1) Doob’s stopping time theorem. Suppose that X = (Xt,Ft)t≥0 is a sub-martingale (martingale) and τ is a Markov time. Then the “stopped” pro-cess Xτ = (Xt∧τ ,Ft)t≥0 is also a submartingale (martingale).

(A2) Hunt’s stopping time theorem. Let X = (Xt,Ft)t≥0 be a submartingale(martingale). Assume that σ = σ(ω) and τ = τ(ω) are bounded stoppingtimes and σ(ω) ≤ τ(ω) for ω ∈ Ω . Then

Xσ ≤ (=) E (Xτ | Fσ) P-a.s. (3.2.1)

The statements of these theorems remain valid also for unbounded stopping timesunder the additional assumption that the family of random variables Xt : t ≥ 0is uniformly integrable.

The results (A1) and (A2) are particular cases of the following general propo-sition:

(A3) Let X = (Xt,Ft)t≥0 be a submartingale (martingale) and let σ and τ betwo stopping times for which

E |Xσ| < ∞, E |Xτ | < ∞ and lim inft→∞ EI(τ > t)|Xt| = 0. (3.2.2)

Then on the set τ ≥ σ

E (Xτ | Fσ) ≥ (=) Xσ P-a.s. (3.2.3)

If in addition P(τ ≥ σ) = 1 then

EXτ ≥ (=) EXσ. (3.2.4)


If B = (Bt)t≥0 is a standard Brownian motion and τ is a stopping time, thenfrom (A3) one may obtain the Wald identities :

EBτ = 0 if E√

τ < ∞, (3.2.5)

EB2τ = Eτ if Eτ < ∞. (3.2.6)

B. Martingale convergence theorem

(B1) Doob’s convergence theorem. Let X = (Xt,Ft)t≥0 be a submartingale with

supt

E |Xt| < ∞ (equivalently: supt

EX+t < ∞ ). (3.2.7)

Then there exists an F∞ -measurable random variable X∞ (where F∞ ≡σ(⋃

t≥0 Ft) ) with E |X∞| < ∞ such that

Xt → X∞ P-a.s. as t → ∞ . (3.2.8)

If the condition (3.2.7) is strengthened to uniform integrability, then theP-a.s. convergence in (3.2.7) also takes place in L1 , i.e.:

(B2) If the family Xt : t ≥ 0 is uniformly integrable, then

E |Xt − X∞| → 0 as t → ∞. (3.2.9)

(B3) Levy’s convergence theorem. Let (Ω,F , (Ft)t≥0, P) be a stochastic basis andξ an integrable F -measurable random variable. Put F∞ = σ

(⋃t≥0 Ft

).

Then P-a.s. and in L1 ,

E (ξ | Ft) → E (ξ | F∞) as t → ∞. (3.2.10)

C. Maximal inequalities

The following two classical inequalities of Kolmogorov and Khintchine gave riseto the field of the so-called ‘martingale inequalities’ (in probability and in mean)for random variables of type

supt≤T

Xt, supt≤T

|Xt| and supt≥0

Xt, supt≥0

|Xt|. (3.2.11)

(C1) Kolmogorov’s inequalities. Suppose that Sn = ξ1 + · · · + ξn , n ≥ 1 , whereξ1, ξ2, . . . are independent random variables with Eξk = 0 , Eξ2

k < ∞ ,k ≥ 1 . Then for any ε > 0 and arbitrary n ≥ 1 ,

P(

max1≤k≤n

|Sk| ≥ ε)≤ ES2

n

ε2. (3.2.12)

If additionally P(|ξk| ≤ c) = 1 , k ≥ 1 , then

P(

max1≤k≤n

|Sk| ≥ ε)≥ 1 − (c + ε)2

ES2n

. (3.2.13)


(C2) Khintchine’s inequalities. Suppose that ξ1, ξ2, . . . are independent Bernoullirandom variables with P(ξj = 1) = P(ξj = −1) = 1/2 , j ≥ 1 , and (cj) isa sequence of real numbers. Then for any p > 0 and any n ≥ 1 ,

Ap

( n∑j=1

c2j

)p/2

≤ E

∣∣∣∣ n∑j=1

cjξj

∣∣∣∣p ≤ Bp

( n∑j=1

c2j

)p/2

(3.2.14)

where Ap and Bp are some universal constants.

Note that in the inequalities of Kolmogorov and Khintchine the sequences (Sn)n≥1

and(∑n

j=1 cjξj

)n≥1

form martingales.

Generalizations of these inequalities are the following inequalities.

(C3) Doob’s inequalities (in probability). Let X = (Xt,Ft)t≥0 be a submartin-gale. Then for any ε > 0 and each T > 0 ,

P(

supt≤T

Xt ≥ ε)≤ 1

εE[X+

T I(

supt≤T

Xt ≥ ε)]

≤ 1ε

EX+T (3.2.15)

andP(

supt≤T

|Xt| ≥ ε)≤ 1

εsupt≤T

E |Xt|. (3.2.16)

If X = (Xt,Ft)t≥0 is a martingale then for all p ≥ 1 ,

P(

supt≤T

|Xt| ≥ ε)≤ 1

εpE |XT |p (3.2.17)

and, in particular, for p = 2 ,

P(

supt≤T

|Xt| ≥ ε)≤ 1

ε2E |XT |2. (3.2.18)

(C4) Doob’s inequalities (in mean). Let X = (Xt,Ft)t≥0 be a non-negative sub-martingale. Then for p > 1 and any T > 0 ,

EXpT ≤ E

(supt≤T

Xt

)p≤(

p

p − 1

)p

EXpT (3.2.19)

and for p = 1

EXT ≤ E supt≤T

Xt ≤ e

e − 1

[1 + E (XT log+XT )

]. (3.2.20)

In particular, if X = (Xt,Ft)t≥0 is a square-integrable martingale, then

E supt≤T

X2t ≤ 4 EX2

T . (3.2.21)


(C5) Burkholder–Davis–Gundy’s inequalities. Suppose that X = (Xt,Ft)t≥0 is amartingale. Then for each p ≥ 1 there exist universal constants A∗

p andB∗

p such that for any stopping time T ,

A∗p E [X ]p/2

T ≤ E supt≤T

|Xt|p ≤ B∗p E [X ]p/2

T (3.2.22)

where [X ]t = 〈Xc〉t +∑

s≤t(∆Xs)2 is the quadratic variation of X .

In the case p > 1 these inequalities are equivalent (because of Doob’s in-equalities in mean for p > 1 ) to the following inequalities:

Ap E [X ]p/2T ≤ E |XT |p ≤ Bp E [X ]p/2

T (3.2.23)

with some universal constants Ap and Bp (whenever E [X ]p/2T < ∞ ).

3.3. Stochastic integral and Ito’s formula

1. The class of semimartingales is rather wide and rich because it is invariantwith respect to many transformations — “stopping”, “localization”, “change oftime”, “absolute continuous change of measure”, “change of filtration”, etc. Itis remarkable and useful that for a semimartingale X the notion of stochasticintegral H · X , that is a cornerstone of stochastic calculus, may be defined for avery large class of integrands H .

Let us briefly give the basic ideas and ways of construction of the stochasticintegral.

Suppose that a function H = (Ht)t≥0 is “very simple”:

H =

⎧⎪⎨⎪⎩Y I0 where Y is F0-measurable,orY Ir,s where Y is Fr-measurable

(3.3.1)

with 0 = (ω, t) : ω ∈ Ω , t = 0 and r, s = (ω, t) : ω ∈ Ω , r < t ≤ s forr < s .

For such “very simple” functions a natural definition of the stochastic integralH · X = (H · X)t : t ≥ 0 should apparently be the following:

(H · X)t =

0 if H = Y I0,

Y (Xs∧t − Xr∧t) if H = Y Ir,s.(3.3.2)

By linearity one can extend this definition to the class of “simple” functions whichare linear combination of “very simple” functions.


It is more interesting that the stochastic integral H · X defined in such away can be extended to the class of

locally bounded predictable processes H (3.3.3)

so that the following properties remain valid:

(a) the process H · X is cadlag;

(b) the mapping H H ·X is linear (i.e. (aH ′ + H ′′) ·X = aH ′ ·X + H ′′ ·X )up to stochastic indistinguishability;

(c) if a sequence (Hn) of predictable processes converges pointwise uniformlyon [0, t] for each t > 0 to a predictable process H and |Hn| ≤ K , whereK is a locally bounded predictable process, then

sups≤t

|(Hn · X)s − (H · X)s| P→ 0 for each t > 0 . (3.3.4)

The stochastic integral H ·X constructed has many natural properties which areusually associated with the notion of ‘integral’:

(1) the mapping H H · X is linear;

(2) the process H · X is a semimartingale;

(3) if X ∈ Mloc then H · X ∈ Mloc ;

(4) if X ∈ V then H · X ∈ V ;

(5) (H · X)0 = 0 and H · X = H · (X − X0) i.e. the stochastic integral is not“sensitive” to the initial value X0 ;

(6) ∆(H · X) = H∆X ;

(7) the stopped process Xτ = (Xτt )t≥0 can be written in the form

Xτt = X0 + (I0,τ · X)t. (3.3.5)

Note that very often we use a more transparent notation∫ t

0 Hs dXs for thestochastic integral (H · X)t .

One can also extend the stochastic integral to a class of predictable processeswhich are not locally bounded. (Note that some “natural” properties of the typeX ∈ Mloc =⇒ H · X ∈ Mloc may fail to hold.)

To present this extension we need a series of new notions.


Let X = (Xt,Ft) be a semimartingale with a continuous martingale partXc . Because Xc ∈ M2

loc the predictable process 〈Xc〉 (called the quadraticcharacteristic of X ) such that

(Xc)2 − 〈Xc〉 ∈ Mloc (3.3.6)

does exist. Define[X ] = 〈Xc〉 +

∑s≤ ·

(∆Xs)2. (3.3.7)

The latter process (called the quadratic variation of the semimartingale X ) canalso be defined in terms of the stochastic integral H · X introduced above bytaking Ht = Xt− for t > 0 . Moreover,

X2 − X20 − 2X− · X = [X ]. (3.3.8)

(Sometimes this identity is taken as a definition of [X ] .)

Suppose that X = X0 + A + M is a decomposition of X with A ∈ V ,M ∈ Mloc , and let H be a predictable process.

We say thatH ∈ Lvar(A) (3.3.9)

if∫ t

0|Hs(ω)| d(Var(A))s < ∞ for all t > 0 and ω ∈ Ω .

We also say thatH ∈ L′

loc(M) (3.3.10)

if the process ((∫ t

0

H2s d[M ]s

)1/2)t≥0

∈ A+loc (3.3.11)

i.e. is locally integrable.

Finally, we say thatH ∈ L(X) (3.3.12)

if one can find a decomposition X = X0 + A + M such that H ∈ Lvar(A) ∩L′

loc(M) .

For functions H ∈ L(X) by definition the stochastic integral is set to be

H · X = H · A + H · M (3.3.13)

where H · A is the Lebesgue–Stieltjes integral and H · M is an integral withrespect to the local martingale M , which for functions H ∈ L′

loc(M) is definedvia a limiting procedure using the well-defined stochastic integrals Hn · X forbounded predictable functions Hn = HI(|H | ≤ n) .

Let us emphasize that the definition of H · X given above assumes theexistence of a decomposition X = X0+A+M such that H ∈ Lvar(A)∩L′

loc(M) .


At a first glance this definition could appear a little strange. But its correctnessis obtained from the following result: if there exists another decomposition X =X0 + A′ + M ′ such that

H ∈ (Lvar(A) ∩ L′loc(M)

) ∩ (Lvar(A′) ∩ L′loc(M

′))

(3.3.14)

then

H · A + H · M = H · A′ + H · M ′ (3.3.15)

i.e. both definitions lead to the same (up to indistinguishability) integral.

2. Along with the quadratic variation [X ] , an important role in stochasticcalculus belongs to the notion of the quadratic covariation [X, Y ] of two semi-martingales X and Y that is defined by

[X, Y ] = 〈Xc, Y c〉 +∑s≤ ·

∆Xs∆Ys. (3.3.16)

Here 〈Xc, Y c〉 is a predictable quadratic covariation of two continuous martingalesXc and Y c i.e. 〈Xc, Y c〉 = 1

4 (〈Xc + Y c〉 − 〈Xc − Y c〉) .

The quadratic covariation has the following properties:

(a) If X ∈ Mloc then [X, X ]1/2 ∈ Aloc .

(b) If X ∈ Mloc is continuous and Y ∈ Mloc is purely discontinuous, then[X, Y ] = 0 .

(c) If X, Y ∈ Mcloc are orthogonal (i.e. XY ∈ Mc

loc ) then [X, Y ] = 〈X, Y 〉 = 0 .

(d) If X and Y are semimartingales and H is a bounded predictable process,then [H · X, Y ] = H · [X, Y ] .

(e) If X is a continuous (purely discontinuous) local martingale, and H is alocally bounded predictable process, then H · X is a continuous (purelydiscontinuous) local martingale.

3. One of the central results of stochastic calculus is the celebrated Ito for-mula.

Let X = (X1, . . . , Xd) be a d -dimensional semimartingale (whose compo-nents are semimartingales). Suppose that a function F ∈ C2 and denote by DiF

and DijF partial derivatives ∂F∂xi

and ∂2F∂xi ∂xj

. Then the process Y = F (X) is


a semimartingale again and the following Ito’s change-of-variable formula holds:

F (Xt) = F (X0) +∑i≤d

DiF (X−) · X i (3.3.17)

+12

∑i,j≤d

Di,jF (X−) · 〈X i,c, Xj,c〉

+∑s≤t

(F (Xs) − F (Xs−) −

∑i≤d

DiF (Xs−)∆X is

).

Let us note some particular cases of this formula and some useful corollaries.

A) If X = (X1, . . . , Xd) is a continuous semimartingale then

F (Xt) = F (X0) +∑i≤d

DiF (X) · X i +12

∑i,j≤d

Di,jF (X) · 〈X i, Xj〉. (3.3.18)

B) If X and Y are two continuous semimartingales then the following for-mula of integration by parts holds:

XtYt = X0Y0 +∫ t

0

Xs dYs +∫ t

0

Ys dXs + 〈X, Y 〉t. (3.3.19)

In particular, we have

X2t = X2

0 + 2∫ t

0

Xs dXs + 〈X〉t. (3.3.20)

4. From Ito’s formula for functions F ∈ C2 one can get by limiting proce-dures its extensions for functions F satisfying less restrictive assumptions thanF ∈ C2 .

(I) The first result in this direction was the Tanaka (or Ito–Tanaka) formulafor a Brownian motion X = B and function F (x) = |x − a| :

|Bt − a| = |B0 − a| +∫ t

0

sgn (Bs − a) dBs + Lat (3.3.21)

where Lat is the local time that the Brownian notion B = (Bt)t≥0 “spends” at

the level a :

Lat = lim

ε↓012ε

∫ t

0

I(|Bs − a| ≤ ε) ds. (3.3.22)

(II) The second result was the Ito–Tanaka–Meyer formula: if the derivativeF ′(x) is a function of bounded variation then

F (Bt) = F (B0) +∫ t

0

F ′(Bs) dBs +12

∫R

Lat F ′′(da) (3.3.23)


where F ′′(da) is defined as a sign measure on R in the sense of distributions.

Subsequently the following two formulae were derived:

(III) The Bouleau–Yor formula states that if the derivative F ′(x) is a locallybounded function then

F (Bt) = F (B0) +∫ t

0

F ′(Bs) dBs − 12

∫R

F ′(a) daLat . (3.3.24)

(IV) The Follmer–Protter–Shiryaev formula states that if the derivative F ′ ∈L2

loc , i.e.∫|x|≤M

(F ′(x))2 dx < ∞ for all M ≥ 0 , then

F (Bt) = F (B0) +∫ t

0

F ′(Bs) dBs +12[F ′(B), B]t (3.3.25)

where [F ′(B), B] is the quadratic covariation of F ′(B) and B :

[F ′(B), B]t (3.3.26)

= P-limn→∞

∑k

(F ′(Btn

k+1∧t

)− F ′(Btnk∧t

))(Btn

k+1∧t − Btnk∧t

)with supk(tnk+1 ∧ t − tnk ∧ t) → 0 .

Between the formulae (I)–(IV) there are the following relationships:

(I) ⊆ (II) ⊆ (III) ⊆ (IV) (3.3.27)

in the sense of expanding to the classes of functions F to which the correspondingformulae are applicable.

There are some generalizations of the results of type (I)–(IV) for semimartin-gale case.

For example, if X is a continuous semimartingale then the Ito–Tanaka for-mula takes the following form similar to the case of a Brownian motion:

|Xt − a| = |X0 − a| +∫ t

0

sgn (Xs − a) dXs + Lat (X) (3.3.28)

where

Lat (X) = lim

ε↓012ε

∫ t

0

I(|Xs| ≤ ε) d〈X〉s. (3.3.29)

If a function F = F (x) is concave (convex or the difference of the two) andX is a continuous semimartingale, then the Ito–Tanaka–Meyer formula takes thefollowing form:

F (Xt) = F (X0) +∫ t

0

12

(F ′

+(Xs) + F ′−(Xs)

)dXs +

12

∫R

Lat F ′′(da). (3.3.30)


An important corollary of this formula is the following occupation times formula: IfΦ = Φ(x) is a positive Borel function then for every continuous semimartingale X∫ t

0

Φ(Xs) d〈X〉s =∫

R

Φ(a) daLat . (3.3.31)

For many problems of stochastic analysis (and, in particular, for optimalstopping problems) it is important to have analogues of Ito’s formula for F (t, Xt)where F (t, x) is a continuous function whose derivatives in t and x are notcontinuous. A particular formula of this kind (derived by Peskir in [163]) will begiven in Subsection 3.5 below (see (3.5.5) and (3.5.9)).

5. Stochastic canonical representation for semimartingales. (a) Probabilisticand analytic methods developed for semimartingales can be considered in somesense as a natural extension of the methods created in the theory of processes withindependent increments. Therefore it is reasonable to recall some results of thistheory.

A stochastic (random) process X = (Xt,Ft) is a Process with IndependentIncrements ( X ∈ PII ) if X0 = 0 and random variables Xt − Xs for t > s areindependent from σ -algebras Fs . Such process is called a process with stationaryindependent increments ( X ∈ PIIS ) if distributions of Xt − Xs depend only ondifference t − s . Note that every deterministic process is a (degenerated) PII -process. In particular every deterministic function of unbounded variation is sucha process and so it is not a semimartingale. We shall exclude this uninterestingcase in order to stay within the semimartingale scheme. (The process X withindependent increments is a semimartingale if and only if for each λ ∈ R thecharacteristic function ϕ(t) = EeiλXt , t ≥ 0 , is a function of bounded variation.)

It is well known that with every process X ∈ PII one can associate a (deter-ministic) triplet (B, C, ν) where B = (Bt)t≥0 , C = (Ct)t≥0 , and ν = ν((0, t]×A)for A ∈ B(R \ 0) , t ≥ 0 , such that

ν((0, t] × A) = Eµ(ω; (0, t] × A) (3.3.32)

with the measure of jumps

µ(ω; (0, t] × A) =∑

0<s≤t

I(∆Xs(ω) ∈ A) (3.3.33)

of the process X . One has Bt ∈ R , Ct ≥ 0 and Ct ≥ Cs for t ≥ s ≥ 0 .

Measure ν has a special name — Levy measure and satisfies the followingproperty: ∫

R

min(x2 ∧ 1) ν((0, t] × dx) < ∞ (3.3.34)

for all t > 0 .


This property enables us to introduce the cumulant function

Kt(λ) = iλBt − λ2

2Ct +

∫ (eiλx − 1 − iλh(x)

)ν((0, t] × dx) (3.3.35)

where h = h(x) is a “truncation” function

h(x) = xI(|x| ≤ 1). (3.3.36)

With this definition for each process X ∈ PII we have for the characteristicfunction

Gt(λ) = EeiλXt , λ ∈ R, t ≥ 0 (3.3.37)

the following equation:

dGt(λ) = Gt−(λ) dKt(λ), G0(λ) = 1. (3.3.38)

The solution of this equation is given by the formula

Gt(λ) = eKt(λ)∏

0<s≤t

(1 + ∆Ks(λ))e−∆Ks(λ) (3.3.39)

which gives a representation of the characteristic function Gt(λ) = EeiλXt viacumulant function Kt(λ) .

The formula for Gt(λ) takes the simple form:

Gt(λ) = eKt(λ) (3.3.40)

in the case of continuous (in probability) processes X . In this case the functionsBt , Ct and ν = ν((0, t]×A) for A ∈ B(R \ 1) are continuous in time, so that∆Kt(λ) = 0 which leads to the given formula (3.3.40).

For processes X ∈ PII ∩ SemiM the following stochastic integral represen-tation holds (sometimes also called the Ito–Levy representation):

Xt = X0 + Bt + Xct +

∫ t

0

∫h(x) d(µ − ν) +

∫ t

0

∫(x − h(x)) dµ (3.3.41)

where (Xct )t≥0 is a continuous Gaussian martingale with E (Xc

t )2 = Ct .

(b) It is interesting and quite remarkable that for semimartingales it is alsopossible to give analogues of the formulae given above.

Let us first introduce some notation.

Let µ = µ(ω; (0, t] × A) be the measure of jumps of a semimartingale X ,and let ν = ν(ω; (0, t] × A) be a compensator of µ , which can be characterizedas a predictable random measure such that for every non-negative predictable (int for fixed x ∈ R ) function W (ω, t, x)

E

∫ ∞

0

∫R

W (ω, t, x)µ(ω; dt × dx) = E

∫ ∞

0

∫R

W (ω, t, x) ν(ω; dt × dx). (3.3.42)


Here the process (∫ t

0

∫R

min(x2, 1) ν(ω; ds × dx))

∈ A+loc. (3.3.43)

With every semimartingale X and the “truncation” function h(x) =xI(|x| ≤ 1) one may associate a new semimartingale X(h) = (X(h)t)t≥0 =∑

s≤t[∆Xs−h(∆Xs)] which is a sum of “big” jumps of X . Then process X(h) =X− X(h) has bounded jumps and consequently is a special semimartingale which(as any process of such type) admits a decomposition

X(h) = X0 + B(h) + M(h) (3.3.44)

with a predictable process B(h) and a local martingale M(h) .

Local martingale M(h) can be written as the sum M c(h) + Md(h) whereM c(h) is a continuous local martingale and Md(h) is a purely discontinuous localmartingale. Therefore

X = X0 + B(h) + M c(h) + Md(h) + X(h). (3.3.45)

For a given “truncation” function h(x) = xI(|x| ≤ 1) (or any other boundedfunction h = h(x) with a compact support and with linear behavior in the vicinityof x = 0 ) we obtain the following canonical representation of X :


∫ t

0

∫R

h(x) d(µ − ν) +∫ t

0

∫(x − h(x)) dµ (3.3.46)

where we denoted B = B(h) and M c(h) = Xc .

Because Xc ∈ Mc,2loc we get by Doob–Meyer decomposition that there exists

a predictable process C = (Ct)t≥0 such that (Xc)2 − C ∈ Mcloc .

By analogy with the case of processes of the class PII we call the set (B, C, ν)a triplet of the predictable characteristics of the semimartingale X . Let us empha-size that for a process of the class PII a triplet (B, C, ν) consists of deterministicobjects but for a general semimartingale the triplet (B, C, ν) consists of randomobjects of predictable character (that is some kind of “stochastic determinancy”).

6. Now let X be a semimartingale with the triplet (B, C, ν) . For this process,by analogy with (3.3.35), introduce a (predictable) cumulant process


2Ct +

∫R

(eiλx − 1 − iλh(x)

)ν(ω; (0, t] × dx) (3.3.47)

where Bt = Bt(ω) , Ct = Ct(ω) , ν(ω; (0, t]×dx) are predictable. For the process(Kt(λ))t≥0 we consider the stochastic differential equation

dEt(λ) = Et−(λ) dKt(λ), E0(λ) = 1. (3.3.48)


The solution of this equation is given by the well-known stochastic exponential

Et(λ) = eKt(λ)∏s≤t

(1 + ∆Ks(λ)

)e−∆Ks(λ). (3.3.49)

It is remarkable that in the case when ∆Kt(λ) = −1 , t > 0 , the process(eiλXt

Et(λ)

)t≥0

∈ Mloc. (3.3.50)

This formula can be considered as a semimartingale analogue of the Kolmogorov–Levy–Khintchine formula for processes with independent increments (see Subsec-tion 4.6).

3.4. Stochastic differential equations

In the class of semimartingales X = (Xt)t≥0 let us distinguish a certain classof processes which have (for the fixed truncation function h ) the triplets of thefollowing special form:

Bt(ω) =∫ t

0

b(s, Xs(ω)) ds, (3.4.1)

Ct(ω) =∫ t

0

c(s, Xs(ω)) ds, (3.4.2)

ν(ω; dt × dy) = dt Kt(Xt(ω); dy) (3.4.3)

where b and c are Borel functions and Kt(x; dy) is a transition kernel withKt(x; 0) = 0 .

Such processes X are usually called diffusion processes with jumps. If ν = 0then the process X is called a diffusion process.

How can one construct such processes (starting with stochastic processes andmeasures of “simple” structure)?

By a process of “simple” structure we shall consider a standard Wiener pro-cess (also called a Brownian motion) and as a random measure we shall takea Poisson random measure p(dt, dx) with the compensator (intensity) q(dt, dx) =dt F (dx) , x ∈ R , where F = F (dx) is a positive σ -finite measure. (It is assumedthat these objects can be defined on the given stochastic basis.)

We shall consider stochastic differential equations of the following form:

dYt = β(t, Yt) dt + γ(t, Yt) dWt + h(δ(t, Yt−; z)

)(p(dt, dz) − q(dt, dz)

)(3.4.4)

+ h′(δ(t, Yt−; z))p(dt, dz)

where h′(x) = x − h(x) , h(x) = xI(|x| ≤ 1) and β(t, y) , γ(t, y) , δ(t, y; z) areBorel functions. (Notice that if the measure p has a jump at a point (t, z) then


∆Yt = δ(t, Yt−; z) .) Although the equation (3.4.4) is said to be “differential” oneshould, in fact, understand it in the sense that the corresponding integral equationholds:

Yt = Y0 +∫ t

0

β(s, Ys) ds +∫ t

0

γ(s, Ys) dWs (3.4.5)

+∫ t

0

∫R

h(δ(s, Ys−; z)

)(p(ds, dz) − q(ds, dz)

)+∫ t

0

∫R

h′(δ(s, Ys−; z))p(ds, dz).

When considering the question about the existence and uniqueness of solu-tions to such equations it is common to distinguish two types of solutions:

(a) solutions-processes (or strong solutions) and(b) solutions-measures (or weak solutions).

In the sequel in connection with the optimal stopping problems we shallconsider only strong solutions. Therefore we cite below the results only for thecase of strong solutions. (For more details about strong and weak solutions see[106] and [127]–[128].)

In the diffusion case when there is no jump component, the following classicalresult is well known.

Theorem 3.2. Consider the stochastic differential equation

(4.2′) dYt = β(t, Yt) dt + γ(t, Yt) dWt

where Y0 = const and the coefficients β(t, y) and γ(t, y) satisfy the local Lips-chitz condition and the condition of linear growth respectively:

(1) for any n ≥ 1 there exists a constant θn such that

|β(s, y) − β(s, y′)| ≤ θn|y − y′|, |γ(s, y) − γ(s, y′)| ≤ θn|y − y′| (3.4.6)

for s ≤ n and |y| ≤ n , |y′| ≤ n; and

(2) for any n ≥ 1

|β(s, y)| ≤ θn(1 + |y|), |γ(s, y)| ≤ θn(1 + |y|) (3.4.7)

for s ≤ n and |y| ≤ n .

Then (on any stochastic basis (Ω,F , (Ft)t≥0, P) on which the Wiener processis defined) a strong solution (i.e. solution Y = (Yt)t≥0 such that Yt is Ft -measurable for each t ≥ 0 ) exists and is unique (up to stochastic indistinguisha-bility).


Theorem 3.3. In the general case of equation (3.4.4) (in the presence of jumps)let us suppose, in addition to the assumptions of the previous theorem, that thereexist functions ρn(x), n ≥ 1, with

∫ρ2

n(x)F (dx) < ∞ such that

|h δ(s, y, x) − h δ(s, y′, x)| ≤ ρn(x)|y − y′|, (3.4.8)

|h′ δ(s, y, x) − h′ δ(s, y′, x)| ≤ ρ2n(x)|y − y′|, (3.4.9)

|h δ(s, y, x)| ≤ ρn(x)(1 + |y|), (3.4.10)

|h′ δ(s, y, x)| ≤ (ρ2n(x) ∧ ρ4

n(x))(1 + |y|) (3.4.11)

for s ≤ n and |y|, |y′| ≤ n .Then (on any stochastic basis (Ω,F , (Ft)t≥0, P) on which the Wiener process

and the Poisson random measure are defined) a strong solution exists and is unique(up to stochastic indistinguishability).

3.5. A local time-space formula

Let X = (Xt)t≥0 be a continuous semimartingale, let c : R+ → R be a continuousfunction of bounded variation, and let F : R+×R → R be a continuous functionsatisfying:

F is C1,2 on C1, (3.5.1)

F is C1,2 on C2 (3.5.2)

where C1 and C2 are given as follows:

C1 = (t, x) ∈ R+×R : x > c(t) , (3.5.3)C2 = (t, x) ∈ R+×R : x < c(t) . (3.5.4)

Then the following change-of-variable formula holds (for a proof see [163]):

F (t, Xt) = F (0, X0) +∫ t

0

12

(Ft(s, Xs+)+Ft(s, Xs−)

)ds (3.5.5)

+∫ t

0

12

(Fx(s, Xs+)+Fx(s, Xs−)

)dXs

+12

∫ t

0

Fxx(s, Xs) I(Xs =c(s)) d〈X, X〉s

+12

∫ t

0

(Fx(s, Xs+)−Fx(s, Xs−)

)I(Xs = c(s)) dc

s(X)

where cs(X) is the local time of X at the curve c given by

cs(X) = P -lim

ε↓012ε

∫ s

0

I(c(r)− ε < Xr < c(r)+ε

)d〈X, X〉r (3.5.6)


and dcs(X) refers to integration with respect to the continuous increasing function

s → cs(X) .

Moreover, if X solves the stochastic differential equation

dXt = b(Xt) dt + σ(Xt) dBt (3.5.7)

where b and σ are locally bounded and σ ≥ 0 , then the following condition:

P(Xs = c(s)

)= 0 for s ∈ (0, t] (3.5.8)

implies that the first two integrals in (3.5.5) can be simplified to read

F (t, Xt) = F (0, X0) +∫ t

0

(Ft+LXF )(s, Xs) I(Xs =c(s)) ds (3.5.9)

+∫ t

0

Fx(s, Xs) σ(Xs) I(Xs =c(s)) dBs

+12

∫ t

0


)I(Xs = c(s)) dc

s(X)

where LXF = bFx +(σ2/2)Fxx is the action of the infinitesimal generator LX onF .

Let us briefly discuss some extensions of the formulae (3.5.5) and (3.5.9)needed below. Assume that X solves (3.5.7) and satisfies (3.5.8), where c : R+ →R is a continuous function of bounded variation, and let F : R+×R → R be acontinuous function satisfying the following conditions instead of (3.5.1)–(3.5.2)above:

F is C1,2 on C1 ∪ C2, (3.5.10)Ft + LXF is locally bounded, (3.5.11)x → F (t, x) is convex, (3.5.12)t → Fx(t, b(t)±) is continuous. (3.5.13)

Then it can be proved that the change-of-variable formula (3.5.9) still holds (cf.[163]). In this case, even if Ft is to diverge when the boundary c is approachedwithin C1 , this deficiency is counterbalanced by a similar behaviour of Fxx

through (3.5.11), and consequently the first integral in (3.5.9) is still well definedand finite. [When we say in (3.5.11) that Ft + LXF is locally bounded, we meanthat Ft + LXF is bounded on K ∩ (C1 ∪ C2) for each compact set in R+×R .]The condition (3.5.12) can further be relaxed to the form where Fxx = F1 + F2

on C1 ∪C2 where F1 is non-negative and F2 is continuous on R+×R . This willbe referred to in Chapter VII as the relaxed form of (3.5.10)–(3.5.13). For moredetails on this and other extensions see [163]. For an extension of the change-of-variable formula (3.5.5) to general semimartingales (with jumps) and local timeon surfaces see [166].


4. Markov processes

4.1. Markov sequences (chains)

1. A traditional approach to the notion of Markov sequence (chain) i.e. discrete-time Markov process—as well as a martingale approach—assumes that we aregiven a filtered probability space

(Ω,F , (Fn)n≥0, P) (4.1.1)

and a phase (state) space (E, E) , i.e. a measurable space E with a σ -algebra Eof its subsets such that one-point sets x belong to E for all x ∈ E .

A stochastic sequence X = (Xn,Fn)n≥0 is called a Markov chain (in awide sense) if the random variables Xn are Fn/E -measurable and the followingMarkov property (in a wide sense) holds:

P(Xn+1 ∈ B | Fn)(ω) = P(Xn+1 ∈ B |Xn)(ω) P -a.s. (4.1.2)

for all n ≥ 0 and B ∈ E (instead of P(Xn+1 ∈ B |Xn)(ω) one often writesP(Xn+1 ∈ B |Xn(ω)) ).

When Fn = FXn ≡ σ(X0, X1, . . . , Xn) , one calls the property (4.1.2) a

Markov property (in a strict sense) and X = (Xn)n≥0 a Markov chain.

From now on it is assumed that the phase space (E, E) is Borel (see e.g.[106]). Under this assumption, it is well known (see [199, Chap. II, § 7]) that thereexists a regular conditional distribution Pn(x; B) such that ( P -a.s. )

P(Xn ∈ B |Xn−1(ω)) = Pn(Xn−1(ω); B), B ∈ E , n ≥ 1. (4.1.3)

In the Markov theory, functions Pn(x; B) are called (Markov) transitionfunctions (from E into E ) or Markov kernels. If Pn(x; B) , n ≥ 1 , do notdepend on n ( = P (x; B) ), the Markov chain (in a wide or strict sense) is said tobe time-homogeneous.

Besides a transition function, another important characteristic of a Markovchain is its initial distribution π = π(B) , B ∈ E :

π(B) = P(X0 ∈ B). (4.1.4)

It is clear that the collection (π, P1, P2, . . .) (or (π, P ) in the time-homogeneouscase) determines uniquely the probability distribution i.e. Law(X | P) of a Markovsequence X = (X0, X1, . . .) .

2. In Chapter I (Section 1.2), when exposing results on optimal stopping, wehave not taken a traditional but more “up-to-date” approach based on the idea

Section 4. Markov processes 77

that the object to start with is neither (Ω,F , (Fn)n≥0, P) nor X = (X0, X1, . . .) ,but a collection of “transition functions” (P1, P2, . . .) , Pn = Pn(x; B) , which mapE into E where (E, E) is a phase space. (In the “time-homogeneous” case onehas to fix only one transition function P = P (x; B) .)

Starting from the collection (P1, P2, . . .) one can construct a family of prob-ability measures Px : x ∈ E on the space (Ω,F) = (E∞, E∞) (e.g. by theIonescu-Tulcea theorem) with respect to which the sequence X = (X0, X1, . . .) ,such that Xn(ω) = xn if ω = (x0, x1, . . .) , is a Markov chain (in a strict sense)for each fixed x ∈ E , and for which Px(X0 = x) = 1 (the Markov chain startsat x ). If π = π(B) is a certain “initial” distribution we denote by Pπ the newdistribution given by Pπ(A) =

∫E Px(A)π(dx) for A ∈ E∞ .

Relative to Pπ it is natural to call the sequence X a Markov chain with theinitial distribution π (i.e. Pπ(X0 ∈ B) = π(B) , B ∈ E ).

3. To expose the theory of Markov chains the following notions of shift op-erator θ and their iterations θn and θτ ( τ is a Markov time) prove to be veryuseful.

An operator θ : Ω → Ω is called a shift operator if for each ω = (x0, x1, . . .)

θ(ω) = (x1, x2, . . .) (4.1.5)

or in other words(x0, x1, . . .)

θ−→ (x1, x2, . . .) (4.1.6)

(i.e. θ shifts the trajectory (x0, x1, . . .) to the left for one position).

Let θ0 = I where I is the unit (identical) transformation (i.e. θ0(ω) = ω ).We define the n -th ( n ≥ 1 ) iteration θn of an operator θ by the formula

θn = θn−1 θ ( = θ θn−1) (4.1.7)

i.e. θn(ω) = θn−1(θ(ω)) .

If τ = τ(ω) is a Markov time ( τ(ω) ≤ ∞ ), one denotes by θτ the operatorwhich acts only on the set Ωτ = ω : τ(ω) < ∞ so that θτ = θn if τ = n , i.e.for all ω such that τ(ω) = n one has

θτ (ω) = θn(ω). (4.1.8)

If H = H(ω) is an F -measurable function (e.g. τ = τ(ω) or Xm = Xm(ω) )one denotes by H θn the function

(H θn)(ω) = H(θn(ω)). (4.1.9)

For an F -measurable function H = H(ω) and a Markov time σ = σ(ω)one defines H θσ only on the set Ωσ = ω : σ(ω) < ∞ and in such a way that


if σ(ω) = n thenH θσ = H θn, (4.1.10)

i.e. if ω ∈ σ(ω) = n then

(H θσ)(ω)(H θn)(ω) = H(θn(ω)). (4.1.11)

In particular

Xm θn = Xm+n, Xm θσ = Xm+σ (on Ωσ ) (4.1.12)

and for finite Markov times τ and σ ,

Xτ θσ = Xτθσ+σ. (4.1.13)

With operators θn : Ω → Ω one can associate the inverse operators θ−1n :

F → F acting in such a way that if A ∈ F then

θ−1n (A) = ω : θn(ω) ∈ A. (4.1.14)

In particular, if A = ω : Xm(ω) ∈ B with B ∈ E , then

θ−1n (X−1

m (B)) = X−1m+n(B). (4.1.15)

4. The Markov property (4.1.2) in the case Fn = FXn (i.e. the Markov

property in a strict sense) and P = Px , x ∈ E , can be written in a little bit moregeneral form:

Px

(Xn+m ∈ B | FX

n

)(ω) = PXn(ω)(Xm ∈ B) Px -a.s. (4.1.16)

If we use the notation of (4.1.3) above and put H(ω) = IB(Xm(ω)) where IB(x)is the indicator of the set B , then, because of

(H θn)(ω) = H(θn(ω)

)= IB

(Xm(θn(ω))

)= IB

(Xn+m(ω)

)(4.1.17)

we get thatEx(H θn | FX

n )(ω) = EXn(ω)H Px -a.s. (4.1.18)

From this, using standard “monotone class” arguments one obtains the followinggeneralized Markov property which is very useful: if H = H(ω) is a bounded (ornon-negative) F -measurable function, then for any initial distribution π and forany n ≥ 0 and x ∈ E ,

Eπ(H θn | FXn )(ω) = EXn(ω)H Px -a.s. (4.1.19)

It is worth emphasizing that EXn(ω)H should be understood as follows: first wetake ψ(x) = ExH and then, by definition, assume that EXn(ω)H = ψ(Xn(ω)) .


Integrating both sides of (4.1.16) we get the celebrated Chapman–Kolmogorov(or Kolmogorov–Chapman) equations (see the original papers [28] and [111]):

Px(Xn+m ∈ B) =∫

E

Py(Xm ∈ B)Px(Xn ∈ dy) (4.1.20)

for x ∈ E and B ∈ E .

5. The property (4.1.16) admits further a very useful generalization calledthe strong Markov property which is formulated as follows. Let (Hn)n≥0 be asequence of bounded (or non-negative) F -measurable functions, and let τ be afinite Markov time, then for any initial distribution π ,

Eπ(Hτ θτ | FXτ )(ω) = ψ(τ(ω), Xτ(ω)(ω)) Pπ -a.s. (4.1.21)

where ψ(n, x) = ExHn . (Here Hτθτ means that if τ(ω) = n then (Hτθτ )(ω) =(Hn θn)(ω) .)

6. Note also two properties of stopping times which prove to be useful indifferent proofs related to the problems of optimal stopping.

Suppose that B ∈ E and

τB = inf n ≥ 0 : Xn ∈ B , σB = inf n > 0 : Xn ∈ B (4.1.22)

are finite and γ is a stopping time. Then τB and σB are stopping times and soare

γ + τB θγ = inf n ≥ γ : Xn ∈ B , (4.1.23)γ + σB θγ = inf n > γ : Xn ∈ B . (4.1.24)

In particular, if γ ≤ τB then from (4.1.23) we get the following formula:

γ + τB θγ = τB . (4.1.25)

(These properties are also valid when γ , τB and σB can take infinite values andthe sets in (4.1.23) and (4.1.24) are empty.)

4.2. Elements of potential theory (discrete time)

1. It was mentioned above that, in the modern theory of time-homogeneous Markovchains X = (Xn)n≥0 with values in a certain phase space (E, E) , the probabilitydistribution of X is uniquely defined by its initial distribution π = π(dx) andtransition function P = P (x; B) , x ∈ E , B ∈ E . Moreover, the probabilitydistribution Px on (E∞, E∞) is uniquely defined by the transition function P =P (x; B) itself.

It is noteworthy that the notion of transition function (or Markov kernel)underlies the field of mathematical analysis (analytic non-probabilistic) which is


called potential theory. Thus it is not surprising that there exists a close connectionbetween this theory and the theory of time-homogeneous Markov chains, and thatthis connection proves to be mutually fruitful.

From the standpoint of optimal stopping problems that we are interestedin, a discussion of some aspects of the potential theory can be very useful sinceboth the material of Chapter I and our further exposition demonstrate that theDirichlet problem in potential theory is related to the Stefan problem (with movingboundary) in optimal stopping theory. We can go even further and say that optimalstopping problems can be interpreted as optimization problems of potential theory(in particular for the Dirichlet problem)!

Let us associate with a transition function P = P (x; B) , x ∈ E , B ∈ E , thelinear (one-step) transition operator Pg acting on functions g = g(x) as follows:

(Pg)(x) =∫

E

g(y)P (x; dy) (4.2.1)

(one often writes Pg(x) ). As the domain DP of the operator P we consider the setof those E -measurable functions g = g(x) for which the integral

∫E

g(y)P (x; dy)is well defined for all x ∈ E . For example, the set E+ of non-negative E -measurable functions is contained in DP , and so is the set bE+ of boundedE -measurable functions.

With the notation I for the unit (identical) operator ( Ig(x) = g(x) ) onecan introduce the ( n -step) transition operators Pn , n ≥ 1 , by the formulaPn = P(Pn−1) with P0 = I . It is clear that for g ∈ DP ,

Png(x) = Exg(Xn). (4.2.2)

If τ is a Markov time (with respect to the filtration (FXn )n≥0 ), we shall

denote by Pτ the operator acting on functions g ∈ DP by the formula

Pτg(x) = Ex

[I(τ < ∞)g(Xτ )

]. (4.2.3)

If g(x) ≡ 1 then Pτ1(x) = Pxτ < ∞ . From operators Pn , n ≥ 0 , one canconstruct the important (in general unbounded) operator

U =∑n≥0

Pn (4.2.4)

called the potential of the operator P (or of the corresponding Markov chain).

If g ∈ E+ it is clear that

Ug =∑n≥0

Png = (I + PU)g (4.2.5)


or in other wordsU = I + PU. (4.2.6)

The function Ug is usually referred to as the potential of the function g .

Putting g(x) = IB(x) we find that

UIB(x) =∑n≥0

ExIB(Xn) = ExNB (4.2.7)

where NB is the number of visits by X to the set B ∈ E . When x ∈ Eis fixed, the function U(x, B) = UIB(x) is a measure on (E, E) . It is calleda potential measure. If B = y i.e. B is a one-point set where y ∈ E , thefunction U(x, y) is usually denoted by G(x, y) and called the Green function.The illustrative meaning of the Green function is clear:

G(x, y) = ExNy (4.2.8)

i.e. the average number of visits to a state y given that X0 = x . It is clear thatthe Green function G(x, y) admits the representation

G(x, y) =∑n≥0

p(n; x, y) (4.2.9)

where p = (n; x, y) = Px(Xn = y) and consequently for g(y) ≥ 0 the potentialUg of a function g is defined by the formula

Ug =∑

y

g(y)G(x, y). (4.2.10)

Remark 4.1. The explanation for the name “potential of the function g ” given toUg(x) lies in analogy of Ug(x) with the Newton potential f(x) for the “mass”distribution with density g(y) , which, e.g. in the case x ∈ R3 , has the form

f(x) =12π

∫R3

g(y) dy

‖x − y‖ (4.2.11)

where ‖x−y‖ is the distance between the points x and y . (According to the lawof Newtonian attraction, the “mass” in R3 exerts influence upon a “unit mass”at point x which is proportional to the gradient of the function f(x) . Undernonrestrictive assumptions on the function g(x) the potential f(x) solves thePoisson equation

12∆f(x) = −g(x) (4.2.12)

where ∆ is the Laplace operator.)In the case of simple symmetrical random walks on the lattice Zd = 0,±1,

±2, . . . d one hasG(x, y) ∼ c3

‖x − y‖ , c3 = const. (4.2.13)


for large ‖x − y‖ , and consequently, according to the formula Ug(x) =∑

y g(y)G(x, y) given above, we find that for ‖x‖ → ∞ , at least when the function g(y)does not vanishes everywhere except a finite numbers of points, one finds that

Ug(x) ∼ c3

∑y

g(y)‖x − y‖ . (4.2.14)

Thus the behavior of the potential Ug(x) for large ‖x‖ is analogous to that ofthe Newton potential f(x) .

More details regarding the preceding considerations may be found in [55,Chap. 1, § 5].

2. Let us relate to the operator P another important operator

L = P − I. (4.2.15)

In the theory of Markov processes this operator is called a generating operator (ofa time-homogeneous Markov chain with the transition function P = P (x; B) ).The domain DL of the operator L is the set of those E -measurable functionsg = g(x) for which the expression Pg − g is well defined.

Let a function h belong to E+ (i.e. h is E -measurable and takes its valuesin R+ ). Its potential H = Uh satisfies the relation

H = h + PH (4.2.16)

(because U = I + PU ). Thus if H ∈ DL then the potential H solves the Poissonequation

LV = −h. (4.2.17)

Suppose that W ∈ E+ is another solution to the equation W = h + PW(or to the equation LW = −h with W ∈ DL ). Because W = h + PW ≥ h , byinduction we find that

W ≥n∑

k=0

Pkh for all n ≥ 1 (4.2.18)

and so W ≥ H . Thus the potential H is recognizable by its property to providethe minimal solution to the system V = h + PV . Recall once again that

H = Uh = Ex

∞∑k=0

h(Xk). (4.2.19)

3. In the theory of optimal stopping a significant role is played by anotherimportant notion, namely the notion of excessive function.


A function f = f(x) , x ∈ E , belonging to the class E+ is said to be exces-sive or superharmonic for an operator P (or P -excessive or P -superharmonic)if

Pf ≤ f (Lf ≤ 0 if Lf is well defined) (4.2.20)

i.e. Exf(X1) ≤ f(x) , x ∈ E .

According to this definition the potential H = Uh of a function h ∈ E+ isan excessive function (since by (4.2.16) we have H = h + PH ≥ PH ).

A function f = f(x) , x ∈ E , from the class E+ is called harmonic (orinvariant) if

Pf = f (Lf = 0 if Lf is well defined) (4.2.21)

i.e. Exf(X1) = f(x) , x ∈ E .

It is important to be aware of the following connection between the notionsof potential theory, theory of Markov chains, and martingale theory.

Let X = (Xn)n≥0 be a one-dimensional time-homogeneous Markov chainwith initial distribution π and transition function P = P (x; B) generating theprobability distribution Pπ on (E∞, E∞) , and let f = f(x) be a P -excessivefunction. Then the sequence

Y = (Yn,FXn , Pπ)n≥0 (4.2.22)

with Yn = f(Xn) is a non-negative (generalized) supermartingale, i.e.

Yn is FXn -measurable and Yn ≥ 0 so that EπYn exists in [0,∞]; (4.2.23)

Eπ(Yn+1 | FXn ) ≤ Yn Pπ-a.s. (4.2.24)

for all n ≥ 0 . If EπYn < ∞ for all n ≥ 0 , then this sequence is an (ordinary)supermartingale.

4. The potential H(x) = Uh(x) of a non-negative function h = h(x) (fromthe class E+ or E+ ) satisfies (4.2.16) and thus solves the Wald–Bellman inequal-ity

H(x) ≥ max(h(x), PH(x)), (4.2.25)

i.e. the potential H(x) of the function h(x) is

(1) a majorant for the function h(x) , and

(2) an excessive function.

In other words, the potential H(x) of the function h(x) is an excessive majorantof this function.

In Chapter I we have already seen that minimal excessive majorants play anextremely important role in the theory of optimal stopping. We now show how the


potential theory answers the question as how to find a minimal excessive majorantof the given non-negative E -measurable function g = g(x) .

To this end introduce the operator Q acting on such functions by the formula

Qg(x) = max(g(x), Pg(x)

). (4.2.26)

Then the minimal excessive majorant s(x) of the function g(x) is given by

s(x) = limn

Qng(x). (4.2.27)

(See Corollary 1.12 where instead of Q , g and s we used the notation Q , Gand V .)

Note that s = s(x) satisfies the Wald–Bellman equation

s(x) = max(g(x), Ps(x)

)(4.2.28)

(cf. the Wald–Bellman inequality (4.2.25)). The equation (4.2.28) implies, in par-ticular, that if the function s ∈ DL and

Cg = x : s(x) > g(x), (4.2.29)Dg = x : s(x) ≤ g(x) ( = E \ Cg) (4.2.30)

thenLs(x) = 0,

s(x) = g(x),x ∈ Cg,

x ∈ Dg.(4.2.31)

The system (4.2.31) is directly connected with the optimal stopping problem

s(x) = sup Exg(Xτ ) (4.2.32)

considered throughout the monograph as already illustrated in Chapter I (see e.g.Theorem 1.11).

5. In potential theory a lot of attention is paid to solving the Dirichlet problem(the “first boundary problem”) for an operator P : Find a non-negative functionV = V (x) , x ∈ E (from one or another class of functions, E+ , E+ , bE+ etc.)such that

V (x) =

PV (x) + h(x), x ∈ C,

g(x), x ∈ D.(4.2.33)

Here C is a given subset of E (“domain”), D = E \C , and h as well as g arenon-negative E -measurable functions.

If we consider only solutions V which belong to DL , then the system (4.2.33)is equivalent to the following system:

LV (x) = −h(x),V (x) = g(x),

x ∈ C,

x ∈ D.(4.2.34)


The equation LV (x) = −h(x) , x ∈ C , as it was mentioned above, bears thename of a Poisson equation in the set C and the problem (4.2.34) itself is calleda Dirichlet problem for the Poisson equation (in the set C ) with a given functiong (in the set D ).

It is remarkable that the solution to this (analytic i.e. non-probabilistic)problem can be obtained by a probabilistic method if we consider a Markov chainX = (Xn)n≥0 constructed from the same transition function P = P (x; B) thathas been used to construct the operator P .

Namely, let X = (Xn)n≥0 be such a Markov chain, and let

τ(D) = infn ≥ 0 : Xn ∈ D. (4.2.35)

(As usual, throughout we put τ(D) = ∞ if the set in (4.2.35) is empty.)

From the theory of Markov processes it is known (see [53], [55]) that if thefunctions h and g belong to the class E+ , then a solution to the Dirichlet problem(4.2.34) does exist and its minimal (non-negative) solution VD(x) is given by

VD(x) = Ex

[I(τ(D) < ∞)g(Xτ(D))

]+ IC(x)Ex

[ τ(D)−1∑k=0

h(Xk)]. (4.2.36)

It is useful to mention some particular cases of the problem (4.2.34) wheng(x) ≥ 0 .

(a) If h = 0 , we seek a function V = V (x) which is harmonic in C (i.e.LV (x) = 0 ) and coincides with the function g in D . In this case the minimalnon-negative solution VD(x) is given by

VD(x) = Ex

[I(τ(D) < ∞)g(Xτ(D))

]. (4.2.37)

In particular, if g(x) ≡ 1 , x ∈ D , then

VD(x) = Pxτ(D) < ∞. (4.2.38)

This result is interesting in the “reverse” sense: the probability Pxτ(D) < ∞to reach the set D in finite time, under the assumption X0 = x ∈ C , is harmonic(as a function of x ∈ C ).

(b) If g(x) = 0 , x ∈ D , and h(x) = 1 , x ∈ C , i.e. we consider the system

V (x) =

PV (x) + 1, x ∈ C,

0, x ∈ D,(4.2.39)

or, equivalently, the system

LV (x) = −1,

V (x) = 0,

x ∈ C,

x ∈ D,(4.2.40)


with V ∈ DL , then the minimal non-negative solution VD(x) is given by

VD(x) = IC(x)Ex

[ τ(D)−1∑k=0

1]

=

Exτ(D), x ∈ C,

0, x ∈ D.(4.2.41)

Thus the expectation Exτ(D) of the time τ(D) of the first entry into the set Dis the minimal non-negative solution of the system (4.2.40).

6. In the class of Markov chains that describe random walks in the phasespace (E, E) , a special place (especially due to analogies with Brownian motion)is taken by simple symmetrical random walks in

E = Zd = 0 ± 1,±2, . . .d (4.2.42)

where d = 1, 2, . . . . One can define such walks X = (Xn)n≥0 constructively byspecifying

Xn = x + ξ1 + · · · + ξn (4.2.43)

where the random d -dimensional vectors ξ1, ξ2, . . . defined on a certain proba-bility space are independent and identically distributed with

P(ξ1 = e) = (2d)−1 (4.2.44)

( e = (e1, . . . , ed) is a standard basis unit vector in Rd i.e. each ei equals either±1 or 0 and ‖e‖ ≡ |e1| + · · · + |ed| = 1 ).

The corresponding operator P has the very simple structure

Pf(x) = Exf(x + ξ1) =12d

∑‖e‖=1

f(x + e), (4.2.45)

and, consequently, the generating operator L = P− I (called a discrete Laplacianand denoted by ∆ ) has the following form:

∆f(x) =12d

∑|e|=1

(f(x + e) − f(x)). (4.2.46)

Here it is natural to reformulate the Dirichlet problem stated above by takinginto account that the exit from the set C ⊆ Zd is only possible through the“boundary” set

∂C = x ∈ Zd : x ∈ C and ‖x − y‖ = 1 for some y ∈ C . (4.2.47)

This fact leads to the following standard formulation of the Dirichlet problem:Given a set C ⊆ Zd and functions h = h(x) , x ∈ C , g = g(x) , x ∈ ∂C , find afunction V = V (x) such that

∆V (x) = −h(x),V (x) = g(x),

x ∈ C,

x ∈ ∂C.(4.2.48)


If the set C consists of a finite number of points then Px(τ(∂C) < ∞) = 1 for allx ∈ C where τ(∂C) = inf n ≥ 0 : Xn ∈ ∂C . This allows one to prove that aunique solution to the problem (4.2.48) for x ∈ C ∪ ∂C is given by the followingformula:

V∂C(x) = Exg(Xτ(∂C)) + IC(x)Ex

[ τ(∂C)−1∑k=0

h(Xk)]. (4.2.49)

In particular, if h = 0 then a unique function which is harmonic in C andequal to g(x) for x ∈ ∂C is given by

V∂C(x) = Exg(Xτ(∂C)). (4.2.50)

Let us also cite some results for the (classical) Dirichlet problem:

∆V (x) = 0,

V (x) = g(x),x ∈ C,

x ∈ ∂C,(4.2.51)

when the set C is unbounded.

If d ≤ 2 then Px(τ(∂C) < ∞) = 1 by the well-known Polya (“recur-rency/transiency”) theorem, and for a bounded function g = g(x) , the solution inthe class of bounded functions on C ∪ ∂C exists, is unique, and can be given bythe same formula as in (4.2.50) above.

It should be noted that even in the case of bounded functions g = g(x)the problem (4.2.51) can have (more than) one unbounded solution. The followingexample is classical.

Let d = 1 , C = Z \ 0 and consequently ∂C = 0 . Taking g(0) = 0we see that every unbounded function V (x) = αx , α ∈ R , solves the Dirichletproblem ∆V (x) = 0 , x ∈ Z \ 0 , and V (0) = g(0) .

When d ≥ 3 , the answer to the question on the existence and uniquenessof a solution to the Dirichlet problem ( ∆V (x) = 0 , x ∈ C , and V (x) = g(x) ,x ∈ ∂C ), even in the case of bounded functions, depends essentially on whetherthe condition Pxτ(∂C) < ∞ = 1 is fulfilled for all x ∈ C . If this is the case,then in the class of bounded functions a solution exists, is unique, and can begiven by the same formula as in (4.2.50) above.

However, if the condition Pxτ(∂C) < ∞ = 1 , x ∈ C , does not hold then(in the case of bounded functions g = g(x) , x ∈ ∂C ) all bounded solutions tothe Dirichlet problem ( ∆V (x) = 0 , x ∈ C , and V (x) = g(x) , x ∈ ∂C ) aredescribed by functions of the following form:

V(α)∂C (x) = Ex

[I(τ(∂C) < ∞)g(Xτ(∂C))

]+ αPxτ(∂C) = ∞ (4.2.52)

where α ∈ R . (For more details see e.g. [122].)


4.3. Markov processes (continuous time)

1. Foundations of the general theory of Markov processes were laid down in thewell-known paper [111] of A. N. Kolmogorov entitled “On analytical methods ofprobability theory” (published in 1931).

This was the first work which clarified the deep connection between proba-bility theory and mathematical analysis and initiated the construction and devel-opment of the theory of Markov processes (in continuous time).

In [111] Kolmogorov did not deal with trajectories of the Markov (as we saynow) process under consideration directly. For him the main object were transitionprobabilities

P (s, x; t, A), 0 ≤ s ≤ t, x ∈ E, A ∈ E (4.3.1)

where (E, E) is a phase (state) space ( E = Rd as a rule).

The transition probability P (s, x; t, A) is interpreted as “the probability fora ‘system’ to get at time t to the set A given that at time s ≤ t the systemwas in a state x ”. The main requirement on the collection of transition prob-abilities P (s, x; t, A)—which determines the Markovian character of system’sevolution—is the assumption that the Chapman–Kolmogorov equation holds:

P (s, x; t, A) =∫

E

P (s, x; u, dy)P (u, y; t, A) (0 ≤ s < u < t). (4.3.2)

2. We now assume that E = R and use the notation F (s, x; t, y) = P (s, x; t,(−∞, y]) . Suppose that the density (in y )

f(s, x; t, y) =∂F (s, x; t, y)

∂y(4.3.3)

exists as well as the following limits:

lim∆↓0

1∆

∫ ∞

−∞(y − x)f(s, x; s + ∆, y) dy ( = b(s, x)), (4.3.4)

lim∆↓0

1∆

∫ ∞

−∞(y − x)2f(s, x; s + ∆, y) dy ( = σ2(s, x)), (4.3.5)

lim∆↓0

1∆

∫ ∞

−∞|y − x|2+δf(s, x; s + ∆, y) dy for some δ > 0. (4.3.6)

The coefficients b(s, x) and σ2(s, x) are called differential characteristics (ordrift coefficient and diffusion coefficient respectively) of the corresponding Markovsystem whose evolution has the “diffusion” character. Under these assumptionsKolmogorov derived the backward parabolic differential equation (in (s, x) ):

−∂f

∂s= b(s, x)

∂f

∂x+

12

σ2(s, x)∂2f

∂x2(4.3.7)


and the forward parabolic differential equation (in (t, y) ):

∂f

∂t= − ∂

∂y

[b(t, y)f

]+

12

∂2

∂y2

[σ2(t, y)f

]. (4.3.8)

(Special cases of the latter equation had been considered earlier by A. D. Fokker[68] and M. Planck [172].)

Kolmogorov also obtained the corresponding equations for Markov systemswith finite or countable set of states (for details see [111]).

It is due to all these equations that the approach proposed by Kolmogorovwas named ‘analytical approach’ as reflected in the title of [111].

The 1940–60s saw a considerable progress in investigations of Markov sys-tems. First of all one should cite the works by K. Ito [97]–[99], J. L. Doob [40],E. B. Dynkin [53] and W. Feller [64] in which, along with transition functions, thetrajectories (of Markov processes) had begun to play an essential role.

Starting with the differential characteristics b(s, x) and σ2(s, x) from Ana-lytical methods, K. Ito [97]–[99] constructed processes X = (Xt)t≥0 as solutionsto stochastic differential equations

Xt = b(t, Xt) dt + σ(t, Xt) dBt (4.3.9)

where the “driving” process B = (Bt)t≥0 is a standard Brownian motion (seeSubsection 4.4 below) and σ = +

√σ2 .

The main contribution of K. Ito consists in proving the following: If thedifferential characteristics b and σ satisfy the Lipschitz condition and increaselinearly (in the space variable) then the equation (4.3.9) has a unique (strong)solution X = (Xt)t≥0 which under certain conditions (e.g. if the differential char-acteristics are continuous in both variables) is a diffusion Markov process (in theKolmogorov sense) such that the differential characteristics of the correspondingtransition function

P (s, x; t, A) = P(Xt ∈ A |Xs = x) (4.3.10)

are just the same as b and σ that are involved in the equation (4.3.9).

Actually K. Ito considered d -dimensional processes X = (X1, . . . , Xd) suchthat the corresponding differential equations are of the form

dX it = bi(t, Xt) dt +

d∑j=1

σij(t, Xt) dBjt (4.3.11)


for 1 ≤ i ≤ d . If we use the notation

aij =d∑

k=1

σikσkj (4.3.12)

L(s, x) =d∑

i=1

bi(s, x)∂f

∂xi+

12

d∑i,j=1

aij(s, x)∂2f

∂xi ∂xj(4.3.13)

L∗(t, y) = −d∑

i=1

∂

∂yi

[bi(t, y)f

]+

12

d∑i,j=1

∂2

∂yi ∂yj

[aij(t, y)f

](4.3.14)

then the backward and forward Kolmogorov equations take respectively the fol-lowing form:

− ∂f

∂s= L(s, x)f, (4.3.15)

∂f

∂t= L∗(t, y)f. (4.3.16)

It is important to notice that in the time-homogeneous case—when aij andbi do not depend on the time parameter ( aij = aij(x) , bi = bi(x) )—the followingequality for the transition function holds:

f(s, x; t, y) = f(0, x; t − s, y), 0 ≤ s < t. (4.3.17)

Puttingg(x; t, y) = f(0, x; t, y) (4.3.18)

we find from the backward equation (4.3.15) that g = g(x; t, y) as a function of(x, t) solves the following parabolic equation:

∂g

∂t= L(x)g (4.3.19)

where

L(x)g =d∑

i=1

bi(x)∂g

∂xi+

12

d∑i,j=1

aij(x)∂2g

∂xi ∂xj. (4.3.20)

3. Let us now address a commonly used definition of Markov process [53].When defining such notions as martingale, semimartingale, and similar, we start(see Subsection 4.1 above) from the fact that all considerations take place on acertain filtered probability space

(Ω,F , (Ft)t≥0, P) (4.3.21)


and the processes X = (Xt)t≥0 considered are such that their trajectories areright-continuous (for t ≥ 0 ), have limits from the left (for t > 0 ) and for everyt ≥ 0 the random variable Xt is Ft -measurable (i.e. X is adapted).

When defining a (time-homogeneous) Markov process in a wide sense onealso starts from a given filtered probability space (Ω,F , (Ft)t≥0, P) and says thata stochastic process X = (Xt)t≥0 taking values in a phase space is Markov in awide sense if

P(Xt ∈ B | Fs)(ω) = P(Xt ∈ B |Xs)(ω) P -a.s. (4.3.22)

for all s ≤ t .

If Ft = FXt ≡ σ(Xs, s ≤ t) then the stochastic process X = (Xt)t≥0 is said

to be Markov in a strict sense.

Just as in the discrete-time case (Subsection 4.1), the modern definition ofa time-homogeneous Markov process places emphasis on both trajectories andtransition functions as well as on their relation. To be more precise, we shallassume that the following objects are given:

(A) a phase space (E, E) ;

(B) a family of probability spaces (Ω,F , (Ft)t≥0; Px, x ∈ E) where each Px

is a probability measure on (Ω,F) ;

(C) a stochastic process X = (Xt)t≥0 where each Xt is Ft/E -measurable.

Assume that the following conditions are fulfilled:

(a) the function P (t, x; B) = Px(Xt ∈ B) is E -measurable in x ;

(b) P (0, x; E \ x) = 0 , x ∈ E ;

(c) for all s, t ≥ 0 and B ∈ E , the following (Markov) property holds:

Px(Xt+s ∈ B | Fs) = P (s, Xt; B) P -a.s.; (4.3.23)

(d) the space Ω is rich enough in the sense that for any ω ∈ Ω and h > 0there exists ω′ ∈ Ω such that Xt+h(ω) = Xt(ω′) for all t ≥ 0 .

Under these assumptions the process X = (Xt)t≥0 is said to be a (time-homogeneous) Markov process defined on (Ω,F , (Ft)t≥0; Px, x ∈ E) and the func-tion P (t, x; B) is called a transition function of this process.

The conditions (a) and (c) imply that Px -a.s.

Px(Xt+s ∈ B | Ft) = PXt(Xs ∈ B), x ∈ E, B ∈ E ; (4.3.24)

this property is called the Markov property of a process X = (Xt)t≥0 satisfyingthe conditions (a)–(d).


In general theory of Markov processes an important role is played by thoseprocesses which, in addition to the Markov property, have the following strongMarkov property : for any Markov time τ = τ(ω) (with respect to (Ft)t≥0 )

Px(Xτ+s ∈ B | Fτ ) = P (s, Xτ ; B) ( Px -a.s. on τ < ∞ ) (4.3.25)

where Fτ = A ∈ F : A ∩ τ ≤ t ∈ Ft for all t ≥ 0 is a σ -algebra of eventsobserved on the time interval [0, τ ] .

Remark 4.2. For Xτ(ω)(ω) to be Fτ/E -measurable we have to impose an ad-ditional restriction—that of measurability—on the process X . For example, itsuffices to assume that for every t ≥ 0 the function Xs(ω) , s ≤ t , definesa measurable mapping from ([0, t] × Ω,B([0, t] × Ft) into the measurable space(E, E) .

4. In the case of discrete time and a canonical space Ω whose elements aresequences ω = (x0, x1, . . .) with xi ∈ E we have introduced shift operators θn

acting onto ω = (x0, x1, . . .) by formulae θn(ω) = ω′ where ω′ = (xn, xn+1, . . .) ,i.e. θn(x0, x1, . . .) = (xn, xn+1, . . .) .

Likewise in a canonical space Ω which consists of functions ω = (xs)s≥0

with xs ∈ E it is also useful to introduce shift operators θt , t ≥ 0 , acting ontoω = (xs)s≥0 by formulae θt(ω) = ω′ where ω′ = (xs+t)s≥0 i.e. θt

[(xs)s≥0

]=

(xs+t)s≥0 .

In the subsequent considerations we shall assume that stochastic processesX = (Xt(ω))t≥0 are given on the canonical space Ω , which consists of functionsω = (xs)s≥0 , and that Xs(ω) = xs .

The notions introduced above imply that the “composition” Xs θt(ω) =Xs(θt(ω)) = Xs+t(ω) , and thus the Markov property (4.3.24) takes the form

Px(Xs θt ∈ B | Ft) = PXt(Xs ∈ B) Px -a.s. (4.3.26)

for every x ∈ E and B ∈ E with Ft = σ(Xs, s ≤ t) .

Similarly, the strong Markov property (4.3.25) assumes the form: for anyMarkov time τ ,

Px(Xs θτ ∈ B | Fτ ) = PXτ (Xs ∈ B) ( Px -a.s. on τ < ∞ ) (4.3.27)

for every x ∈ E and B ∈ E where θτ (ω) by definition equals θτ(ω)(ω) ifτ(ω) < ∞ .

The following useful property can be deduced from the strong Markov prop-erty (4.3.27):

Eπ(H θτ | Fτ ) = EXτ H ( Pπ -a.s. on τ < ∞ ) (4.3.28)


for any initial distribution π , any F -measurable bounded (or non-negative) func-tional H = H(ω) and all Markov times τ . Similarly, from the strong Markovproperty one can deduce an analogue of the property (4.1.21).

We conclude this subsection with the remark that many properties presentedin Subsection 4.1 in the case of Markov chains, for example properties (4.1.12),(4.1.13), (4.1.23)–(4.1.25), are also valid for Markov processes in the continuous-time case (with an evident change in notation).

Remark 4.3. Denoting P (s, x; t, B) = P(Xt ∈ B |Xs = x) recall that the con-ditional probabilities P(Xt ∈ B |Xs = x) are determined uniquely only up toa PXs -null set (where PXs( ·) = P(Xs ∈ ·) is the law of Xs ). This means thatin principle there are different versions of transition functions P (s, x; t, B) sat-isfying some or other “good” properties. Among such desired properties one isthat the transition functions satisfy the Chapman–Kolmogorov equations (4.3.2).The Markov property (4.3.22) (for time-homogeneous or time-inhomogeneous pro-cesses) does not guarantee that (4.3.2) holds for all x but only for PXs -almostall x in E . In the case of discrete time and discrete state space, the Chapman–Kolmogorov equations ( p

(n+m)ij =

∑k p

(n)ik p

(m)kj ) are automatically satisfied when

the Markov property holds (for the case of discrete time and arbitrary state space(E, E) see [199, Vol. 2, Chap. VIII, § 1]). In the case of continuous time, however,the validity of the Chapman–Kolmogorov equations is far from being evident.It was shown in [118], nonetheless, that in the case of universally measurable(e.g. Borelian) space (E, E) there always exist versions of transition probabilitiessuch that the Chapman–Kolmogorov equations hold. Taking this into account,and without further mentioning it, in the sequel we shall consider only transitionfunctions for which the Chapman–Kolmogorov equations are satisfied.

4.4. Brownian motion (Wiener process)

1. The process of Brownian motion, also called a Wiener process, is interestingfrom different points of view: this process is both martingale and Markov andhas a magnitude of important applications. In this subsection we give only basicdefinitions and a number of fundamental properties which mainly relate theseprocesses to various stopping times.

A one-dimensional (standard) Wiener process W = (Wt)t≥0 is a processdefined on a probability space (Ω,F , P) satisfying the following properties:

(a) W0 = 0 ;

(b) the trajectories of (Wt)t≥0 are continuous functions;

(c) the increments Wtk−Wtk−1 , Wtk−1 −Wtk−2 , . . . , Wt1 −Wt0 are indepen-

dent (for any 0 = t0 < t1 < · · · < tk , k ≥ 1 );

(d) the random variables Wt −Ws , s ≤ t , have the normal distribution with

E(Wt − Ws) = 0, D(Wt − Ws) = t − s. (4.4.1)


Thus a Wiener process W = (Wt)t≥0 is, by definition, a Gaussian process withindependent increments. It is clear that such a process is Markov (in a wide sense).

A Brownian motion is a process B = (Bt)t≥0 defined on a filtered probabilityspace (Ω,F , (Ft), P) such that:

(α) B0 = 0 ;

(β) the trajectories of B = (Bt)t≥0 are continuous functions;

(γ) the process B = (Bt)t≥0 is a square-integrable martingale with respectto the filtration (Ft)t≥0 (i.e. each Bt is Ft -measurable, E |Bt|2 < ∞and E (Bt | Fs) = Bs for s ≤ t ) such that P -a.s.

E[(Bt − Bs)2 | Fs

]= t − s, s ≤ t. (4.4.2)

The well-known “Levy characterization theorem” (see e.g. [174, p. 150]) impliesthat such a process is Gaussian with independent increments as well as E (Bt−Bs)= 0 and E (Bt − Bs)2 = t − s . Thus B = (Bt)t≥0 is a Wiener process in theabove sense.

The converse, in a certain sense, is also true: If W = (Wt)t≥0 is a Wienerprocess then it can easily be checked that this process is a square-integrable Gaus-sian martingale with respect to the filtration (FW

t )t≥0 .

In the sequel we will not distinguish between these processes. Every time itwill be clear from the context which of the two is meant (if at all relevant).

2. Let us list some basic properties of a Brownian motion B = (Bt)t≥0

assumed to be defined on a filtered probability space (Ω,F , (Ft), P) .

• The probability P(Bt ≤ u) for t > 0 and u ∈ R is determined by

P(Bt ≤ u) =∫ u

−∞ϕt(y) dy (4.4.3)

where

ϕt(y) =1√2πt

e−y2/(2t) (4.4.4)

is a fundamental solution to the Kolmogorov forward equation

∂ϕt(y)∂t

=12

∂2ϕt(y)∂y2

. (4.4.5)

• The density

f(s, x; t, y) =∂P(Bt ≤ y |Bs = x)

∂y, 0 < s < t (4.4.6)


is given by

f(s, x; t, y) =1√

2π(t − s)exp

− (y − x)2

2(t − s)

(4.4.7)

and solves for s < t ,

∂f(s, x; t, y)∂s

= −12

∂2f(s, x; t, y)∂x2

(backward equation) (4.4.8)

and for t > s ,

∂f(s, x; t, y)∂t

=12

∂2f(s, x; t, y)∂y2

(forward equation). (4.4.9)

It is clear that f(0, 0; t, y) = ϕt(y) .

• The joint density

ϕt1,...,tn(y1, . . . , yn) =∂nP(Bt1 ≤ y1, . . . , Btn ≤ yn)

∂y1 · · · ∂yn(4.4.10)

for 0 < t1 < t2 < · · · < tn is given by

ϕt1,...,tn(y1, . . . , yn) (4.4.11)= ϕt1(y1)ϕt2−t1(y2 − y1) · · ·ϕtn−tn−1(yn − yn−1).

• EBt = 0 , EB2t = t , cov(Bs, Bt) = EBsBt = min(s, t) , E |Bt| =

√2t/π .

• The following property of self-similarity hold for any a > 0 :

Law(Bat; t ≥ 0) = Law(a1/2Bt; t ≥ 0). (4.4.12)

• Together with a Brownian motion B = (Bt)t≥0 the following processes:

B(1)t = −Bt for t ≥ 0, (4.4.13)

B(2)t = tB1/t for t > 0 with B

(2)0 = 0, (4.4.14)

B(3)t = Bt+s − Bs for s ≥ 0, (4.4.15)

B(4)t = BT − BT−t for 0 ≤ t ≤ T with T > 0 (4.4.16)

are also Brownian motions.

• For every fixed t ≥ 0 ,

Law(

maxs≤t

Bs

)= Law(|Bt|) (4.4.17)


and hence

E maxs≤t

Bs = E |Bt| =

√2t

π. (4.4.18)

The former assertion may be viewed as a variant of the reflection principle forBrownian motion (Andre [4], Bachelier [7, 1964 ed., p. 64], Levy [125, p. 293]usually stated in the following form: for t > 0 and x ≥ 0 ,

P(

maxs≤t

Bs ≥ x)

= 2P(Bt ≥ x) ( = P(|Bt| ≥ x)) (4.4.19)

whence (4.4.17) can be written as

P(

maxs≤t

Bs ≤ x)

= Φ(

x√t

)− Φ

(−x√t

). (4.4.20)

The property (4.4.20) extends as follows (see [107, p. 368] or [197, pp. 759–760]):for t > 0 , x ≥ 0 , µ ∈ R and σ > 0 ,

P(

maxs≤t

(µs + σBs) ≤ x)

= Φ(

x − µt

σ√

t

)− e2µx/σ2

Φ(−x − µt

σ√

t

). (4.4.21)

This property implies the following useful facts:

P(

maxt≥0

(µt + σBt) ≤ x)

= exp(

2µx

σ2

)if µ < 0, (4.4.22)

P(

maxt≥0

(µt + σBt) ≤ x)

= 0 if µ ≥ 0. (4.4.23)

• The following statement (“Levy distributional theorem”) is a natural ex-tension of the property (4.4.17):

Law(

maxs≤t

Bs − Bt, maxs≤t

Bs; t ≥ 0)

= Law(|Bt|, Lt; t ≥ 0

)(4.4.24)

where Lt is the local time of a Brownian motion B on [0, t] :

Lt = limε↓0

12ε

∫ t

0

I(|Bs| < ε) ds. (4.4.25)

• Let Ta = inft ≥ 0 : Bt = a where a > 0 . Then P(Ta < ∞) = 1 ,ETa = ∞ , and the density

γa(t) =d

dtP(Ta ≤ t) (4.4.26)

is given byγa(t) =

a√2πt3

e−a2/(2t). (4.4.27)


It may be noted that

γa(t) = − ∂

∂aϕt(a). (4.4.28)

• LetTa,b = inf t ≥ 0 : Bt = a + bt (4.4.29)

where a > 0 and b ∈ R . Then the density

γa,b(t) =d

dtP(Ta,b ≤ t) (4.4.30)

is given by

γa,b(t) =a√2πt3

exp− (a + bt)2

2t

(4.4.31)

(see [40, p. 397], [130, p. 526] and also (4.6.17) below). If b = 0 then γa,0(t) = γa(t)(see (4.4.27)).

When b = 0 one should separate the cases b < 0 and b > 0 .

If b < 0 then P(Ta,b < ∞) =∫∞0 γa,b(t) dt = 1 .

If b > 0 then

P(Ta,b < ∞) =∫ ∞

0

γa,b(t) dt = e−2ab. (4.4.32)

• Blumenthal’s 0-1 law for Brownian motion. Suppose that Px , x ∈ R ,are measures on the measurable space of continuous functions ω = (ω(t))t≥0

such that process Bt(ω) = ω(t) , t ≥ 0 , is a Brownian motion starting at x .Denote by (F

t )t≥0 the natural filtration of the process B = (Bt(ω))t≥0 , i.e.F

t = σBs, s ≤ t , t ≥ 0 , and let (F tt )t≥0 be the right-continuous filtration

given byF+

t =⋂s>t

Fs . (4.4.33)

Then for every A from F+0 and for all x ∈ R the probability Px(A) takes only

two values: either 0 or 1.

• Laws of the iterated logarithm. Let B = (Bt)t≥0 be a standard Brownianmotion (i.e. B starts from zero and the measure P = P0 is such that EBt = 0and EB2

t = t for t ≥ 0 ).The law of the iterated logarithm at infinity states that

P

(lim sup

t↑∞

Bt√2t log log t

= 1, lim inft↑∞

Bt√2t log log t

= −1)

= 1. (4.4.34)

The law of the iterated logarithm at zero states that

P

(lim sup

t↓0

Bt√2t log log(1/t)

= 1, lim inft↓0

Bt√2t log log(1/t)

= −1)

= 1. (4.4.35)


3. Consider a measurable space (C,B(C)) consisting of continuous functionsω = (ωt)t≥0 . If this space (C,B(C)) is endowed with a Wiener measure P thenthe canonical process B = (Bt)t≥0 with Bt(ω) = ωt becomes a Wiener process(Brownian motion) starting at time t = 0 from the point ω0 = 0 .

Introduce, for all x ∈ R , the processes Bx = (Bxt (ω))t≥0 by setting

Bxt (ω) = x + Bt(ω) ( = x + ωt). (4.4.36)

Denote by Px the measure on (C,B(C)) induced by this process.

In order to keep ourselves within the framework of notation used in paragraph3 of Subsection 4.3 while defining an “up-to-date” notion of a Markov process,from now on we denote (Ω,F) = (C,B(C)) i.e. we assume that (Ω,F) is themeasurable space of continuous functions ω = (ωt)t≥0 with the Borel σ -algebraB(C) . Let Ft = σωs : s ≤ t , t ≥ 0 , and let Px be measure induced by aWiener measure and mappings sending (ωs)s≥0 to (x + ωs)s≥0 as stated above.

On the constructed filtered spaces

(Ω,F , (Ft); Px, x ∈ R) (4.4.37)

consider the canonical process X = (Xt(ω))t≥0 with Xt(ω) = ωt where ω =(ωt)t≥0 is a trajectory from Ω .

It follows immediately that the conditions (a)–(d), stated in paragraph 3 ofSubsection 4.3 while defining the notion of a Markov process, are fulfilled. Indeed,the definition of measures Px itself implies that the transition function

P(t, x; B) = Px(Xt ∈ B) (4.4.38)

coincides with the probability P(x + Bt ∈ B) which evidently is B -measurablein x for any Borel set B and for any t ≥ 0 .

It is also clear that P (0, x; R \ x) = P(x + B0 ∈ R \ x) = 0 becauseB0 = 0 . To verify the property (c) it suffices to show that if f is a boundedfunction then for any x ∈ R ,

Ex(f(Xt+s) | Ft) = EXtf(Xs) Px -a.s. (4.4.39)

Recall that EXsf(Xt) is the function ψ(x) = Exf(Xt) with Xs inserted in placeof x .

To this end it suffices in turn to prove a somewhat more general assertion: ifg(x, y) is a bounded function then

Ex

[g(Xt, Xt+s − Xt) | Ft

]= ψg(Xt) (4.4.40)

whereψg(x) =

∫R

g(x, y)1√2πs

e−y2/(2s) dy. (4.4.41)


(The equality (4.4.39) results from (4.4.40) if we put g(x, y) = f(x + y) andx = Xt .)

The standard technique for proving formulae of type (4.4.40) may be de-scribed as follows.

Assume first that g(x, y) has the special form g(x, y) = g1(x)g2(y) . Forsuch a function

ψg(x) = g1(x)∫

R

g2(y)1√2πs

e−y2/(2s) dy. (4.4.42)

The left-hand side of (4.4.40) is equal to

g1(Xt)Ex

[g2(Xt+s − Xt) | Ft

]. (4.4.43)

Below we shall prove that

Ex

[g2(Xt+s − Xt) | Ft

]= Exg2(Xt+s − Xt) Px -a.s. (4.4.44)

Then from (4.4.43) we find that for the function g(x, y) = g1(x)g2(y) ,

Ex

[g(Xt, Xt+s − Xt) | Ft

]= g1(Xt)Exg2(Xt+s − Xt) (4.4.45)= g1(Xt)Eg2(Bt+s − Bt) = ψg(Xt) Px-a.s.

Using “monotone class” arguments we obtain that the property (4.4.45) remainsvalid for arbitrary measurable bounded functions g(x, y) . (For details see e.g. [50,Chap. 1, § 1] and [199, Chap. II, § 2].)

Thus it remains only to prove the property (4.4.44). Let A ∈ Ft . Accordingto the definition of conditional expectations we have to show that

Ex

[g2(Xt+s − Xt); A

]= Px(A)Exg2(Xt+s − Xt). (4.4.46)

For the sets A of the form

A = ω : Xt1 ∈ C1, . . . , Xtn ∈ Cn (4.4.47)

where 0 ≤ t1 < · · · < tn ≤ t and Ci are Borel sets, (4.4.46) follows directly fromthe properties of Brownian motion and the fact that Law(X . | Px) = Law(B. +x | P) .

To pass from the special sets A just considered to arbitrary sets A from Ft

one uses “monotone class” arguments (see again the above cited [50, Chap. 1, § 1]and [199, Chap. II, § 2]).

4. The above introduced process of Brownian motion X starting at an arbi-trary point x ∈ R has, besides the established Markov property, the strong Markovproperty which can be formulated in the following (generalized) form (cf. (4.3.25)


and (4.1.16)): If H = H(ω) is a bounded F -measurable functional (F = B(C) ),then for any x ∈ R and any Markov time τ ,

Ex(H θτ | Fτ ) = EXτ H ( Px -a.s. on τ < ∞ ). (4.4.48)

(For a proof see e.g. [50, Chap. 1, § 5].)

5. According to our definition, a time τ is a Markov time with respect to afiltration (Gt)t≥0 if for every t ≥ 0 ,

τ ≤ t ∈ Gt. (4.4.49)

There are also other definitions. For example, sometimes a time τ is said tobe Markov if for all t > 0 ,

τ < t ∈ Gt. (4.4.50)

Sinceτ < t =

⋃n

τ ≤ t − 1

n

∈ Gt (4.4.51)

a Markov time in the sense of the first definition (i.e. one satisfying (4.4.49)) isnecessarily Markov in the second sense. The inverse assertion in general is nottrue. Indeed,

τ ≤ t =⋂n

τ < t + 1

n

∈ G+t (4.4.52)

whereG+

t =⋂u>t

Gu. (4.4.53)

Therefore if G+t ⊃ Gt then the property to be a Markov time in the first sense

(i.e. in the sense of (4.4.49)) in general does not imply this property in the secondsense.

However from (4.4.53) it is clear that if the family (Gt)t≥0 is continuous fromthe right (i.e. G+

t = Gt for all t ≥ 0 ) then both definitions coincide.

Now let us consider a Brownian filtration (Ft)t≥0 where Ft = σ(Xs, s ≤ t) .Form a continuous-from-the-right filtration (F+

t )t≥0 by setting F+t =

⋂u>t Fu .

It turns out that the Markov property (see paragraph 3 above) of a Brownianmotion X = (Xt)t≥0 with respect to the filtration (Ft)t≥0 remains valid forthe larger filtration (F+

t )t≥0 . So when we deal with a Brownian motion thereis no restriction to assume from the very beginning that the initial filtration isnot (Ft)t≥0 but (F+

t )t≥0 . This assumption, as was explained above, simplifiesa verification of whether one or another time τ is Markov. The strong Markovproperty also remains valid when we pass to the filtration (F+

t )t≥0 (see e.g. [50,Chap. 1]).


4.5. Diffusion processes

1. Diffusion is widely interpreted as a (physical and mathematical) model de-scribing the evolution of a “particle” moving continuously and chaotically. In thisconnection it is worthwhile to mention that the term superdiffusion is related tothe random motion of a “cloud of particles” (see e.g. [54]). In the mathemati-cal theory of stochastic processes it is the Brownian motion process (i.e. Wienerprocess) which is taken as a basic diffusion process.

In the modern theory of Markov processes, which was initiated, as we alreadymentioned above, by the Kolmogorov treatise On analytical methods in probabilitytheory [111], the term ‘diffusion’ refers to a special class of continuous Markovprocesses specified as follows.

Let X = (Xt)t≥0 be a time-homogeneous Markov process in a phase space(E, E) defined on a filtered space (Ω,F , (Ft)t≥0; Px, x ∈ E) with transition func-tion P (t, x; B) . Let Tt be a shift operator acting on measurable functions f =f(x) , x ∈ E , by the formula:

Ttf(x) = Exf(Xt)(

=∫

E

f(y)P (t, x; dy))

. (4.5.1)

(The functions f = f(x) are assumed to be such that Exf(Xt) is well defined;very often the notation Ptf(x) is used instead of Ttf(x) .)

The Markov property implies that the operators Tt , t ≥ 0 , constitute asemi-group, i.e. TsTt = Ts+t for s, t ≥ 0 .

The operator

Af(x) = limt↓0

Ttf(x) − f(x)t

(4.5.2)

is called the infinitesimal operator (of either the Markov process X or the semi-group (Tt)t≥0 or the transition function P (t, x; B) ).

Another important characteristic of the Markov process X is its character-istic operator

Af(x) = limU↓x

Tτ(U)f(x) − f(x)Exτ(U)

(4.5.3)

whereTτ(U)f(x) = Exf(Xτ(U)), (4.5.4)

τ(U) is the time of the first exit from the neighborhood U of a point x and“ U ↓ x ” means that the limit is taken as the neighborhood U is diminishing intothe point x . (More details about the operators introduced and their relation canbe found in [53].)

2. Assume that E = Rd . A continuous Markov process X = (Xt)t≥0 satisfy-ing the strong Markov property is said to be a diffusion process if its characteristic


operator Af(x) is well defined for any function f ∈ C2(x) i.e. for any functionwhich is continuously differentiable in a neighborhood of the point x . It turns out(see [53, Chap. 5, § 5] that for diffusion processes and f ∈ C2(x) the operatorAf(x) is a second order operator

Lf(x) =d∑

i,j=1

aij(x)∂2f(x)∂xi∂xj

+d∑

i=1

bi(x)∂f(x)∂xi

− c(x)f(x) (4.5.5)

where c(x) ≥ 0 and∑d

i,j=1 aij(x)λiλj ≥ 0 (the ellipticity condition) for anyλ1, . . . , λd . (Cf. Subsection 4.3.)

The functions aij(x) and bi(x) are referred to as diffusion coefficients anddrift coefficients respectively. The function c(x) is called a killing coefficient (ordiscounting rate).

3. In Subsection 4.3 it was already noted that for given functions aij(x) andbi(x) K. Ito provided a construction (based on a stochastic differential equation)of a time-homogeneous Markov process whose diffusion and drift coefficients co-incide with the functions aij(x) and bi(x) . In Subsection 3.4 we also consideredproblems related to stochastic differential equations in the case of “diffusion withjumps”.

4.6. Levy processes

1. Brownian motion considered above provides an example of a process which,apart from having the Markov property, is remarkable for being a process with(stationary) independent increments. Levy processes, which will be consideredin this subsection, are also processes with (stationary) independent increments.Thus it is natural first to list basic definitions and properties of such processestaking also into account that these processes are semimartingales (discussed inSubsection 3.3).

Let (Ω,F , (Ft)t≥0, P) be a filtered probability space. A stochastic processX = (Xt)t≥0 is called a process with independent increments if

(α) X0 = 0 ;

(β) the trajectories of (Xt)t≥0 are right-continuous (for t ≥ 0 ) and havelimits from the left (for t > 0 );

(γ) the variables Xt are Ft -measurable, t ≥ 0 , and the increments Xt1 −Xt0 , Xt2 − Xt1 , . . . , Xtn − Xtn−1 are independent for all 0 ≤ t0 < t1 <· · · < tn , n ≥ 1 .

Any such process X can be represented in the form X = D + S whereD = (Dt)t≥0 is a deterministic function (maybe of unbounded variation) andS = (St)t≥0 is a semimartingale (see [106, Chap. II, Theorem 5.1]).


Because D is a non-random process, and as such not interesting from theviewpoint of “stochastics”, and the process S is both semimartingale and a processwith independent increments, from now on we will assume that all processes withindependent increments we consider are semimartingales.

Let (B, C, ν) be the triplet of predictable characteristics of a semimartin-gale X (for details see Subsection 3.3). A remarkable property of processes withindependent increments (which are semimartingales as well) is that their triplets(B, C, ν) are deterministic (see [106, Chap. II, Theorem 4.15]).

As in Section 3.3 by means of the triplet (B, C, ν) define the cumulant


2Ct +

∫R


)ν((0, t]×dx

)(4.6.1)

( λ ∈ R , t ≥ 0 ) where h = h(x) is a truncation function (the standard oneis h(x) = xI(|x| ≤ 1) ) and ν is the compensator of the measure of jumps ofthe process X . With the cumulant Kt(λ) , t ≥ 0 , we associate the stochasticexponential E(λ) = (Et(λ))t≥0 defined as a solution to the equation

dEt(λ) = Et−(λ) dKt(λ), E0(λ) = 1. (4.6.2)

A solution to this equation is the function

Et(λ) = eKt(λ)∏s≤t

(1 + ∆Ks(λ)

)e−∆Ks(λ). (4.6.3)

The Ito formula (page 67) immediately implies that for ∆Kt(λ) = −1 , t ≥ 0 ,the process M(λ) = (Mt(λ),Ft)t≥0 given by

Mt(λ) =eiλXt

Et(λ)(4.6.4)

is a martingale for every λ ∈ R , and so (since Et(λ) is deterministic) the charac-teristic function of Xt equals Et(λ) , i.e.

EeiλXt = Et(λ). (4.6.5)

In particular, if Bt , Ct and ν((0, t] × A) are continuous functions in t then∆Kt(λ) = 0 and (4.6.5) becomes the well-known (generalized) Kolmogorov–Levy–Khintchine formula:

EeiλXt = exp(Kt(λ)

). (4.6.6)

2. Levy processes are processes whose triplets have the very special structure:

Bt = bt, Ct = ct and ν(dt, dx) = dt F (dx) (4.6.7)


where F = F (dx) is a measure on R such that F (0) = 0 and∫R

min(1, x2)F (dx) < ∞. (4.6.8)

A Levy process has stationary independent increments, is continuous in probabil-ity, and has no fixed time of jump with probability 1.

Apart from Brownian motion, a classical example of a Levy process is thePoisson process N = (Nt)t≥0 (with parameter a > 0 ) which is characterizedby its piecewise constant trajectories with unit jumps. If T1, T2, . . . are times ofjumps then the random variables T1 − T0 (with T0 = 0 ), T2 − T1 , T3 − T2 , . . .are independent and identically distributed with P(T1 > t) = e−at (exponentialdistribution). It is clear that Nt =

∑n≥0 I(Tn ≤ t) . For the Poisson process one

has b = a , c = 0 ν(dt, dx) = a dt δ1(dx) where δ1 is a measure concentratedat the point 1 . Thus from (4.6.6) we find that

EeiλNt = exp(at(eiλ − 1)

). (4.6.9)

Another example of a jump-like Levy process is given by the so-called com-pound Poisson process. By definition it is a process with the triplet (0, 0, F ) (withrespect to the truncation function h ≡ 0 ) where the measure F = F (dx) is suchthat F (R) < ∞ . Such a process X admits the following explicit construction:

Xt =Nt∑

k=0

ξk (4.6.10)

where ξ0 = 0 , (ξk)k≥1 is a sequence of independent and identically distributedrandom variables with distribution F (dx)/F (R) and N is a Poisson process withparameter a = F (R) .

3. An important subclass of the class of Levy processes is formed by (strictly)α -stable processes X = (Xt)t≥0 which are characterized by the following self-similarity property: for any c > 0 ,

Law(Xct; t ≥ 0) = Law(c1/αXt; t ≥ 0) (4.6.11)

where 0 < α ≤ 2 .

For such processes the characteristic function is given by the following Levy–Khintchine representation:

EeiλXt = exp(tψ(λ)

)(4.6.12)

where

ψ(λ) =

⎧⎪⎨⎪⎩iµλ − σα|λ|α

(1 − iβ(sgn λ) tan

πα

2

), α = 1,

iµλ − σ|λ|(1 + iβ

2π

(sgn λ) log |λ|), α = 1

(4.6.13)


with parameters0 < α ≤ 2, |β| ≤ 1, σ > 0, µ ∈ R. (4.6.14)

In the symmetrical case,ψ(λ) = −σα|λ|α. (4.6.15)

Denote by Sα(σ, β, µ) the stable laws with parameters α , σ , β , µ . Un-fortunately the explicit form of this distribution is known only for a few specialvalues of these parameters. These are as follows:

• S2(σ, 0, µ) = N (µ, 2σ2) — the normal distribution.

• S1(σ, 0, µ) — Cauchy’s distribution with the density

σ

π((x − µ)2 + σ2). (4.6.16)

• S1/2(σ, 1, µ) — the unilateral stable (Levy–Smirnov) distribution with thedensity √

σ

2π

1(x − µ)3/2

exp(− σ

2(x − µ)

), x ∈ (µ,∞). (4.6.17)

4. The exposed results on properties of the characteristic functions of pro-cesses with independent increments and Levy processes are commonly referred toas results of analytical probability theory. There is another approach to studyingsuch processes that is based on stochastic analysis of trajectories.

Let again X = (Xt)t≥0 be a process with independent increments that isalso a semimartingale. Then according to Subsection 3.3 the following canonicalrepresentation holds:


∫ t

0

∫R

h(x) d(µ − ν) +∫ t

0

∫(x − h(x)) dµ. (4.6.18)

In this case:

Bt is a deterministic function;

Xct is a Gaussian process such that DXc

t = Ct ;

ν = ν((0, t]×dx) is a deterministic function; and

µ = µ(ω; (0, t]×dx) is a Poisson measure (see [106, Chap. II, § 1c]).

If the compensator ν has the form ν = ν((0, t] × dx) = t F (dx) (as in thecase of Levy processes), the measure µ is called a homogeneous Poisson measure.Then the random variable µ((0, t] × A) has a Poisson distribution such that

Eµ((0, t]×A) = ν((0, t]×A). (4.6.19)


The representation (4.6.18) is often called an Ito–Levy representation. In the caseof Levy processes one has Bt = bt , Ct = ct and, as already noted, ν((0, t]×dx) =t F (dx) .

5. Basic transformations

5.1. Change of time

1. When solving optimal stopping problems, if we want to obtain solutions in theclosed form, we have to resort to various transformations both of the processesunder consideration as well as of the equations determining one or another char-acteristic of these processes.

The best known and most important such transformations are:

(a) change of time (often applied simultaneously with (b) below);

(b) change of space;

(c) change of measure;

and others (killing, creating, . . . ).

2. In this subsection we concentrate on basic ideas related merely to changeof time. In the next subsection we shall deal with problems of change of spacetogether with methods of change of time because it is by combination of thesemethods that one succeeds to obtain results on transformation of “complicated”processes into “simple”ones.

The problem of change of time can be approached in the following way.

Imagine we have a process X = (Xt)t≥0 with rather complicated structuree.g. a process with differential dXt = σ(t, Xt) dWt where (Wt)t≥0 is a Wienerprocess. Such a process, as we shall see, can be represented in the form

X = X T (5.1.1)

(i.e. in the form Xt = XT (t) , t ≥ 0 ) where X = (Xθ)θ≥0 is a certain “simple”process in the “new” time θ and θ = T (t) is a certain change of time exercisingthe transformation of the “old” time t into a “new” time θ .

Moreover the process X = (Xθ)θ≥0 can be chosen to be a very simpleprocess—a Brownian motion.

This can be realized in the following way.

Let

T (t) =∫ t

0

σ2(u, Xu) du (5.1.2)

andT (θ) = inft : T (t) > θ. (5.1.3)

Section 5. Basic transformations 107

Assume that σ2(s, Xs) > 0 ,∫ t

0 σ2(s, Xs) ds < ∞ , t > 0 , and∫∞0 σ2(s, Xs) ds =

∞ P-a.s. Then T (t) is a continuous increasing process, T = inf t : T (t) = θ and so ∫ bT (θ)

0

σ2(u, Xu) du = T(T (θ)

)= θ. (5.1.4)

Notice that the latter formula yields that

dT (θ)dθ

=1

σ2(T (θ), X bT (θ)). (5.1.5)

In the time-homogeneous case when σ depends only on x ( σ = σ(x) ) the mea-sure m = m(dx) defined by

m(dx) =1

σ2(x)dx (5.1.6)

is called the speed measure (“responsible” for the change of time).

In the “new” time θ let

Xθ = X bT (θ) =∫ bT (θ)

0

σ(u, Xu) dWu. (5.1.7)

Immediately we see that

E Xθ = 0, (5.1.8)

E X2θ = E

∫ bT (θ)

0

σ2(u, Xu) du = θ. (5.1.9)

The process X = (Xθ)θ≥0 is a martingale with respect to the filtration (F bT (θ))θ≥0

and thus—by the “Levy characterization theorem” (page 94)—is a Brownian mo-tion.

So the processX = X T (5.1.10)

is a Brownian motion, and it is clear that the inverse passage—to the process X —is realized by the formula

X = X T (5.1.11)

that means, in more details, that

Xt = XT (t) = XRt0 σ2(s,Xs)ds, t ≥ 0. (5.1.12)

In this way we have obtained the well-known result that a (local) martingaleXt =

∫ t

0 σ(s, Xs) dWs can be represented as a time change (by θ = T (t) ) of a


new Brownian motion X , i.e.X = X T (5.1.13)

(see Lemma 5.1 below).

Let us consider the same problem—on the representation of a “complicated”process Xt =

∫ t

0 σ(s, Xs) dWs by means of a “simple” process (in our case bymeans of a Brownian motion)—in terms of transition probabilities of a (Markov)process X .

Assume that the coefficient σ = σ(s, x) is such that the equation dXt =σ(t, Xt) dWt with X0 = 0 has a unique strong solution which is a Markov process.(Sufficient conditions for this can be found in [94].) Denote by f = f(s, x; t, y) itstransition density:

f(s, x; t, y) =∂P(Xt ≤ y |Xs = x)

∂y. (5.1.14)

This density satisfies the forward equation (for t > s )

∂f

∂t=

12

∂2

∂y2

(σ2(t, y)f

)(5.1.15)

and the backward equation (for s < t )

∂f

∂s= −1

2σ2(s, x)

∂2f

∂x2. (5.1.16)

Leaving unchanged the space variable introduce the new time θ = T (t)( =

∫ t

0σ2(u, Xu) du ) and denote θ′ = T (s) for s < t . Under these assumptions

letf ′(θ′, x; θ, y) = f(s, x; t, y). (5.1.17)

The function f ′ = f ′(θ′, x; θ, y) is non-negative, satisfies the normalizing condition∫f ′(θ′, x; θ, y) dy = 1 (θ′ < θ) (5.1.18)

and the Chapman–Kolmogorov equation (see (4.3.2)). It is not difficult to derivethe corresponding forward and backward equations (in the “new θ -time”):

∂f ′

∂θ=

12

∂2f ′

∂y2and

∂f ′

∂θ′= −1

2∂2f ′

∂y2(5.1.19)

which follows from

∂f ′

∂θ=

∂f

∂t

1∂θ∂t

=∂f

∂t

1∂T (t)

∂t

=12

σ2(t, y)∂2f

∂y2

1σ2(t, y)

=12

∂2f

∂y2(5.1.20)

(and in an analogous way for the backward equation).


It is clear that the transition function f ′ = f ′(θ′, x; θ, y) is the transitionfunction of a Brownian motion X = (Xθ)θ≥0 given by the Laplace formula:

f ′(θ′, x; θ, y) =1√

2π(θ− θ′)exp

(− (y − x)2

2(θ− θ′)

). (5.1.21)

3. Let us dwell on certain general principles of changing the “new” time θinto the “old” time t and vice versa. To this aim assume that we are given afiltered probability space (Ω,F , (Ft)t≥0, P) .

One says that a family of random variables T = (T (θ))θ≥0 performs a changeof time (to be more precise, a change of the “new” θ -time into the “old” t -time)if the following two conditions are fulfilled:

(a) (T (θ))θ≥0 is a right-continuous non-decreasing family of random variablesT (θ) which in general take their values in [0,∞] ;

(b) for every θ ≥ 0 the variable T (θ) is a Markov time, i.e. for all t ≥ 0 ,

T (θ) ≤ t ∈ Ft. (5.1.22)

Assume that the initial filtered probability space is the space (Ω,F , (Fθ)θ≥0, P)and we are given a family of random variables T = (T (t))t≥0 with the propertyanalogous to (a) above and such that for all t ≥ 0 the variables T (t) are Markovtimes i.e. for all θ ≥ 0 ,

T (t) ≤ θ ∈ Fθ. (5.1.23)

Then we say that the family T = (T (t))t≥0 changes the “old” t -time into the“new” θ -time. The primary method of construction of the system (T (θ))θ≥0 (and,in an analogous way, of T = (T (t))t≥0 ) consists in the following.

Let a process A = (At)t≥0 defined on a filtered probability space (Ω,F ,(Ft)t≥0, P) be such that A0 = 0 , the variables At are Ft -measurable and itstrajectories are right-continuous (for t ≥ 0 ) and have limits from the left (fort > 0 ).

DefineT (θ) = inft ≥ 0 : At > θ, θ ≥ 0 (5.1.24)

where as usual inf(∅) = ∞ . It can be verified that the system T = (T (θ))θ≥0

determines a change of time in the sense of the above definition.

In connection with transforming the process X = (Xt)t≥0 into a new processX = (Xθ)θ≥0 with Xθ = X bT (θ) notice that to ensure F bT (θ) -measurability of thevariables X bT (θ) one has to impose certain additional measurability conditions on


the process X . For example it suffices to assume that the process X = (Xt)t≥0

is progressively measurable, i.e. such that for any t ≥ 0 ,

(ω, s) ∈ Ω×[0, t] : Xs(ω) ∈ B ∈ Ft × B([0, t]). (5.1.25)

(This property is guaranteed if X is right-continuous.)

Letting as above T (t) = At , t ≥ 0 , and T (θ) = inft : T (t) > θ we easilyfind that the following useful properties hold:

(i) if the process A = (At)t≥0 is increasing and right-continuous then

T (T (t)) = t, T (T (θ)) = θ, T (θ) = T−1(θ), T (t) = T−1(t); (5.1.26)

(ii) for non-negative functions f = f(t) ,∫ bT (b)

0

f(t) d(T (t)) =∫ b

0

f(T (θ)) dθ, (5.1.27)∫ T (a)

0

f(θ) d(T (θ)) =∫ a

0

f(T (t)) dt. (5.1.28)

4. The following two lemmas are best known results on change of time instochastic analysis.

Lemma 5.1. (Dambis–Dubins–Schwarz) Let X = (Xt)t≥0 be a continuous localmartingale defined on a filtered probability space (Ω,F , (Ft)t≥0, P) with X0 = 0and 〈X〉∞ = ∞, and let

T (t) = 〈X〉t, (5.1.29)

T (θ) = inf t ≥ 0 : T (t) > θ . (5.1.30)

Then

(a) the process X = (Xθ)θ≥0 given by

Xθ = X bT (θ) (5.1.31)

is a Brownian motion;

(b) the process X can be reconstructed from the Brownian motion X by for-mulae:

Xθ = XT (t), t ≥ 0. (5.1.32)

Lemma 5.2. Let X = (Xt)t≥0 be a counting process, X0 = 0, with compensatorA = (At)t≥0 and the Doob–Meyer decomposition X = M+A where M = (Mt)t≥0

is a local martingale. Let the compensator A be continuous (then A = 〈M〉 ).


Define T (t) = At (= 〈Mt〉) and

T (θ) = inf t ≥ 0 : T (t) > θ . (5.1.33)

Under the assumption that T (t) ↑ ∞ as t → ∞ the process X = (Xθ)θ≥0 givenby

T (θ) = X bT (θ) (5.1.34)

is a standard Poisson process (with parameter 1). The process X = (Xt)t≥0 canbe reconstructed from the process X = (Xθ)θ≥0 by

Xt = XT (t). (5.1.35)

A proof can be found e.g. in [128].

5.2. Change of space

1. For the first time the question about transformations of (diffusion) processes into“simple” processes (like Brownian motion) was considered by A. N. Kolmogorovin “Analytical methods” [111]. He simultaneously considered both change of time( t → θ , s → ϑ ) and change of space ( x → ξ , y → η ) with the purpose totransform the transition function f = f(s, x; t, y) into a new transition func-tions g = g(ϑ, ξ; θ, η) which satisfies simpler (forward and backward) differentialequations than the equations for f = f(s, z; t, y) . To illustrate this consider anOrnstein–Uhlenbeck process X = (Xt)t≥0 solving

dXt =(α(t) − β(t)Xt

)dt + σ(t) dWt, X0 = x0. (5.2.1)

The coefficients of this equation are assumed to be such that∫ t

0

∣∣∣∣α(s)γ(s)

∣∣∣∣ ds < ∞,

∫ t

0

∣∣∣∣σ(s)γ(s)

∣∣∣∣2 ds < ∞ for t > 0, (5.2.2)∫ t

0

∣∣∣∣σ(s)γ(s)

∣∣∣∣2 ds ↑ ∞ as t ↑ ∞ where γ(s) = exp(−∫ t

0

β(u) du

). (5.2.3)

It is not difficult to check, e.g. using Ito’s formula, that

Xt = γ(t)[x0 +

∫ t

0

α(s)γ(s)

ds +∫ t

0

σ(s)γ(s)

dWs

]. (5.2.4)

Introducing the “new” time θ = T (t) with

T (t) =∫ t

0

(σ(s)γ(s)

)2

ds (5.2.5)


and letting T (θ) = inft ≥ 0 : T (t) > θ as in Subsection 5.1 we find (fromLemma 5.1 above) that the process

Bθ =∫ bT (θ)

0

σ(s)γ(s)

dWs (5.2.6)

is a Brownian motion (Wiener process) and

Xt = ϕ(t) + γ(t)BT (t) (5.2.7)

where

ϕ(t) = γ(t)[x0 +

∫ t

0

α(s)γ(s)

ds

]. (5.2.8)

Following Kolmogorov [111] introduce the function

Ψ(t, y) =y

γ(t)−∫ t

0

α(u)γ(u)

du (5.2.9)

and the functiong(ϑ, ξ; θ, η) =

1(∂Ψ(t,y)

∂y

) f(s, x; t, y) (5.2.10)

where f(s, x; t, y) is the transition density of the process X (with x0 = 0 ) and

η = Ψ(t, y), ξ = Ψ(s, x), (5.2.11)

ϑ =∫ s

0

(σ(u)γ(u)

)2

du, θ =∫ t

0

(σ(u)γ(u)

)2

du.

Taking into account this notation it is not difficult to verify (see also the formulae(5.2.22) and (5.2.23) below) that

∂g

∂θ=

12

∂2g

∂η2and

∂g

∂ϑ= −1

2∂2g

∂ξ2. (5.2.12)

Thus the transformation of time

t θ = T (t) (s ϑ = T (s)) (5.2.13)

and the transformation of space

y η = Ψ(t, y) (x ξ = Ψ(s, x)) (5.2.14)

turn the function f = f(s, x; t, y) into the function g = g(ϑ, ξ; θ, η) which is thetransition function of a Brownian motion. Since Ψ(t, Xt) =

∫ t

0σ(s)γ(s) dWs we have

Ψ(T (θ), X bT (θ)) =∫ bT (θ)

0

σ(s)γ(s)

ds = Bθ. (5.2.15)


So the change of time T (θ) and the function Ψ = Ψ(t, y) allow one to construct(in the “new” time θ ) a Brownian motion B (which in turn allows, by reversechange of time T (t) , to construct the process X according to (5.2.7)). All thismakes the equations (5.2.12) transparent.

2. The preceding discussion about an Ornstein–Uhlenbeck process admitsan extension to more general diffusion processes. Namely, following Kolmogorov[111], assume that f = f(s, x; t, y) is a transition density satisfying the followingforward and backward equation:

∂f

∂t= − ∂

∂y

[b(t, y)f

]+

12

∂2

∂y2

[a(t, y)f

], (5.2.16)

∂f

∂s= −b(s, x)

∂f

∂x− 1

2a(s, x)

∂2f

∂x2(5.2.17)

respectively. In other words, we consider a diffusion Markov process X = (Xt)t≥0

which is a strong solution to the stochastic differential equation

dXt = b(t, Xt) dt + σ(t, Xt) dWt (5.2.18)

where W = (Wt)t≥0 is a Wiener process and σ2(t, x) = a(t, x) . (The coefficientsb(t, x) and σ(t, x) are assumed to be such that the process X is Markov.)

As in the previous subsection introduce a change of time θ = T (t) ( ϑ =T (s) ) and a change of the space variable η = Ψ(t, y) ( ξ = Ψ(s, x) ).

We assume that T (t) is an increasing function from the class C1 , the func-tion Ψ(t, y) belongs to the class C1,2 and ∂Ψ(t, y)/∂y > 0 . Putting

g(ϑ, ξ; θ, η) =f(s, x; t, y)

∂Ψ(t,y)∂y

, (5.2.19)

B(θ, η) =12a(t, y)∂2Ψ(t,y)

∂y2 + b(t, y)∂Ψ(t,y)∂y + ∂Ψ(t,y)

∂t

∂T (t)∂t

, (5.2.20)

A(θ, η) =a(t, y)

(∂Ψ(t,y)

∂y

)2

∂T (t)∂t

(5.2.21)

(and taking into account the connection between the variables introduced) we findthat the non-negative function g = g(ϑ, ξ; θ, η) satisfies the normalizing condition(∫

Rg(ϑ, ξ; θ, η) dη = 1 ) and solves the Chapman–Kolmogorov equation as well as

the following forward and backward equations (in (θ, η) and (ϑ, ξ) respectively):

∂g

∂θ= − ∂

∂η

[B(θ, η)g

]+

12

∂2

∂η2

[A(θ, η)g

](5.2.22)

∂g

∂ϑ= −B(ϑ, ξ)

∂g

∂ξ− 1

2A(ϑ, η)

∂2g

∂ξ2. (5.2.23)


In other words, if the process X has characteristics (b, a) then the process X =(Xθ)θ≥0 with

Xθ = Ψ(T (θ), X bT (θ)

)(5.2.24)

is a (diffusion) process with characteristics (B, A) i.e. it satisfies the equation

dXθ = B(θ, Xθ) dθ + Σ(θ, Xθ) dWθ (5.2.25)

where Σ2 = A and W = (Wθ)θ≥0 is a Wiener process.

3. Let us address the case when the initial process X = (Xt)t≥0 is a time-homogeneous Markov diffusion process solving

dXt = b(Xt) dt + σ(Xt) dWt, σ(x) > 0. (5.2.26)

In this case it is natural to choose Ψ to be homogeneous in the sense that Ψ =Ψ(y) ; according to the tradition we shall denote this function by S = S(y) . From(5.2.20) we see that for the coefficient B = B(η) to be equal to zero, the functionS(y) must satisfy the equation ( a = σ2 ),

12a(y)S′′(y) + b(y)S′(y) = 0 (5.2.27)

i.e.S′′(y)S′(y)

= −2b(y)a(y)

. (5.2.28)

Solving this equation we find that

S(y) = c1 + c2

∫ y

y0

exp−∫ z

y0

2b(v)σ2(v)

dv

dz (5.2.29)

where c1 , c2 and y0 are constants.

This function (to be more precise any positive of these functions) is called ascale function. If we let θ = t (i.e. the “new” time coincides with the old one sothat T (t) = t ) then (5.2.21) yields

A(η) = σ2(y)(S′(y)

)2. (5.2.30)

Thus from (5.2.27) and (5.2.30) we conclude that for Yt = S(Xt)

dYt = σ(Xt)S′(Xt) dWt (5.2.31)

where W = (Wt)t≥0 is a Wiener process. The Dambis–Dubins–Schwarz lemma(page 110) implies that if

T1(t) =∫ t

0

(σ(Xu)S′(Xu)

)2du and T1(θ) = inf t : T1(t) > θ (5.2.32)


then in the “new” θ -time the process B = (Bθ)θ≥0 given by

Bθ = Y bT1(θ) = S(X bT1(θ)

)=∫ bT1(θ)

0

σ(Xu)S′(Xu) dBu (5.2.33)

is a Wiener process. Since Yt = S(Xt) we have Xt = S−1(Yt) . Here Yt = BT1(t) .Therefore

Xt = S−1(BT1(t)

). (5.2.34)

5.3. Change of measure

1. The essence of transformations which are related to a change of measure (andhave also many applications in solving optimal stopping problems) may be de-scribed as follows.

The change of time aims to represent the “complicated” process X = (Xt)t≥0

as a composition X = X T (i.e. Xt = XT (t) , t ≥ 0 ) where X = (Xθ)θ≥0 is a“simple” process and T = T (t) is a change of time.

Unlike the change of time (which, so to speak, makes the speed of movementalong trajectories to vary) a change of measure does not deal with transformationsof trajectories of X but transforms the initial probability measure P into anotherprobability measure P (which is equivalent to P ) in such a way that

Law(X | P) = Law(X | P) (5.3.1)

where X = (Xt)t≥0 is a “simple” process (with respect to the measure P ).

To illustrate this consider a filtered probability space (Ω,F , (Ft)t≥0, P) witha Brownian motion B = (Bt,Ft)t≥0 and an adapted integrable process b =(bt,Ft)t≥0 defined on this space.

Consider an Ito process X = (Xt,Ft)t≥0 solving

dXt = bt dt + dBt. (5.3.2)

Form a new measure P such that its restriction Pt = P|Ft onto the σ -algebraFt is given by

dPt = Zt dP (5.3.3)

where

Zt = exp∫ t

0

(bs(ω) − bs(ω)

)dBs − 1

2

∫ t

0

(bs(ω) − bs(ω)

)2ds

(5.3.4)

and b = ( bt,Ft)t≥0 is another adapted integrable process. Assume that∫ t

0

(bs(ω)

−bs(ω))2

ds < ∞ P-a.s. and EZt = 1 for every t > 0 . Notice that the required


measure P satisfying the property Pt = Zt dP for t > 0 can certainly be con-structed on the σ -algebra

∨t≥0 Ft ( = σ

(⋃t≥0 Ft

)) under the assumption that

the set Ω is a space of functions ω = ω(t) , t ≥ 0 which are right-continuous (fort ≥ 0 ) and have limits from the left (for t > 0 ) and that all processes consideredare canonical (see e.g. [209]).

According to the well-known Girsanov theorem for Brownian motion (see[77]) the process X with respect to the new measure P becomes a Brownianmotion; this can be expressed as

Law(X | P) = Law(B | P). (5.3.5)

In other words, the transition from the measure P to the measure P “kills” thedrift of the process X .

Now, keeping ourselves within the framework of Ito processes, consider thecase when the change of measure P P results in a change of drift b b of theprocess X where b = ( bt,Ft)t≥0 is another adapted integrable process.

To this end introduce new measures Pt , t ≥ 0 as above by letting dPt =Zt dP where Zt is specified in (5.3.4) and the coefficients b and b are again suchthat EZt = 1 , t > 0 . In this case the Girsanov result says that with respect tothe new measure P the process B = (Bt,Ft)t≥0 given by

Bt = Bt −∫ t

0

(bs − bs

)ds (5.3.6)

is a Brownian motion and satisfies

dXt = bt(ω) dt + dBt. (5.3.7)

This can also be expressed otherwise in the following way. Assume that, alongwith the process X , another process X solving

dXt = bt(ω) dt + dBt (5.3.8)

is defined on the initial probability space. Then

Law(X | P) = Law(X | P). (5.3.9)

In particular if b ≡ 0 we obtain the result formulated earlier that the transitionP P “kills” the drift b of the process X (i.e. transforms b into b ≡ 0 ). Ifb ≡ 0 (i.e. X = B ) then the transition P P by means of the process

Zt = exp∫ t

0

bs(ω) dBs − 12

∫ t

0

(bs(ω)

)2ds

, t ≥ 0 (5.3.10)

“creates” the drift of the Brownian motion, i.e. B. ∫ .

0 bs(ω) ds + B .


The exposed results of I. V. Girsanov [77] gave rise to a wide cycle of re-sults bearing the name of Girsanov theorems (for martingales, local martingales,semimartingales, etc.).

Let us cite several of them referring to the monograph [106, Chap. III, §§ 3b–3e] for details and proofs.

2. The case of local martingales. Assume that on the initial filtered probabilityspace (Ω,F , (Ft)t≥0) we have two probability measures P and P such thatPloc P i.e. Pt Pt , t ≥ 0 where Pt = P|Ft and Pt = P|Ft , t ≥ 0 .

Let Zt = dPt/dPt . Assume that M = (Mt,Ft)t≥0 is a local martingale withM0 = 0 such that the P -quadratic covariation [M, Z] has P -locally integrablevariation. In this case [M, Z] has a P -compensator 〈M, Z〉 (see Subsection 3.3).

Lemma 5.3. (Girsanov’s theorem for local martingales) The process

M = M − 1Z−

· 〈M, Z〉 (5.3.11)

i.e. the process M = (Mt,Ft)t≥0 with M = Mt −∫ t

01

Zs− d〈M, Z〉s is a P -localmartingale.

This result can be interpreted in the following way: the process M obtainedfrom a local P -martingale M by means of (5.3.11) is not in general a localmartingale with respect to the measure P (this process is semimartingale) butwith respect to the new measure P .

3. The case of semimartingales. Assume that the process X is a semimartin-gale (with respect to the measure P ) with the triplet of predictable characteristics(B, C, ν) . Consider a measure P such that Ploc P .

Lemma 5.4. 1. With respect to the measure P the process X is again a semi-martingale (with a triplet (B, C, ν )) .

2. The triplets (B, C, ν) and (B, C, ν ) are related by the formulae

B = B + β · C + h(Y − 1) ∗ ν,

C = C, (5.3.12)ν = Y · ν

where

• h = h(x) is a truncation function (the standard one is h(x) = xI(|x| ≤ 1));

• β = (βt(ω))t≥0 is an adapted process specified by

β =d〈Zc, Xc〉

d〈Xc〉I(Z− > 0)

Z−(5.3.13)


where Zc is the continuous component of the process Z = (Zt)t≥0 withZt = dPt/dPt;

• Y = Y (ω; t, x) is specified by

Y = EPµ

(Z

Z−I(Z− > 0)

∣∣ P) (5.3.14)

where EPµ is averaging with respect to the measure MP

µ on(Ω×R+×P,F⊗

B(R+) ⊗ B(R)), specified by the formula X ∗ MP

µ = E (W ∗ µ) for all non-negative measurable functions W = W (ω, t, x) , and P is the predictableσ -algebra, P = P⊗B(R) , where P is the predictable σ -algebra on Ω×R+ .

(For more details see [106, Chap. III, § 3d].)

4. The above results answered one of the questions related to the change ofmeasure, namely the question as what the initial process (or its characteristics) be-comes if instead of the initial probability measure P we consider another measureP such that Ploc P .

Another question naturally arises as how to describe all measures P satisfy-ing the property Ploc P .

We assume that the measures P and P are probability distributions of asemimartingale X = (Xt)t≥0 defined on the filtered space (Ω,F , (Ft)t0 ) whereΩ consists of the functions ω = ω(t) , t ≥ 0 which are right-continuous and havethe limits from the left, and the process X is canonically defined (i.e. Xt(ω) =ω(t) for t ≥ 0 ).

General results from the theory of semimartingales (see [106, Chap. III, § 4d,Lemma 4.24] imply that the process Z = (Zt)t≥0 given by Zt = dPt/dPt admitsthe representation

Z = H · Xc + W ∗ (µ − ν) + N (5.3.15)

where the processes H = (Ht(ω)) and W = (Wt(ω; t, x)) with t ≥ 0 , x ∈ R

satisfy certain integrability conditions, Xc is the continuous martingale compo-nent of X , µ is the measure of jumps of X and ν is the compensator of µ . In(5.3.15) it is the local martingale N = (Nt)t≥0 which is troublesome because itlacks a simple description. In some simple cases N ≡ 0 and then the representa-tion (5.3.15) takes the form

Z = H · Xc + W ∗ (µ − ν). (5.3.16)

Starting from (5.3.16), under the assumption that Zt ≥ 0 and EZt = 1 one canconstruct different measures P by letting dPt = Zt dPt for t ≥ 0 .

If the initial process X is a Brownian motion or, more generally, a processwith independent increments (in particular a Levy process) then the representa-tion (5.3.16) is valid.


In the general case, when one cannot rely on getting the representation(5.3.16), one has to choose the way of construction of a concrete martingale Z

that leads to the possibility of constructing a measure P satisfying the propertyPloc P .

Among these methods one far-famed is based on the Esscher transformation.The essence of this method may be described as follows.

Let X = (Xt)t≥0 be a semimartingale on (Ω,F , (Ft)t≥0, P) with the triplet(B, C, ν) . Let K(λ) = (Kt(λ))t≥0 be the cumulant process given by


2Ct +

∫R


)ν(ω; (0, t]×dx) (5.3.17)

where λ ∈ (−∞,∞) , and form a new positive process Z(λ) = (Zt(λ))t≥0 bysetting

Zt(λ) = expλXt − Kt(λ)

(5.3.18)

whereKt(λ) = log Et(K(λ)) (5.3.19)

and E = E(K(λ)) is the stochastic exponential ( dEt(K(λ)) = Et−(K(λ)) dKt(λ) ,E0(K(λ)) = 1 ). It turns out (see [106, second ed.]) that Zt(λ) admits the repre-sentation of the form

Zt(λ) = E(

λXc +eλx − 1

W (λ)∗ (µX − ν)

)(5.3.20)

where Wt(λ) =∫

(eλx − 1) ν(t × dx) and µX is the measure of jumps of theprocess X .

From (5.3.20) it follows that the process Z(λ) = (Zt(λ))t≥0 is a positivelocal martingale (with respect to P ). Thus if this process is a martingale thenEZt(λ) = 1 for each t > 0 and one may define a new probability measureP(λ) P such that for each t > 0

dPt(λ)dPt

= Zt(λ). (5.3.21)

The measure P(λ) constructed in such a way is called the Esscher measure.

5.4. Killing (discounting)

The essence of transformations called “killing” (“discounting”) and “creation” isdeeply rooted in the derivation of the diffusion equation (due to Fick [66] in 1855upon mimicking Fourier’s 1822 derivation of the heat equation).


If γ = γ(t, x) denotes the concentration of N Brownian particles suspendedin a fluid (where N is large), and Fick’s law of diffusion [66] applies, then thediffusion equation holds:

γt + K = div(D gradγ) − div(vγ) + C (5.4.1)

=∑

i(Dγxi)xi −∑

i(vγ)xi + C

where K = K(t, x) corresponds to the disappearance (killing) of Brownian par-ticles, D = D(t, x) is the diffusion coefficient, v = v(t, x) is the velocity of thefluid, and C = C(t, x) corresponds to the appearance (creation) of Brownianparticles.

Since p := γ/N for large N may be interpreted as the transition densityof the position process X = (Xt)t≥0 , it follows that p = p(t, x) solves the samediffusion equation:

pt + K = div(D grad p) − div(vp) + C. (5.4.2)

To simplify the notation let us assume in the sequel that the setting is one-dimensional ( i.e. x ∈ R ). Then (5.4.2) reads as follows:

pt + K = (Dpx)x − (vp)x + C. (5.4.3)

Assuming for the moment that K = C ≡ 0 in (5.4.3) and that X is Marko-vian we know that p = p(t, x) solves the Kolmogorov forward equation (cf. [111]):

pt = −(ρp)x −(σ2

2p)

xx(5.4.4)

where ρ = ρ(t, x) is the drift and σ = σ(t, x) > 0 is the (mathematical) diffusioncoefficient. A direct comparison of (5.4.3) with K = C ≡ 0 and (5.4.4) showsthat σ =

√2D and ρ = v + Dx . If the terms K and C are to be incorporated

in (5.4.4) one may set R = K−C and consider the following reformulation of theequation (5.4.3):

pt + R = (Dpx)x − (vp)x (5.4.5)

where R = R(t, x) may take both positive and negative values.

The following particular form of R is known to preserve the Markov propertyof a transformed (killed or created) process X to be defined:

R = λp (5.4.6)

where λ = λ(x) > 0 corresponds to killing and λ = λ(x) < 0 corresponds tocreation. The equation (5.4.4) then reads as follows:

pt + λp = −(ρp)x +(σ2

2p)

xx. (5.4.7)


The process X = (Xt)t≥0 is obtained by “killing” the sample paths of X at therate λ = λ(x) where λ > 0 and “creation” of new sample paths of X at therate λ = λ(x) where λ < 0 . In probabilistic terms it means that the transitionfunction of X is given by

Pt(x, A) = Px(Xt ∈ A) = Ex

[e−

Rt0 λ(Xs) dsIA(Xt)

](5.4.8)

for x ∈ R and A ∈ B(R) with t ≥ 0 . The infinitesimal operator of X is givenby

L eX = LX − λI (5.4.9)

where I is the identity operator.

To verify (5.4.9) note that

ExF (Xt) − F (x)t

=ExF (Xt) − F (x)

t+

ExF (Xt) − ExF (Xt)t

(5.4.10)

=ExF (Xt) − F (x)

t+

Ex

[exp

(− ∫ t

0λ(Xs) ds − 1

)F (Xt)

]t

→ LXF − λF

as t ↓ 0 upon assuming that λ is continuous (at x ) and that we can exchangethe limit and the integral for the final convergence relation (sufficient conditionsfor the latter are well known).

Recalling that (5.4.3) can be written as

pt = L∗Xp (5.4.11)

where L∗X denotes the adjoint of LX , we see by (5.4.9) that (5.4.7) reads as

follows:pt = L∗

eXp (5.4.12)

which is in agreement with preceding facts.

When λ > 0 is a constant there is a simple construction of the killed pro-cess X . Let ζ be a random variable that is exponentially distributed with param-eter λ (i.e. Px(ζ > t) = e−λt for t > 0 ) and independent of X under each Px .The process X can then be defined as follows:

Xt :=

Xt if t < ζ,

∂ if t ≥ ζ(5.4.13)

where ∂ is a fictitous point (“cemetery”) outside Ω . All functions defined onΩ ∪ ∂ are assumed to take value zero at ∂ .

Notes. For further details on the material reviewed in Chapter II we refer tostandard textbooks on stochastic processes found in the Bibliography.

Chapter III.

Optimal stopping and free-boundary problems

In the end of Chapter I we have seen that the optimal stopping problem fora Markov process X with the value function V is equivalent to the problem offinding the smallest superharmonic function V which dominates the gain functionG on the state space E . In this case, moreover, the first entry time of X into thestopping set D = V = G is optimal. This yields the following representation:

V (x) = ExG(XτD ) (1)

for x ∈ E . Due to the Markovian structure of X , any function of the form(1) is intimately related to a deterministic equation which governs X in mean(parabolic/elliptic PDEs [partial differential equations] when X is continuous, ormore general PIDEs [partial integro-differential equations] when X has jumps).

The main purpose of the present chapter is to unveil and describe the previousconnection. This leads to differential or integro-differential equations which thefunction V solves. Since the optimal stopping set D is unknown and has tobe determined among all possible candidate sets D in (1), it is clear that thisconnection and the equations obtained play a fundamental role in search for thesolution to the problem. It is worthwhile to recall that when X is a Markov chainthe analogous problems have been considered in Subsection 4.2 in the context ofdiscrete-time “potential theory”.

In order to focus on the equations only, and how these equations actuallyfollow from the Markovian structure, we will adopt a formal point of view in thesequel where everything by definition is assumed to be sufficiently regular andvalid as needed to make the given calculations possible. Most of the time such aset of sufficient conditions is easily specified. Sometimes, however, it may be morechallenging to determine such sufficient conditions precisely. In any case, we willmake use of the facts exposed in the present chapter mostly in a suggestive waythroughout the monograph.

124 Chapter III. Optimal stopping and free-boundary problems

6. MLS formulation of optimal stopping problems

1. Throughout we will adopt the setting and notation of Subsection 2.2. Thus,we will consider a strong Markov process X = (Xt)t≥0 defined on a filteredprobability space (Ω,F , (Ft)t≥0, Px) and taking values in a measurable space(E, E) where for simplicity we will assume that E = Rd for some d ≥ 1 andE = Bd is the Borel σ -algebra on Rd . It is assumed that the process X starts atx under Px for x ∈ E and that the sample paths of X are right-continuous andleft-continuous over stopping times. It is also assumed that the filtration (Ft)t≥0

is right-continuous (implying that the first entry times to open and closed setsare stopping times). In addition, it is assumed that the mapping x → Px(F ) ismeasurable for each F ∈ F . It follows that the mapping x → Ex(Z) is measurablefor each random variable Z . Finally, without loss of generality we will assumethat (Ω,F) equals the canonical space (E[0,∞), E [0,∞)) so that the shift operatorθt : Ω → Ω is well defined by θt(ω)(s) = ω(t+s) for ω ∈ Ω with t, s ≥ 0 .

2. Given measurable (continuous) functions M, L, K : E → R satisfyingintegrability conditions implying (2.2.1), consider the optimal stopping problem

V = supτ

E

(M(Xτ ) +

∫ τ

0

L(Xt) dt + sup0≤t≤τ

K(Xt))

(6.0.1)

where the first supremum is taken over stopping times τ of X (or more generallywith respect to (Ft)t≥0 ). In the case of a finite horizon T ∈ [0,∞) it is assumedthat 0 ≤ τ ≤ T , and in the case of infinite horizon it is assumed that 0 ≤ τ < ∞ .In (6.0.1) we admit that any of the functions M , L or K may be identicallyequal to zero.

3. Clearly, the three terms following E in (6.0.1) provide three differentperformance measures. The first two are due to Mayer and Lagrange in the classicalcalculus of variations while the third one is more recent (see the notes in the end ofthe chapter for a historical account). Note that M in (6.0.1) stands for Mayer, Lfor Lagrange, and S for supremum (soon to be introduced below). This explainsthe term MLS in the title of the section.

4. For simplicity of exposition we will assume in the sequel that K(x) = x forall x ∈ E with E = R . Note that when K is strictly monotone (and continuous)for example, then K(X) = (K(Xt))t≥0 may define another Markov process, andby writing M(Xt) = (M K−1)(K(Xt)) and L(Xt) = (L K−1)(K(Xt)) we seeno loss of generality in the previous assumption.

Introduce the following processes:

It =∫ t

0

L(Xs) ds (integral process), (6.0.2)

St = sup0≤s≤t

Xs (supremum process) (6.0.3)

Section 6. MLS formulation of optimal stopping problems 125

for t ≥ 0 . Then the process Z = (Zt)t≥0 given by

Zt = (It, Xt, St) (6.0.4)

is Markovian (under general assumptions) and (6.0.1) reads as follows:

V = supτ

EG(Zτ ) (6.0.5)

where G(z) = M(x) + a + s for z = (a, x, s) ∈ R3 . We thus see that the optimalstopping problem (6.0.1) may be viewed as the optimal stopping problem (2.2.2)so that the general optimal stopping results of Subsection 2.2 are applicable. Notethat the process Z is three-dimensional in general.

5. Despite the fact that the general results are applicable, it turns out thatthe specific form of stochastic processes I = (It)t≥0 and S = (St)t≥0 makesa direct approach to (6.0.1) or (6.0.5) more fruitful. The key feature of I is itslinearity; to enable it to start at arbitrary points one sets

Iat = a + It (6.0.6)

for a ∈ R and t ≥ 0 . The key feature of S is its constancy and a strict increaseonly at times when equal to X ; to enable it to start at arbitrary points one sets

Sst = s ∨ St (6.0.7)

for s ≥ x in R and t ≥ 0 . With the choice of (6.0.6) and (6.0.7) the Markovianstructure of Z remains preserved relative to Px under which X starts at x ∈ R .Moreover, if Xx = (Xx

t )t≥0 starts at x under P and we set

Zzt = (Ia

t , Xxt , Ss

t ) (6.0.8)

for z = (a, x, s) , then the family of probability measures Pz = Law(Zz | P) de-fined on the canonical space is Markovian. This point of view proves useful inclassifying ad-hoc solutions in terms of general theory (see e.g. Subsection 13.2).The problem (6.0.5) consequently reads

V (x) = supτ

EzG(Zτ ) (6.0.9)

and the general optimal stopping theory of Chapter I is applicable.

6.1. Infinite and finite horizon problems

We have already pointed out in Chapter I that it is important to enable theprocess to start at arbitrary points in the state space (since the problem thencan be studied by means of the value function). The fact whether the horizon T


in (6.0.1) is infinite or finite is closely related. Some of these issues will now beaddressed.

1. Consider the case when the horizon T in (6.0.1) or (6.0.9) is infinite.Then, on the one hand, the problem is simpler than the finite horizon problem,however, only if we can come up with an explicit expression as the candidate forV (these expressions are obtained by solving the equations which govern Z inmean, typically ODEs [ordinary differential equations] in the infinite horizon case,and PDEs [partial differential equations] in the finite horizon case, at least whenZ is continuous). On the other hand, if such explicit expressions are not available,then the infinite horizon problem may be more difficult (in terms of characterizingthe solution via existence and uniqueness claims) than the corresponding finitehorizon problem (see e.g. Section 27). This is due to the fact that the Wald–Bellman equations (cf. Chapter I) are not directly available in the infinite horizoncase, while for example in the case of a diffusion process X we can characterize theoptimal stopping boundaries in terms of nonlinear integral equations that can bequite similarly solved by backward induction (cf. Chapters VI and VII for details).

2. Consider the case when the horizon T in (6.0.1) or (6.0.9) is finite. Thenthe time variable becomes important (as the remaining time goes to zero) and(unless It itself is already of this type) needs to be added to (6.0.4) so that Z

extended to Z reads as follows:

Zt = (t, It, Xt, St) (6.1.1)

for t ≥ 0 . The process Z = (Zt)t≥0 is Markovian and the optimal stoppingproblem (6.0.5) i.e. (6.0.9) extends as follows:

V (t, z) = sup0≤τ≤T−t

Et,zG(Zt+τ ) (6.1.2)

where Zt = z under Pt,z and G(z) = G(t, z) equals G(z) for z = (t, z) , but notethat G could also be a new function depending on both t and z . Note moreoverthat X itself could be of the form Xt = (t, Yt) for t ≥ 0 where Y = (Yt)t≥0 isa Markov process, so that (6.1.2) even if G ≡ G reads

V (t, y) = sup0≤τ≤T−t

Et,yM(t+τ, Yt+τ ) (6.1.3)

where Yt = y under Pt,y and M stands for G . Various particular cases of theproblem (6.1.3) will be studied in Chapters VI–VIII below.

6.2. Dimension of the problem

Dimension of the problem refers to the minimal dimension of an underlying Markovprocess which leads to the solution. The latter Markov process does not need to

Section 6. MLS formulation of optimal stopping problems 127

be the same as the initial Markov process. For example, the problem (6.0.9) isthree-dimensional generally since Z is a three-dimensional Markov process, butdue to the linear structure of I (or Ito’s formula to a similar end) it is also possibleto view the problem as being two-dimensional, both being valid when the horizonis infinite. On the other hand, when the horizon is finite, then the same problemis four-dimensional generally, but due to the linear structure of I may also beviewed as three-dimensional.

To determine the dimension of a problem is not always a simple matter. Wewill see in Chapter IV how the initial dimension can be reduced by the methodof time change (Section 10), the method of space change (Section 11), and themethod of measure change (Section 12). These three methods are stochastic bytheir nature and each corresponds to a deterministic change of variables thatreduces the initial more complicated equation (e.g. PDE) to a simpler equation(e.g. ODE). This equivalence is best tested and understood via specific examples(cf. Sections 10–12 and Sections 26–27).

6.3. Killed (discounted) problems

One is often more interested in the killed (discounted) version of the optimalstopping problem (6.0.1) that reads

V = supτ

E

(e−λτ M(Xτ ) +

∫ τ

0

e−λtL(Xt) dt + e−λτ sup0≤t≤τ

K(Xt))

(6.3.1)

where the killing (discounting) process λ = (λt)t≥0 is given by

λt =∫ t

0

λ(Xs) ds (6.3.2)

for a measurable (continuous) function λ : E → R+ called the killing (discount-ing) rate.

The problem (6.3.1) reduces to the initial problem (6.0.1) by replacing theunderlying Markov process X with the new Markov process X which correspondsto the “killing” of the sample paths of X at the “rate” λ(X) (cf. Subsection 5.4).The infinitesimal generator of X is given by

L eX = LX − λI (6.3.3)

where I is the identity operator (see (5.4.10)) and all what is said above or belowfor X extends to X by replacing LX with L eX . Specific killed (discounted)problems are studied in Chapter VII below.


7. MLS functionals and PIDE problems

Throughout the section we will adopt the setting and notation of Subsection 2.2(recalled in the beginning of Section 6 above).

Motivated by the representation (1) on page 123 let us assume that we aregiven a (bounded) open set C ⊆ E and let us consider

τD = inf t ≥ 0 : Xt ∈ D (7.0.4)

where D = Cc (= E \ C) . Given a mapping G : D → R the question then arisesto determine a differential (or integro-differential) equation solved by

F (x) = ExG(XτD ) (7.0.5)

for x ∈ E . Moreover, when the representation (1) on page 123 gets the morespecific form (6.0.1), we see that the question naturally splits into three new sub-questions (corresponding to M , L and K ).

The purpose of the present section is to describe answers to these questions.These answers are based on a fundamental link between probability (Markovprocess) and analysis (differential/integral equation). When connected with themeaning of the representation (1) on page 123 this will lead to the formulation ofa free-boundary problem in Section 8.

1. For further reference let us recall the following three facts playing the keyrole in the sequel.

• The strong Markov property of X can be expressed as

Ex(H θτ | Fτ ) = EXτ H (7.0.6)

where τ is a stopping time and H is a (bounded or non-negative) measurablefunctional (see (4.3.28)).

• If σ ≤ τ where σ is a stopping time and τ is a hitting/entry time to aset, then

τ = σ + τ θσ. (7.0.7)

• For all stopping times τ and σ we have

Xτ θσ = Xσ+τθσ . (7.0.8)

(For (7.0.6) recall (4.3.28), for (7.0.7) recall (4.1.25), and for (7.0.8) recall(4.1.13).)

2. The mean-value kinematics of the process X is described by the charac-teristic operator LX defined on a function F : E → R as follows:

LXF (x) = limU↓x

ExF (XτUc ) − F (x)ExτUc

(7.0.9)

Section 7. MLS functionals and PIDE problems 129

where the limit is taken over a family of open sets U shrinking down to x inE .

Under rather general conditions it is possible to establish (see [53]) that thecharacteristic operator is an extension of the infinitesimal operator LX definedon a function F : E → R as follows:

LXF (x) = limt↓0

ExF (Xt) − F (x)t

(7.0.10)

for x ∈ E .

We will not find it necessary to specify the domains of these two operatorsand for this reason the same symbol LX will be used to denote both. Very often wewill refer to LX as the infinitesimal generator of the process X . Its infinitesimalrole is uniquely determined through its action on sufficiently regular (smooth)functions F for which both limits (7.0.9) and (7.0.10) exist and coincide. Thisleads to the equations which will now be described.

3. From the general theory of Markov processes (see [53]) we know that (ona given relatively compact subset of E ) the infinitesimal generator LX takes thefollowing integro-differential form:

LXF (x) = λ(x)F (x) +d∑

i=1

ρi(x)∂F

∂xi(x) +

d∑i,j=1

σij(x)∂2F

∂xi ∂xj(x) (7.0.11)

+∫

Rd\0

(F (y) − F (x) −

d∑i=1

(yi − xi)∂F

∂xi(x))

ν(x, dy)

where λ corresponds to killing (when positive) or creation (when negative), ρ isthe drift coefficient, σ is the diffusion coefficient, and ν is the compensator ofthe measure µ of jumps of X .

If X is continuous then ν ≡ 0 and LX does not contain the final (integral)term in (7.0.11). If X cannot be killed or created then λ ≡ 0 and LX doesnot contain the initial term in (7.0.11). Similar interpretations hold for the driftcoefficient ρ and the diffusion coefficient σ . Each of the four terms λ , ρ , σ andν in (7.0.11) has a transparent meaning (e.g. when Xt denotes the position ofa particle in the fluid under external influence or the value of a stock price in afinancial market).

4. Regular boundary. We will say that the boundary ∂C of C is regular(for D ) if each point x from ∂C is regular (for D ) in the sense that Px(σD =0) = 1 where σD = inf t > 0 : Xt ∈ D . Thus, if X starts at a regular pointfor D , then X enters D immediately after taking off. It turns out that thenotion of regularity of ∂C is intimately related to a regularity of the mapping(7.0.5) at ∂C (in the sense of continuity or smoothness).


To simplify the notation let us agree in the sequel that ∂C stands for Dwhen X is discontinuous. It would be sufficient in fact to include only those pointsin D that can be reached by X when jumping from C (apart from the boundarypoints of C ).

7.1. Mayer functional and Dirichlet problem

1. Given a continuous function M : ∂C → R consider

F (x) = ExM(XτD) (7.1.1)

for x ∈ E . The function F solves the Dirichlet problem:

LXF = 0 in C, (7.1.2)

F∣∣∂C

= M (7.1.3)

(cf. (4.2.50) and (4.2.51)).

Indeed, given x ∈ C choose a (bounded) open set U such that x ∈ U ⊆ C .By the strong Markov property (7.0.6) with (7.0.7) and (7.0.8) we have

ExF (XτUc ) = ExEXτUcM(XτD) = ExEx

(M(XτD) θτUc

∣∣FτUc

)(7.1.4)

= ExM(XτUc+τDθτUc

)= ExM(XτD) = F (x).

Hence we see thatlimU↓x


≡ 0 (7.1.5)

proving (7.1.2) as claimed. The condition (7.1.3) is evident.

2. Continuity on C . Let us assume that X is continuous, and let xn ∈ Cconverge to x ∈ ∂C as n → ∞ . Then

F (xn) = ExnM(XτD) = EM(XxnτD

) −→ EM(XxσD

) = ExM(XσD ) (7.1.6)

where σD = inf t > 0 : Xt ∈ D and the convergence takes place since both Xand M are continuous. Moreover, if x is regular for D , then σD = 0 under Px

and thus ExM(XσD ) = M(x) = F (x) . This shows:

If ∂C is a regular boundary (for D ), then F is continuous on C . (7.1.7)

In particular, if ∂C is a regular boundary (for D ), then F is continuous at ∂C .Only special C however will have the power of making F smooth at ∂C . This isintimately related to the principle of smooth fit discussed in Subsection 9.1 below:Such a set C will be optimal.

If X is discontinuous, then F also may generally be discontinuous at ∂C .Only special C however will have the power of making F continuous at ∂C .


This is intimately related to the principle of continuous fit discussed in Subsection9.2 below: Such a set will be optimal.

3. Smoothness in C . Let us assume that X is continuous and let us considerthe Dirichlet problem (7.1.2)–(7.1.3) with LX in (7.0.11) without the λ -term andthe N -term. Standard PDE results then state that if ρ and σ are sufficientlysmooth and ∂C is sufficiently regular, then there exists a solution F to (7.1.2)–(7.1.3) which is (equally) smooth in C and continuous on C . (Frequently F willbe smooth at least as ρ and σ .) To avoid any discussion of ∂C which generally isnot easily accessible in free-boundary problems, we can apply the preceding PDEresult locally around the given point x ∈ C to B = b(x, r) ⊆ C where r > 0 issufficiently small. This gives the existence of a solution f to the Dirichlet problem:

LXf = 0 in B, (7.1.8)

f∣∣∂B

= F (7.1.9)

such that f is smooth in B and continuous on B . (Obviously ∂B is taken to besufficiently regular.) Applying Ito’s formula (page 67) to f(Xt) , setting t = τBc ,taking Ex on both sides upon making use of (7.1.8) and the optional samplingtheorem (page 60) using localization arguments if needed, we find by means of(7.1.9) that f(x) = ExF (XτBc ) which in turn equals F (x) by (7.1.4) above.Thus F equals f on B and hence is smooth at x in C .

The preceding technique enables one to use the smoothness results from thePDE theory and carry them over to the function F defined in (7.1.1). Note thatthis requires only conditions on ρ and σ and that the smoothness of F holdsonly in the interior C of C and not at ∂C generally. Since in all examples tobe studied below the remaining details above are easily verified, we will freely usethe smoothness of F in C without further mention (cf. Chapters VI–VIII).

4. Killed version. Given a continuous function M : ∂C → R consider

F (x) = Ex

(e−λτD M(XτD)

)(7.1.10)

for x ∈ E where λ = (λt)t≥0 is given by

λt =∫ t

0

λ(Xs) ds (7.1.11)

for a measurable (continuous) function λ : E → R+ . The function F solves the(killed) Dirichlet problem:

LXF = λF in C, (7.1.12)

F∣∣∂C

= M. (7.1.13)

Indeed, replacing X by the killed process X it follows that (7.1.10) readsas

F (x) = ExM(XτD) (7.1.14)


and the infinitesimal generator of X is given by

L eX = LX − λI (7.1.15)

(recall Subsection 6.3 above). Applying (7.1.2) and (7.1.3) to (7.1.14) and (7.1.15)we see that (7.1.12) and (7.1.13) hold as claimed.

7.2. Lagrange functional and Dirichlet/Poisson problem

1. Given a continuous function L : C → R consider

F (x) = Ex

∫ τD

0

L(Xt) dt (7.2.1)

for x ∈ E . The function F solves the Dirichlet/Poisson problem:

LXF = −L in C, (7.2.2)

F∣∣∂C

= 0 (7.2.3)

(cf. (4.2.49) and (4.2.48)).

Indeed, given x ∈ C choose a (bounded) open set U such that x ∈ U ⊆ C .By the strong Markov property (7.0.6) with (7.0.7) and (7.0.8) we have

ExF (XτUc ) = Ex EXτUc

∫ τD

0

L(Xt) dt (7.2.4)

= ExEx

(∫ τD

0

L(Xt) dt θτUc

∣∣∣ FτUc

)= Ex

∫ τDθτUc

0

L(Xt θτUc ) dt = Ex

∫ τD−τUc

0

L(XτUc+t) dt

= Ex

∫ τD

τUc

L(Xs) ds = Ex

∫ τD

0

L(Xt) dt − Ex

∫ τUc

0

L(Xt) dt

= F (x) − Ex

∫ τUc

0

L(Xt) dt

upon substituting τUc + t = s . Hence we see that

limU↓x


= limU↓x

(− 1

ExτUc

Ex

∫ τUc

0

L(Xt) dt

)= −L(x) (7.2.5)

by the continuity of L . This proves (7.2.2) while (7.2.3) is evident.

2. Continuity on C . The same arguments as in Subsection 7.1 above showthat if ∂C is a regular boundary (for D ), then F is continuous on C .


3. Smoothness in C . The same techniques as in Subsection 7.1 above enableone to use the smoothness results from the PDE theory and carry them over tothe function F defined in (7.2.1).

4. Killed version. Given a continuous function L : C → R consider

F (x) = Ex

∫ τD

0

e−λtL(Xt) dt (7.2.6)

for x ∈ E where λ = (λt)t≥0 is given by

λt =∫ t

0

λ(Xs) ds (7.2.7)

for a measurable (continuous) function λ : E → R+ . The function F solves the(killed) Dirichlet/Poisson problem:

LXF = λF − L in C, (7.2.8)

F∣∣∂C

= 0. (7.2.9)

Indeed, replacing X by the killed process X it follows that (7.2.6) reads as

F (x) = Ex

∫ τD

0

L(Xt) dt (7.2.10)


L eX = LX − λI (7.2.11)


7.3. Supremum functional and Neumann problem

In this subsection we will assume that X is continuous and takes values in E = R .Set

St = max0≤s≤t

Xs (7.3.1)

for t ≥ 0 . Then (X, S) = (Xt, St)t≥0 is a Markov process with the state spaceE = (x, s) ∈ E2 : x ≤ s and S increases only when X = S i.e. at the (main)diagonal of E . We have (X0, S0) = (x, s) under Px,s for (x, s) ∈ E .

Given a (bounded) open set C ⊆ E let us consider

τD = inf t ≥ 0 : (Xt, St) ∈ D (7.3.2)

where D = Cc . Then (7.1.2)–(7.1.3) extends as follows.


1. Given a continuous function M : ∂C → R consider

F (x, s) = Ex,sM(XτD , SτD ) (7.3.3)

for (x, s) ∈ E . The function F solves the Neumann problem:

LXF = 0 for x < s with s fixed , (7.3.4)

∂F

∂s(x, s) = 0 for x = s, (7.3.5)

F∣∣∂C

= M. (7.3.6)

Indeed, since (X, S) can be identified with X when off the diagonal in E ,the identities (7.3.4) and (7.3.6) follow in the same way as (7.1.2) and (7.1.3).

To verify (7.3.5) we shall first note that the proof of (7.1.2) shows that

limU↓(x,s)

Ex,sF (XτUc , SτUc ) − F (x, s)Ex,sτUc

≡ 0 (7.3.7)

for (x, s) ∈ E . In particular, this holds for all points (s, s) at the diagonal of E .

Next, without loss of generality, assume that F is sufficiently smooth (e.g.C2,1 ). Applying Ito’s formula (page 67) to F (Xt, St) , taking Es,s on both sidesand applying the optional sampling theorem (page 60) to the continuous martin-gale (localized if needed) which appears in the identity obtained, we get

Es,sF (Xt, St) − F (s, s)t

= Es,s

(1t

∫ t

0

(LXF )(Xs, Ss) ds

)(7.3.8)

+ Es,s

(1t

∫ t

0

∂F

∂s(Xs, Ss) ds

)−→ LXF (s, s) +

∂F

∂s(s, s) lim

t↓0Es,s(St − s)

t

as t ↓ 0 . Due to σ > 0 we have t−1Es,s(St − s) → ∞ as t ↓ 0 , and therefore thelimit in (7.3.8) does not exist (and is finite) unless (7.3.5) holds.

Facts on the continuity on C , smoothness in C , and a killed version of(7.3.3) carry over from Subsections 7.1 and 7.2 above to the present case of functionF without major changes. Further details in this direction will be omitted.

2. Given a measurable (continuous) function L : E → R set

It =∫ t

0

L(Xs) ds (7.3.9)

for t ≥ 0 . Then (I, X, S) = (It, Xt, St)t≥0 is a Markov process with the statespace E = (a, x, s) ∈ E3 : x ≤ s . We have (I0, X0, S0) = (a, x, s) under Pa,x,s .


Given a (bounded) open set C ⊆ E let us consider

τD = inf t ≥ 0 : (It, Xt, St) ∈ D (7.3.10)

where D = Cc . A quick way to reduce this case to the preceding case above is toreplace the Markov process X in the latter with the Markov process X = (I, X)coming out of the former. This leads to the following reformulation of (7.3.3)–(7.3.6) above.

Given a continuous function M : ∂C → R consider

F (a, x, s) = Ea,x,sM(IτD , XτD , SτD) (7.3.11)

for (a, x, s) ∈ E . The function F solves the Neumann problem:

LXF = −L for x < s with s fixed, (7.3.12)

∂F

∂s(a, x, s) = 0 for x = s, (7.3.13)

F∣∣∂C

= M. (7.3.14)

Indeed, replacing X by X = (I, X) we find that

LX = LX + L. (7.3.15)

Hence (7.3.12)–(7.3.14) reduce to (7.3.4)–(7.3.6). 3. Facts on the continuity on C , smoothness in C , and a killed version

of (7.3.11) carry over from Subsections 7.1 and 7.2 above to the present caseof function F without major changes. Further details in this direction will beomitted.

7.4. MLS functionals and Cauchy problem

1. Given a continuous function M : E → R consider

F (t, x) = ExM(Xt) (7.4.1)

for (t, x) ∈ R+ × E . The function F solves the Cauchy problem:

∂F

∂t= LXF in R+×E, (7.4.2)

F (0, x) = M(x) for x ∈ E. (7.4.3)

Indeed, let us show how (7.4.1) reduces to (7.1.1) so that (7.1.2)–(7.1.3)become (7.4.2)–(7.4.3).


For this, define a new Markov process by setting

Ys = (t−s, Xs) (7.4.4)

for s ≥ 0 . Then the infinitesimal generator of Y = (Ys)s≥0 equals

LY = − ∂

∂s+ LX . (7.4.5)

Consider the exit time of Y from the open set C = (0,∞)×E given by

τD = inf s ≥ 0 : Ys ∈ D (7.4.6)

where D = Cc . Then obviously τD ≡ t and thus

F (t, x) = Et,xM(YτD ) (7.4.7)

where M(u, x) = M(x) for u ≥ 0 . In this way (7.4.1) has been reduced to (7.1.1)and thus by (7.1.2) we get

LY F = 0. (7.4.8)

From (7.4.5) we see that (7.4.8) is exactly (7.4.2) as claimed. The condition (7.4.3)is evident.

2. Killed version (Mayer). Given a continuous function M : E → R consider

F (t, x) = Exe−λtM(Xt) (7.4.9)

for (t, x) ∈ R+×E where λ = (λt)t≥0 is given by

λt =∫ t

0

λ(Xs) ds (7.4.10)

for a measurable (continuous) function λ : E → R+ . The function F solves the(killed) Cauchy problem:

∂F

∂t= LXF − λF in R+ × E, (7.4.11)

F (0, x) = M(x) for x ∈ E. (7.4.12)

Indeed, replacing X by the killed process X it follows that (7.4.9) reads as

F (t, x) = ExM(Xt) (7.4.13)


L eX = LX − λI (7.4.14)



The expression (7.4.9) is often referred to as the Feynman–Kac formula.

3. Given a continuous function L : E → R consider

F (t, x) = Ex

(∫ t

0

L(Xs) ds

)(7.4.15)

for (t, x) ∈ R+×E . The function F solves the Cauchy problem:

∂F

∂t= LXF + L in R+×E, (7.4.16)

F (0, x) = 0 for x ∈ E. (7.4.17)

Indeed, this can be shown in exactly the same way as in the proof of (7.4.2)–(7.4.3) above by reducing (7.4.15) to (7.2.1) so that (7.2.2)–(7.2.3) become(7.4.16)–(7.4.17).

4. Killed version (Lagrange). Given a continuous function L : E → R con-sider

F (t, x) = Ex

∫ t

0

e−λsL(Xs) ds (7.4.18)

for (t, x) ∈ R+×E where λ = (λt)t≥0 is given by

λt =∫ t

0

λ(Xs) ds (7.4.19)

for a measurable (continuous) function λ : E → R+ . The function F solves the(killed) Cauchy problem:

∂F

∂t= LXF − λF − L, (7.4.20)

F (0, x) = 0 for x ∈ E. (7.4.21)

Indeed, replacing X by the killed process X it follows that (7.4.18) readsas

F (t, x) = Ex

∫ t

0

L(Xs) ds (7.4.22)


L eX = LX − λI (7.4.23)

(recall Subsection 6.3 above). Applying (7.4.16) and (7.4.17) to (7.4.22) and(7.4.23) we see that (7.4.20) and (7.4.21) hold as claimed.


The expression (7.4.18) is often referred to as the Feynman–Kac formula.

5. Mixed case (Bolza). Given M , L and λ as above, consider

F (t, x) = Ex

(e−λtM(Xt) +

∫ t

0

e−λsL(Xs) ds

)(7.4.24)

for (t, x) ∈ R+×E . The function F solves the (killed) Cauchy problem:

∂F

∂t= LXF − λF + L in R+×E, (7.4.25)

F (0, x) = M(x) for x ∈ E. (7.4.26)

Indeed, this follows from (7.4.11)–(7.4.12) and (7.4.20)–(7.4.21) using linear-ity.

6. Mixed case (general). Given M , L and λ as above, consider

F (t, x, s) = Ex,s

(e−λtM(Xt) +

∫ t

0

e−λsL(Xs) ds + e−λtSt

)(7.4.27)

for (t, x, s) ∈ R+×E where (X0, S0) = (x, s) under Px,s . The function F solvesthe (killed) Cauchy problem:

∂F

∂t= LXF − λF + L for x < s with s fixed, (7.4.28)

∂F

∂s(t, x, s) = 0 for x = s with t ≥ 0, (7.4.29)

F (0, x, s) = M(x) + s for x ≤ s. (7.4.30)

Indeed, replacing (X, S) by the killed process (X, S) we see that (7.4.27)reads as follows:

F (t, x, s) = Ex,s

(M(Xt) +

∫ t

0

L(Xs) ds + St

)(7.4.31)

and the infinitesimal operator of (X, S) is given by:

LX − λI for x < s, (7.4.32)

∂

∂s= 0 for x = s. (7.4.33)

Setting M(x, s) = M(x) + s we see that (7.4.31) reduces to (7.4.24) so that(7.4.28)–(7.4.30) follow by (7.4.25)–(7.4.26) using (7.4.32)–(7.4.33).


7.5. Connection with the Kolmogorov backward equation

Recall that the Kolmogorov backward equation (in dimension one) reads (see(4.3.7))

∂f

∂t+ ρ

∂f

∂x+

σ2

2∂2f

∂x2= 0 (7.5.1)

where f is the transition density function of a diffusion process X given by

f(t, x; u, y) =d

dyP(Xu ≤ y |Xt = x) (7.5.2)

for t < u in R+ and x , y in E .

1. Let us show that the equation (7.5.1) in relation to (7.5.2) has the samecharacter as the equation (7.1.2) in relation to (7.1.1). For this, let us assume thatwe know (7.1.2) and let us show that this yields (7.5.1). If we define a new Markovprocess by setting

Zs = (t+s, Xt+s) (7.5.3)

with Z0 = (t, x) under Pt,x , then the infinitesimal generator of Z = (Zs)s≥0

equals

LZ =∂

∂s+ LX (7.5.4)

where LX = ρ ∂/∂x + (σ2/2) ∂2/∂x2 .

Consider the exit time from the set C = [0, u)×E given by

τD = inf s ≥ 0 : Zs ∈ D (7.5.5)

where D = Cc . Then obviously τD ≡ u and thus

H(t, x) = Et,xM(ZτD ) = Et,xM(Zu) (7.5.6)

satisfies the equation (7.1.2) where M(t, x) = M(x) and M is a given functionfrom E to R . This yields

LZH = 0 (7.5.7)

which in view of (7.5.4) reduces to (7.5.1) by approximation.

In exactly the same way (choosing M in (7.5.6) to be an indicator function)one sees that

F (t, x; u, y) = P(Xu ≤ y |Xt = x) (7.5.8)

satisfies the equation (7.5.1) i.e.

∂F

∂t+ ρ

∂F

∂x+

σ2

2∂2F

∂x2= 0 (7.5.9)


and (7.5.1) follows by (formal) differentiation of (7.5.9) with respect to y (recall(7.5.2) above). Note also that M in (7.1.1) does not need to be continuous for(7.1.2) to be valid (inspect the proof).

On the other hand, note that (7.5.1) in terms of (7.5.3) reads as follows:

LZf = 0 (7.5.10)

and likewise (7.5.9) reads as follows:

LZF = 0. (7.5.11)

This shows that (7.5.1) and (7.5.9) are the same equations as (7.1.2) above.

2. The preceding considerations show that (7.5.1) in relation to (7.5.2) and(7.1.2) in relation to (7.1.1) are refinements of each other but the same equations.Given the central role that (7.1.1)–(7.1.2) play in the entire section above, whereall equations under consideration can be seen as special cases of these relations, itis clear that the derivation of (7.1.2) via (7.1.4) above embodies the key Markovianprinciple which govern all these equations. This is a powerful unifying tool whicheveryone should be familiar with.

3. Time-homogeneous case (present-state and future-time mixed). Whenρ and σ in (7.5.1) do not depend on time, i.e. when the process X is time-homogeneous, then

f(t, x; u, y) = f(0, x; u − t, y) (7.5.12)

so that (7.5.1) becomes

−∂f

∂s+ ρ

∂f

∂x+

σ2

2∂2f

∂x2= 0 (7.5.13)

where f = f(0, x; s, y) . Note that this equation has the same form as the equation(7.4.2). Moreover, on closer inspection one also sees that the proof of (7.4.2) followsthe same pattern as the proof of (7.5.1) via (7.1.2) above.

Finally, note also when the process X is time-homogeneous that the semi-group formulation of the Kolmogorov backward (and forward) equation (see (4.3.7)and (4.3.8)) has the following form:

d

dtPtf = LXPtf (= PtLXf) (7.5.14)

where Ptf(x) = Exf(Xt) for a bounded (continuous) function f : E → R .

Notes. Stochastic control theory deals with three basic problem formulationswhich were inherited from classical calculus of variations (cf. [67, pp. 25–26]).Given the equation of motion

dXt = ρ(Xt, ut) dt + σ(Xt, ut) dBt (7.5.15)


where (Bt)t≥0 is standard Brownian motion, consider the optimal control problem

infu

Ex

(∫ τD

0

L(Xt, ut) dt + M(XτD

))

(7.5.16)

where the infimum is taken over all admissible controls u = (ut)t≥0 applied beforethe exit time τ

D= inf t>0 : Xt /∈ C for some open set C = Dc and the process

(Xt)t≥0 starts at x under Px . If M ≡ 0 and L = 0 , the problem (7.5.16) issaid to be Lagrange formulated. If L ≡ 0 and M = 0 , the problem (7.5.16) issaid to be Mayer formulated. If both L = 0 and M = 0 , the problem (7.5.16) issaid to be Bolza formulated.

The Lagrange formulation goes back to the 18th century, the Mayer formula-tion originated in the 19th century, and the Bolza formulation [20] was introducedin 1913. We refer to [19, pp. 187–189] with the references for a historical ac-count of the Lagrange, Mayer and Bolza problems. Although the three problemformulations are formally known to be equivalent (see e.g. [19, pp. 189–193] or[67, pp. 25–26]), this fact is rarely proved to be essential when solving a concreteproblem.

Setting Zt = L(Xt, ut) or Zt = M(Xt) , and focusing upon the sample patht → Zt for t ∈ [0, τD ] , we see that the three problem formulations measure theperformance associated with a control u by means of the following two functionals:∫ τ

D

0

Zt dt & ZτD

(7.5.17)

where the first one represents the surface area below (or above) the sample path,and the second one represents the sample-path’s terminal value. In addition tothese two functionals, it is suggested by elementary geometric considerations thatthe maximal value of the sample path

max0≤t≤τ

D

Zt (7.5.18)

provides yet another performance measure which, to a certain extent, is moresensitive than the previous two ones. Clearly, a sample path can have a smallintegral but still a large maximum, while a large maximum cannot be detected bythe terminal value either.

A purpose of the present chapter was to point out that the problem formula-tions based on a maximum functional can be successfully added to optimal controltheory (calculus of variations) and optimal stopping. This suggests a number ofnew avenues for further research upon extending the Bolza formulation (6.1.2) tooptimize the following expression:

Ex

(∫ τD

0

L(Xt, ut) dt + M(XτD

) + max0≤t≤τ

D

K(Xt, ut))

(7.5.19)

where some of the maps K , L and M may also be identically zero.


Optimal stopping problems for the maximum process have been studied bya number of authors in the 1990’s (see e.g. [103], [45], [185], [159], [85]) and thesubject seems to be well understood now. The present monograph will exposesome of these results in Chapters IV–VIII below.

Chapter IV.

Methods of solution

8. Reduction to free-boundary problem

Throughout we will adopt the setting and notation of Subsection 2.2. Thus X =(Xt)t≥0 is a strong Markov process (right-continuous and left-continuous overstopping times) taking values in E = Rd for some d ≥ 1 .

1. Given a measurable function G : E → R satisfying needed regularityconditions, consider the optimal stopping problem

V (x) = supτ

ExG(Xτ ) (8.0.1)

where the supremum is taken over all stopping times τ of X , and X0 = x underPx with x ∈ E . In Chapter I we have seen that the problem (8.0.1) is equivalentto the problem of finding the smallest superharmonic function V : E → R (to beequal to V ) which dominate the gain function G on E . In this case, moreover,the first entry time τD of X into the stopping set D = V = G is optimal,and C = V > G is the continuation set. As already pointed out at the end ofChapter I, it follows that V and C should solve the free-boundary problem

LX V ≤ 0 (V minimal), (8.0.2)

V ≥ G (V > G on C & V = G on D) (8.0.3)

where LX is the infinitesimal generator of X (cf. Chapter III above). It is im-portant to realize that both V and C are unknown in the system (8.0.2)–(8.0.3)(and both need to be determined).

2. Identifying V = V , upon invoking sufficient conditions at the end ofChapter I that make this identification possible, it follows that V admits thefollowing representation:

V (x) = ExG(XτD ) (8.0.4)

144 Chapter IV. Methods of solution

for x ∈ E where τD is the first entry time of X into D given by

τD = inf t ≥ 0 : Xt ∈ D . (8.0.5)

The results of Chapter III (for the Dirichlet problem) become then applicable andaccording to (7.1.2)–(7.1.3) it follows that

LXV = 0 in C, (8.0.6)

V∣∣D

= G∣∣D

. (8.0.7)

Note that (8.0.6) stands in accordance with the general fact from Chapter I that(V (Xt∧τD))t≥0 is a martingale. Note also that X is multidimensional and thuscan generally be equal to the time-integral-space-maximum process considered inChapter III (including killed versions of these processes as well). In this way wesee that the Dirichlet problem (8.0.6)–(8.0.7) embodies all other problems (Dirich-let/Poisson, Neumann, Cauchy) considered in Chapter III. Fuller details of theseproblem formulations are easily reconstructed in the present setting and for thisreason will be omitted.

3. The condition (8.0.2) states that V is the smallest superharmonic function(which dominates G ). The two properties “smallest” and “superharmonic” play adecisive role in the selection of the optimal boundary ∂C (i.e. sets C and D ) inthe sense that only special sets C (i.e. D ) will qualify to meet these properties.Indeed, assuming that G is smooth (in a neighborhood of ∂C ) the followinggeneral picture (stated more as a ‘rule of thumb’) is valid.

If X after starting at ∂C enters int (D) immediately (e.g. when X is adiffusion and ∂C is sufficiently regular e.g. Lipschitz) then the condition (8.0.2)leads to

∂V

∂x

∣∣∣∂C

=∂G

∂x

∣∣∣∂C

(smooth fit) (8.0.8)

where d = 1 is assumed for simplicity (in the case d > 1 one should replace ∂/∂xin (8.0.8) by ∂/∂xi for 1 ≤ i ≤ d ). However, if X after starting at ∂C does notenter int (D) immediately (e.g. when X has jumps and no diffusion componentwhile ∂C may still be sufficiently regular e.g. Lipschitz) then the condition (8.0.2)leads to

V∣∣∂C

= G∣∣∂C


The more precise meaning of these conditions will be discussed in Section 9 below.

8.1. Infinite horizon

Infinite horizon problems in dimension one are generally easier than finite hori-zon problems since the equation (8.0.6) (or its killed version) can often be solvedexplicitly (in a closed form) yielding a candidate function to which a verification

Section 8. Reduction to free-boundary problem 145

procedure (stochastic calculus) can be applied. Especially transparent in this con-text is the case of one-dimensional diffusions X where the existence of explicitsolutions (scale function, speed measure) reduces the study of optimal stopping tothe case of standard Brownian motion.

1. To illustrate the latter in more detail, let us assume that X is a one-dimensional diffusion solving the following SDE (stochastic differential equation):

dXt = ρ(Xt) dt + σ(Xt) dBt (8.1.1)

and let us consider the optimal stopping problem

V (x) = supτ

ExG(Xτ ) (8.1.2)

where the supremum is taken over all stopping times τ of X , and X0 = x underPx with x ∈ R .

Denoting by S the scale function of X (see (5.2.29)) and writing

G(Sτ ) = G(S−1 S(Xτ )

)= (G S−1)(S(Xτ )) = G(Mτ ) = G(Bστ ) (8.1.3)

where G = G S−1 is a new gain function, M = S(X) is a continuous localmartingale and B is a standard Brownian motion with σt = 〈M, M〉t (by Lemma5.1), we see that (8.0.1) reads

V (x) = supσ

ES(x)G(Bσ) (8.1.4)

where the supremum is taken over all stopping times σ of B (recall that τ isa stopping time of M if and only if στ is a stopping time of B ). This showsthat the optimal stopping problem (8.1.2) is equivalent to the optimal stoppingproblem (8.1.4). Moreover, it is easily verified by Ito’s formula (page 67) that

στ = 〈M, M〉τ =∫ τ

0

S′(Xs)2 σ(Xs)2 ds (8.1.5)

for every stopping time of X i.e. M . This identity establishes a transparent one-to-one correspondence between the optimal stopping time in the problem (8.1.2)and the optimal stopping time in the problem (8.1.4): having one of them given,we can reconstruct the other, and vice versa.

2. Recalling that the infinitesimal generator of the Brownian motion Bequals (1/2) ∂2/∂x2 we see that a smooth function V : R → R is superharmonicif and only if V ′′ ≤ 0 i.e. if and only if V is concave. This provides a transparentgeometric interpretation of superharmonic functions for standard Brownian mo-tion. Making use of the scale function S and exploiting the equivalence of (8.1.2)


V

G

Figure IV.1: An obstacle G and the rope V depicting the superharmoniccharacterization.

and (8.1.4), this geometric interpretation (in somewhat less transparent form) ex-tends from B to general one-dimensional diffusions X considered in (8.1.2). Sincethese details are evident but somewhat lengthy we shall omit further discussion(see Subsection 9.3 below).

The geometric interpretation of superharmonic functions for B leads to anappealing physical interpretation of the value function V associated with the gainfunction G : If G depicts an obstacle, and a rope is put above G with both endspulled to the ground, the resulting shape of the rope coincides with V (see FigureIV.1). Clearly the fit of the rope and the obstacle should be smooth wheneverthe obstacle is smooth (smooth fit). A similar interpretation (as in Figure IV.1)extends to dimension two (membrane) and higher dimensions. This leads to a classof problems in mathematical physics called the “obstacle problems”.

8.2. Finite horizon

Finite horizon problems (in dimension one or higher) are more difficult than infinitehorizon problems since the equation (8.0.6) (or its killed version) contains the ∂/∂tterm and most often cannot be solved explicitly (in a closed form). Thus, in thiscase it is not possible to produce a candidate function to which a verificationprocedure is to be applied. Instead one can try to characterize V and C (i.e. D )by means of the free-boundary problem derived above. A more refined method (justas in the case of two algebraic equations with two unknowns) aims at expressingV in terms of ∂C and then deriving a (nonlinear) equation for ∂C . This line ofargument will be presented in more detail in Subsection 14.1 below, and examplesof application will be given in Chapters VI–VIII below.

Section 9. Superharmonic characterization 147

1. To illustrate the former method in more detail, let us consider the optimalstopping problem


Et,xG(t+τ, Xt+τ ) (8.2.1)

where the supremum is taken over all stopping times τ of X , and Xt = x underPt,x for (t, x) ∈ [0, T ] × E . At this point it is useful to recall our discussion inSubsections 2.2 and 6.1 explaining why X needs to be replaced by the time-spaceprocess Zt = (t, Xt) in the finite-horizon formulation (8.0.1). It implies that thepreceding discussion leading to the free-boundary problem (8.0.2)–(8.0.3) as wellas (8.0.6)–(8.0.7) and (8.0.8) or (8.0.9) applies to the process Z instead of X .In the case when X is a diffusion, and when ∂C is sufficiently regular (e.g.Lipschitz), we see that (8.0.6)–(8.0.7) and (8.0.8) read:

Vt + LXV = 0 in C, (8.2.2)

V∣∣D

= G∣∣D

, (8.2.3)

∂V

∂x

∣∣∣∂C

=∂G

∂x

∣∣∣∂C

(smooth fit) (8.2.4)

where d = 1 is assumed in (8.2.4) for simplicity (in the case d > 1 one shouldreplace ∂/∂x in (8.2.4) by ∂/∂xi for 1 ≤ i ≤ d ). It should be noted in (8.2.3)that all points (T, x) belong to D when x ∈ E . In the case when X has jumpsand no diffusion component, and when ∂C may still be sufficiently nice (e.g.Lipschitz), the condition (8.2.4) needs to be replaced by

V∣∣∂C

= G∣∣∂C


The question of existence and uniqueness of the solution to the free-boundaryproblem (8.2.2)–(8.2.3) with (8.2.4) or (8.2.5) will be studied through specificexamples in Chapters VI–VIII.

2. Another class of problems coming from mathematical physics fits intothe free-boundary setting above. These are processes of melting and solidificationleading to the “Stefan free-boundary problem”. Imagine a chunk of ice (at temper-ature G ) immersed in water (at temperature V ). Then the ice-water interface(as a function of time and space) will coincide with the optimal boundary (sur-face) ∂C . This illustrates a basic link between optimal stopping and the Stefanproblem.

9. Superharmonic characterization

In this section we adopt the setting and notation from the previous section. Recallthat the value function from (8.0.1) can be characterized as the smallest superhar-monic function (relative to X ) which dominates G (on E ). As already pointed


G

A B

C0

C0x | VA,B(x)

Figure IV.2: The function x → VA,B(x) from (9.0.7) above when X is aMarkov process with diffusion component.

out in the previous section, the two properties “smallest” and “superharmonic”play a decisive role in the selection of the optimal boundary ∂C (i.e. sets C andD ).

To illustrate the preceding fact in further detail, let us for simplicity assumethat E = R and that C equals a bounded open interval in E (often this factis evident from the form of X and G ). By general theory (Chapter I) we thenknow that the exit time

τA,B = inf t ≥ 0 : Xt /∈ (A, B) (9.0.6)

is optimal in (8.0.1) for some A and B to be found.

Given any two candidate points A and B and inserting τA,B into (8.0.1)as a candidate stopping time, we get the function

VA,B(x) = ExG(XτA,B

)(9.0.7)

for x ∈ E . Clearly VA,B(x) = G(x) for x /∈ (A, B) and only those A and Bare to be considered for which VA,B(x) ≥ G(x) for all x ∈ E .

When X is a Markov process with diffusion component then VA,B will besmooth on (A, B) but only continuous at A and B as Figure IV.2 shows.

If we move A and B along the state space and examine what happens withthe resulting function VA,B at A and B , typically we will see that only for aspecial (unique) pair of A and B , will the continuity of VA,B at A and B turn


G

A B

x | VA,B(x)

Figure IV.3: The function x → VA,B(x) from (9.0.7) above when X is aMarkov process with jumps (without diffusion component).

into smoothness. This is a variational way to experience the principle of smoothfit.

When X has jumps and no diffusion component then VA,B will be con-tinuous/smooth on (A, B) but only discountinuous at A and B as Figure IV.3shows.

If we move A and B along the state space and examine what happens withthe resulting function VA,B at A and B , typically we will see that only for aspecial (unique) pair of A and B , the discontinuity of VA,B at A and B willturn into continuity. (A mixed case of Figure IV.2 at A and Figure IV.3 at B , orvice versa, is possible as well.) This is a variational way to experience the principleof continuous fit.

Specific examples of Figure IV.2 and Figure IV.3 (including a mixed case)are studied in Chapter VI (see figures in Sections 23 and 24).

9.1. The principle of smooth fit

As already pointed out above, the principle of smooth fit (see (8.0.8)) states thatthe optimal stopping boundary (point) is selected so that the value function issmooth at that point. The aim of this subsection is to present two methods which(when properly modified if needed) can be used to verify the smooth fit principle.


For simplicity of exposition we will restrict our attention to the infinite horizonproblems in dimension one. The second method extends to higher dimensions aswell.

Given a regular diffusion process X = (Xt)t≥0 with values in E = R and ameasurable function G : E → R satisfying the usual (or weakened) integrabilitycondition, consider the optimal stopping problem

V (x) = supτ

ExG(Xτ ) (9.1.1)

where the supremum is taken over all stopping times τ of X , and X0 = xunder Px with x ∈ E . For simplicity let us assume that the continuation setC equals (b,∞) and the stopping set D equals (−∞, b] where b ∈ E is theoptimal stopping point. We want to show (under natural conditions) that V isdifferentiable at b and that V ′(b) = G′(b) (smooth fit).

Method 1. First note that for ε > 0 ,

V (b + ε) − V (b)ε

≥ G(b + ε) − G(b)ε

(9.1.2)

since V ≥ G and V (b) = G(b) . Next consider the exit time

τε = inf t ≥ 0 : Xt /∈ (b − ε, b + ε) (9.1.3)

for ε > 0 . Then

EbV(Xτε) = V (b + ε)Pb(Xτε = b + ε) + V (b − ε)Pb(Xτε = b − ε) (9.1.4)

= V (b + ε)Pb(Xτε = b + ε) + G(b − ε)Pb(Xτε = b − ε).

Moreover, since V is superharmonic (cf. Chapter I), we have

EbV(Xτε) ≤ V (b) = V (b)Pb

(Xτε = b + ε

)+ G(b)Pb

(Xτε = b − ε

). (9.1.5)

Combining (9.1.4) and (9.1.5) we get(V (b + ε) − V (b)

)Pb

(Xτε = b + ε

)(9.1.6)

≤ (G(b) − G(b − ε))Pb

(Xτε = b − ε

).

Recalling that Pb

(Xτε = b + ε

)=(S(b) − S(b − ε)

)/(S(b + ε) − S(b − ε)

)and

Pb

(Xτε = b − ε

)=(S(b + ε) − S(b)

)/(S(b + ε) − S(b − ε)

), where S is the scale

function of X , we see that (9.1.6) yields

V (b + ε) − V (b)ε

≤ G(b) − G(b − ε)ε

S(b + ε) − S(b)S(b) − S(b − ε)

(9.1.7)

−→ G′(b)S′(b)S′(b)

= G′(b)


as ε ↓ 0 whenever G and S are differentiable at b (and S′(b) is differentfrom zero). Combining (9.1.7) and (9.1.2) and letting ε ↓ 0 we see that V isdifferentiable at b and V ′(b) = G′(b) . In this way we have verified that thefollowing claim holds:

If G and S are differentiable at b , then V is differentiable at b andV ′(b) = G′(b) i.e. the smooth fit holds at b .

(9.1.8)

The following example shows that differentiability of G at b cannot be omitted in(9.1.8). For a complementary discussion of Method 1 (filling the gaps of V beingsuperharmonic and S′(b) being different from zero) see Subsection 9.3 below.

Example 9.1. Let Xt = x + Bt − t for t ≥ 0 and x ∈ R , and let G(x) = 1 forx ≥ 0 and G(x) = 0 for x < 0 . Consider the optimal stopping problem (9.1.1).Then clearly V (x) = 1 for x ≥ 0 , and the only candidate for an optimal stoppingtime when x < 0 is

τ0 = inf t ≥ 0 : Xt = 0 (9.1.9)

where inf(∅) = ∞ . Then (with G(X∞) = G(−∞) = 0 ),

V (x) = ExG(Xτ0) = Ex

(0 · I(τ0 = ∞) + 1 · I(τ0 < ∞)

)(9.1.10)

= Px(τ0 < ∞).

Using Doob’s resultP(

supt≥0

(Bt − αt) ≥ β)

= e−2αβ (9.1.11)

for α, β > 0 , it follows that

Px(τ0 < ∞) = P(

supt≥0

(x + Bt − t) ≥ 0)

(9.1.12)

= P(

supt≥0

(Bt − t) ≥ −x)

= e2x

for x < 0 . This shows that V (x) = 1 for x ≥ 0 and V (x) = e2x for x < 0 .Note that V ′(0+) = 0 = 2 = V ′(0−) . Thus the smooth fit does not hold at theoptimal stopping point 0 . Note that G is discontinuous at 0 .

Moreover, if we take any continuous function G : R → R satisfying 0 <G(x) < V (x) for x ∈ (−∞, 0) and G(x) = 1 for x ∈ [0,∞) , and consider theoptimal stopping problem (9.1.1) with G instead of G , then (due to ExG(Xτ0) ≥ExG(Xτ0) > G(x) for x ∈ (−∞, 0) ) we see that it is never optimal to stop in(−∞, 0) . Clearly it is optimal to stop in [0,∞) , so that τ0 is optimal again andwe have

V (x) = ExG(Xτ0) = ExG(Xτ0) = V (x) (9.1.13)

for all x ∈ R . Thus, in this case too, we see that the smooth fit does not hold atthe optimal stopping point 0 . Note that G is not differentiable at b = 0 .


Method 2. Assume that X equals the standard Brownian motion B [or anyother (diffusion) process where the dependence on the initial point x is explicitand smooth]. Then as above we first see that (9.1.2) holds. Next let τε∗ = τ∗(b+ε)denote the optimal stopping time for V (b + ε) , i.e. let

τε∗ = inf t ≥ 0 : Xt ≤ b (9.1.14)

under Pb+ε . Since Law(X | Pb+ε) = Law(Xb+ε | P) where Xb+εt = b + ε + Bt

under P , we see that τε∗ is equally distributed as σε

∗ = inf t ≥ 0 : b + ε + Bt ≤b = inf t ≥ 0 : Bt ≤ −ε under P , so that τε

∗ ↓ 0 and σε∗ ↓ 0 as ε ↓ 0 . This

reflects the fact that b is regular for D (relative to X ) which is needed for themethod to be applicable. We then have

V (b + ε) − V (b)ε

≤ EG(b + ε + Bσε∗) − EG(b + Bσε∗)ε

(9.1.15)

since V (b+ε) = Eb+εG(Xτε∗ ) = EG(b+ε+Bσε∗) and V (b) ≥ EbG(Xτε∗ ) = EG(b+Bσε∗) . By the mean value theorem we have

G(b + ε + Bσε∗

)− G(b + Bσε∗) = G′(b + Bσε∗ + θε)ε (9.1.16)

for some θ ∈ (0, 1) . Inserting (9.1.16) into (9.1.15) and assuming that

|G′(b + Bσε∗ + θε)| ≤ Z (9.1.17)

for all ε > 0 (small) with some Z ∈ L1(P) , we see from (9.1.15) using (9.1.16)and the Lebesgue dominated convergence theorem that

lim supε↓0

(V (b + ε) − V (b)

ε

)≤ G′(b). (9.1.18)

Since Bτε∗ = −ε note that (9.1.17) is satisfied if G′ is bounded (on a neighborhoodcontaining b ). From (9.1.2) it follows that

lim infε↓0

(V (b + ε) − V (b)

ε

)≥ G′(b). (9.1.19)

Combining (9.1.18) and (9.1.19) we see that V is differentiable at b and V ′(b) =G′(b) . In this way we have verified that the following claim holds:

If G is C1 (on a neighborhood containing b ) then V isdifferentiable at b and V ′(b)=G′(b) i.e. the smooth fit holds at b.

(9.1.20)

On closer inspection of the above proof, recalling that Bσε∗ = −ε , one seesthat (9.1.16) actually reads

G(b+ε+Bσε∗) − G(b+Bσε∗) = G(b) − G(b− ε) (9.1.21)


which is in line with (9.1.7) above since S(x) = x for all x (at least when X is astandard Brownian motion). Thus, in this case (9.1.18) holds even if the additionalhypotheses on Z and G′ stated above are removed. The proof above, however,is purposely written in this more general form, since as such it also applies tomore general (diffusion) processes X for which the (smooth) dependence on theinitial point is expressed explicitly, as well as when C is not unbounded, i.e. whenC = (b, c) for some c ∈ (b,∞) . In the latter case, for example, one can replace(9.1.14) by

τε∗ = inf t ≥ 0 : Xt ≤ b or Xt ≥ c (9.1.22)

under Pb+ε and proceed as outlined above making only minor modifications.

The following example shows that regularity of the optimal point b for D(relative to X ) cannot be omitted from the proof.

Example 9.2. Let Xt = −t for t ≥ 0 , let G(x) = x for x ≥ 0 , let G(x) = H(x)for x ∈ [−1, 0] , and let G(x) = 0 for x ≤ −1 , where H : [−1, 1] → R is asmooth function making G (continuous and) smooth (e.g. C1 ) on R . Assumemoreover that H(x) < 0 for all x ∈ (−1, 0) (with H(−1) = H(0) = 0 ). Thenclearly (−1, 0) is contained in C , and (−∞,−1] ∪ [0,∞) is contained in D . Itfollows that V (x) = 0 for x ≤ 0 and V (x) = x for x > 0 . Hence V is notsmooth at the optimal boundary point 0 . Recall that G is smooth everywhereon R (at 0 as well) but 0 is not regular for D (relative to X ).

Notes. The principle of smooth fit appears for the first time in the workof Mikhalevich [136]. Method 1 presented above was inspired by the method ofGrigelionis and Shiryaev [88] (see also [196, pp. 159–161]) which uses a Taylorexpansion of the value function at the optimal point (see Subsection 9.3 belowfor a deeper analysis). Method 2 presented above is due to Bather [11] (see also[215]). This method will be adapted and used in Chapters VI–VIII below. Forcomparison note that this method uses a Taylor expansion of the gain functionwhich is given a priori. There are also other derivations of the smooth fit thatrely upon the diffusion relation Xt ∼

√t for small t (see e.g. [30, p. 233] which

also makes use of a Taylor expansion of the value function). Further references aregiven in the Notes to Subsection 9.3 and Section 25 below.

9.2. The principle of continuous fit

As already pointed out above, the principle of continuous fit states that the optimalstopping boundary (point) is selected so that the value function is continuous atthat point. The aim of this subsection is to present a simple method which (whenproperly modified if needed) can be used to verify the continuous fit principle.For simplicity of exposition we will restrict our attention to the infinite horizonproblems in dimension one.


Given a Markov process X = (Xt)t≥0 (right-continuous and left-continuousover stopping times) taking value in E = R and a measurable function G : E → R

satisfying the usual (or weakened) integrability condition, consider the optimalstopping problem

V (x) = supτ

ExG(Xτ ) (9.2.1)

where the supremum is taken over all stopping times τ of X , and X0 = x underPx with x ∈ E . For simplicity let us assume that the continuation set C equals(b,∞) and the stopping set D equals (−∞, b] where b ∈ E is the optimalstopping point. We want to show that (under natural conditions) V is continuousat b and that V (b) = G(b) (continuous fit).

Method. First note that

V (b+ε)− V (b) ≥ G(b+ε) − G(b) (9.2.2)

for ε > 0 . Next let τε∗ denote the optimal stopping time for V (b + ε) , i.e. let

τε∗ = inf t ≥ 0 : Xt ≤ b (9.2.3)

under Pb+ε . Then we have

V (b+ε)− V (b) ≤ EG(Xb+ε

τε∗

)− EG(Xb

τε∗

)(9.2.4)

since V (b) ≥ EbG(Xτε∗

)= EG

(Xb

τε∗

). Clearly τε

∗ ↓ ρ ≥ 0 as ε ↓ 0 . (In general,this ρ can be strictly positive which means that b is not regular and in turncan imply the breakdown of the smooth fit at b .) It follows that Xb

τε∗→ Xb

ρ asε ↓ 0 by the right continuity of X . Moreover, if the following time-space (Fellermotivated) condition on X holds:

Xb+εt+h → Xb

t P -a.s. (9.2.5)

as ε ↓ 0 and h ↓ 0 , then we also have Xb+ετε∗ → Xb

ρ P -a.s. as ε ↓ 0 . Combiningthese two convergence relations, upon assuming that G is continuous and

G(Xb+ε

τε∗

)− G(Xb

τε∗

) ≤ Z (9.2.6)

for some Z ∈ L1(P) , we see from (9.2.4) that

lim supε↓0

(V (b+ε)− V (b)

) ≤ 0 (9.2.7)

by Fatou’s lemma. From (9.2.2) on the other hand it follows that

lim infε↓0

(V (b+ε)− V (b)

) ≥ 0. (9.2.8)


Combining (9.2.7) and (9.2.8) we see that V is continuous at b (and V (b) =G(b) ). In this way we have verified that the following claim holds:

If G is continuous at b and bounded (or (9.2.6) holds), andX satisfies (9.2.5), then V is continuous at b (and V (b) = G(b) ),i.e. the continuous fit holds at b .

(9.2.9)

Taking G(x) = e−x for x ≥ 0 and G(x) = −x for x < 0 , and lettingXt = −t for t ≥ 0 , we see that V (x) = 1 for x ≥ 0 and V (x) = −x for x < 0 .Thus V is not continuous at the optimal stopping point 0 . This shows that thecontinuity of G at b cannot be omitted in (9.2.9). Note that 0 is regular for D(relative to X ).

Further (more illuminating) examples of the continuous fit principle (whenX has jumps) will be given in Sections 23 and 24.

Notes. The principle of continuous fit was recognized as a key ingredient ofthe solution in [168] and [169]. The proof given above is new.

9.3. Diffusions with angles

The purpose of this subsection (following [167]) is to exhibit a complementaryanalysis of the smooth fit principle in the case of one-dimensional (regular) diffu-sions.

1. Recall that the principle of smooth fit states (see (8.0.8)) that the optimalstopping point b which separates the continuation set C from the stopping setD in the optimal stopping problem

V (x) = supτ

ExG(Xτ ) (9.3.1)

is characterized by the fact that V ′(b) exists and is equal to G′(b) . Typically, noother point b separating the candidate sets C and D will satisfy this identity,and most often V ′′(b) will either fail to exist or will not be equal to G′′(b) . Theseunique features of the smooth fit principle make it a powerful tool in solvingspecific problems of optimal stopping. The same is true in higher dimensions butin the present subsection we focus on dimension one only.

Regular diffusion processes form a natural class of Markov processes X in(9.3.1) for which the smooth-fit principle is known to hold in great generality. Onthe other hand, it is easy to construct examples which show that the smooth fitV ′(b) = G′(b) can fail if the diffusion process X is not regular as well as thatV need not be differentiable at b if G is not so (see Example 9.1 above). Thusregularity of the diffusion process X and differentiability of the gain function Gare minimal conditions under which the smooth fit can hold in greater generality.In this subsection we address the question of their sufficiency (recall (9.1.8) above).


Our exposition can be summarized as follows. Firstly, we show that thereexists a regular diffusion process X and a differentiable gain function G suchthat the smooth fit condition V ′(b) = G′(b) fails to hold at the optimal stoppingpoint b (Example 3.5). Secondly, we show that the latter cannot happen if thescale function S is differentiable at b . In other words, if X is regular and bothG and S are differentiable at b , then V is differentiable at b and V ′(b) = G′(b)(Theorem 3.3) [this was derived in (9.1.8) using similar means]. Thirdly, we givean example showing that the latter can happen even when d+G/dS < d+V/dS <d−V/dS < d−V/dS at b (Example 3.2). The relevance of this fact will be reviewedshortly below.

A. N. Kolmogorov expressed the view that the principle of smooth fit holdsbecause “diffusions do not like angles” (this is one of the famous tales of the secondauthor). It hinges that there must be something special about the diffusion processX in the first example above since the gain function G is differentiable. We willbriefly return to this point in the end of the present subsection.

2. Let X = (Xt)t≥0 be a diffusion process with values in an interval J ofR . For simplicity we will assume that X can be killed only at the end-pointsof J which do not belong to J . Thus, if ζ denotes the death time of X , thenX is a strong Markov process such that t → Xt is continuous on [0, ζ) , andthe end-points of J at which X can be killed act as absorbing boundaries (oncesuch a point is reached X stays there forever). We will denote by I = (l, r) theinterior of J .

Given c ∈ J we will let

τc = inf t > 0 : Xt = c (9.3.2)

denote the hitting time of X to c . We will assume that X is regular in thesense that Pb(τc < ∞) = 1 for every b ∈ I and all c ∈ J . It means that Icannot be decomposed into smaller intervals from which X could not exit. It alsomeans that b is regular for both D1 = (l, b] and D2 = [b, r) in the sense thatPb(τDi = 0) = 1 where τDi = inf t > 0 : Xt ∈ Di for i = 1, 2 . In particular,each b ∈ I is regular for itself in the sense that Pb(τb = 0) = 1 .

Let S denote the scale function of X . Recall that S : J → R is a strictlyincreasing continuous function such that

Px(τa <τb) =S(b)−S(x)S(b)−S(a)

& Px(τb <τa) =S(x)−S(a)S(b)−S(a)

(9.3.3)

for a < x < b in J . Recall also that the scale function can be characterized(up to an affine transformation) as a continuous function S : J → R such that(S(Xt∧τl∧τr))t≥0 is a continuous local martingale.

3. Let G : J → R be a measurable function satisfying

E sup0≤t<ζ

|G(Xt)| < ∞. (9.3.4)


Consider the optimal stopping problem

V (x) = supτ

ExG(Xτ ) (9.3.5)

for x ∈ J where the supremum is taken over all stopping times τ of X (i.e. withrespect to the natural filtration FX

t = σ(Xs : 0 ≤ s ≤ t) generated by X fort ≥ 0 ).

Following the argument of Dynkin and Yushkevich [55, p. 115], take c < x <d in J and choose stopping times τ1 and τ2 such that EcG(Xτ1) ≥ V (c) − εand EdG(Xτ2) ≥ V (d) − ε where ε is given and fixed. Consider the stoppingtime τε = (τc + τ1 θτc) I(τc <τd) + (τd + τ2 θτd

) I(τd <τc) obtained by applyingτ1 after hitting c (before d ) and τ2 after hitting d (before c ). By the strongMarkov property of X it follows that

V (x) ≥ ExG(Xτε) (9.3.6)

= ExG(Xτc+τ1θτc) I(τc <τd) + ExG(Xτd+τ2θτd

) I(τd <τc)

= ExG(Xτ1) θτc I(τc <τd) + ExG(Xτ2) θτdI(τd <τc)

= Ex EXτc(G(Xτ1)) I(τc <τd) + Ex EXτd

(G(Xτ2)) I(τd <τc)

= Ec(G(Xτ1)) Px(τc <τd) + Ed(G(Xτ2)) Px(τd <τc)

≥ (V (c)−ε)S(d)−S(x)S(d)−S(c)

+ (V (d)−ε)S(x)−S(c)S(d)−S(c)

= V (c)S(d)−S(x)S(d)−S(c)

+ V (d)S(x)−S(c)S(d)−S(c)

− ε

where the first inequality follows by definition of V and the second inequalityfollows by the choice of τ1 and τ2 . Letting ε ↓ 0 in (9.3.6) one concludes that

V (x) ≥ V (c)S(d)−S(x)S(d)−S(c)

+ V (d)S(x)−S(c)S(d)−S(c)

(9.3.7)

for c < x < d in J . This means that V is S-concave (see e.g. [174, p. 546]). Itis not difficult to verify that V is superharmonic if and only if V is S-concave(recall (2.2.8) above).

4. In exactly the same way as for concave functions (corresponding to S(x) =x above) one then sees that (9.3.7) implies that

y → V (y)−V (x)S(y)− S(x)

is decreasing (9.3.8)

on J for every x ∈ I . It follows that

−∞ <d+V

dS(x) ≤ d−V

dS(x) < +∞ (9.3.9)


for every x ∈ I . It also follows that V is continuous on I since S is continuous.Note that continuity of V is obtained for measurable G generally. This links(9.3.6)–(9.3.7) with a powerful (geometric) argument applicable in one dimensiononly (compare it with the multidimensional results of Subsection 2.2).

5. Let us now assume that b ∈ I is an optimal stopping point in the problem(9.3.5). Then V (b) = G(b) and hence by (9.3.8) we get

G(b+ε)−G(b)S(b+ε)−S(b)

≤ V (b+ε)−V (b)S(b+ε)−S(b)

≤ V (b−δ)−V (b)S(b−δ)−S(b)

≤ G(b−δ)−G(b)S(b−δ)−S(b)

(9.3.10)

for ε > 0 and δ > 0 where the first inequality follows since G(b+ε) ≤ V (b+ε)and the third inequality follows since −V (b−δ) ≤ −G(b−δ) (recalling also that Sis strictly increasing). Passing to the limit for ε ↓ 0 and δ ↓ 0 this immediatelyleads to

d+G

dS(b) ≤ d+V

dS(b) ≤ d−V

dS(b) ≤ d−G

dS(b) (9.3.11)

whenever d+G/dS and d−G/dS exist at b . In this way we have reached theessential part of Salminen’s result [180, p. 96]:

Theorem 9.3. (Smooth fit through scale) If dG/dS exists at b , then dV/dSexists at b and

dV

dS(b) =

dG

dS(b) (9.3.12)

whenever V (b) = G(b) for b ∈ I .

In particular, if X is on natural scale (i.e. S(x) = x ) then the smooth fitcondition

dV

dx(b) =

dG

dx(b) (9.3.13)

holds at the optimal stopping point b as soon as G is differentiable at b .

The following example shows that equalities in (9.3.11) and (9.3.12) may failto hold even though the smooth fit condition (9.3.13) holds.

Example 9.4. Let Xt = F (Bt) where

F (x) =

x1/3 if x ∈ [0, 1],

−|x|1/3 if x ∈ [−1, 0)(9.3.14)

and B is a standard Brownian motion in (−1, 1) absorbed (killed) at either −1or 1 . Since F is a strictly increasing and continuous function from [−1, 1] onto[−1, 1] , it follows that X is a regular diffusion process in (−1, 1) absorbed (killed)at either −1 or 1 .

Consider the optimal stopping problem (9.3.5) with

G(x) = 1 − x2 (9.3.15)


for x ∈ (−1, 1) . Set Xxt = F (x+Bt) for x ∈ (−1, 1) and let B be defined on

(Ω,F , P) so that B0 = 0 under P . Since F is increasing (and continuous) it canbe verified that

Law(Xx|P) = Law(C |PF (x)) (9.3.16)

where Ct(ω) = ω(t) is the coordinate process (on a canonical space) that isMarkov under the family of probability measures Pc for c ∈ (−1, 1) with Pc(C0 =c) = 1 (note that each c ∈ (−1, 1) corresponds to F (x) for some x ∈ (−1, 1)given and fixed).

In view of (9.3.16) let us consider the auxiliary optimal stopping problem

V (x) = supτ

EG(x+Bτ ) (9.3.17)

where G = GF and the supremum is taken over all stopping times τ of B (upto the time of absorption at −1 or 1 ). Note that

G(x) = 1 − |x|2/3 (9.3.18)

for x ∈ (−1, 1) . Since V is the smallest superharmonic (i.e. concave) functionthat dominates G (cf. Chapter I), and clearly V (−1) = V (1) = 0 , it follows that

V (x) =

1 − x if x ∈ [0, 1],1 + x if x ∈ [−1, 0).

(9.3.19)

From (9.3.16) we see that V (x) = V (F−1(x)) and since F−1(x) = x3 , it followsthat

V (x) =

1 − x3 if x ∈ [0, 1],1 + x3 if x ∈ [−1, 0).

(9.3.20)

Comparing (9.3.20) with (9.3.15) we see that b = 0 is an optimal stopping point.Moreover, it is evident that the smooth fit (9.3.13) holds at b = 0 , both derivativesbeing zero. However, noting that the scale function of X equals S(x) = x3 for x ∈[−1, 1] (since S(X) = F−1(F (B)) = B is a martingale), it is straightforwardlyverified from (9.3.15) and (9.3.20) that

d+G

dS= −∞ <

d+V

dS= −1 <

d−V

dS= 1 <

d−G

dS= +∞ (9.3.21)

at the optimal stopping point b = 0 .

6. Note that the scale function S in the preceding example is differentiableat the optimal stopping point b but that S′(b) = 0 . This motivates the followingextension of Theorem 9.3 above (recall (9.1.8) above).


Theorem 9.5. (Smooth fit) If both dG/dx and dS/dx exist at b , then dV/dxexists at b and

dV

dx(b) =

dG

dx(b) (9.3.22)

whenever V (b) = G(b) for b ∈ I .

Proof. Assume first that S′(b) = 0 . Multiplying by (S(b+ε)−S(b))/ε in (9.3.10)we get

G(b+ε)−G(b)ε

≤ V (b+ε)−V (b)ε

(9.3.23)

≤ S(b+ε)−S(b)ε

(G(b−δ)−G(b))/(−δ)(S(b−δ)−S(b))/(−δ)

.

Passing to the limit for ε ↓ 0 and δ ↓ 0 , and using that S′(b) = 0 , it follows thatd+V/dx = dG/dx at b . (Note that one could take ε = δ in this argument.)

Similarly, multiplying by (S(b−δ)−S(b))/(−δ) in (9.3.10) we get

(G(b+ε)−G(b))/ε

(S(b+ε)−S(b))/ε

S(b−δ)−S(b)−δ

≤ V (b−δ)−V (b)−δ

≤ G(b−δ)−G(b)−δ

. (9.3.24)

Passing to the limit for ε ↓ 0 and δ ↓ 0 , and using that S′(b) = 0 , it followsthat d−V/dx = dG/dx at b . (Note that one could take ε = δ in this argument.)Combining the two conclusions we see that dV/dx exists at b and (9.3.22) holdsas claimed.

To treat the case S′(b) = 0 we need the following simple facts of real analysis.

Lemma 9.6. Let f : R+ → R and g : R+ → R be two continuous functionssatisfying:

f(0) = 0 and f(ε) > 0 for ε > 0 ; (9.3.25)g(0) = 0 and g(δ) > 0 for δ > 0 . (9.3.26)

Then for every εn ↓ 0 as n → ∞ there are εnk↓ 0 and δk ↓ 0 as k → ∞ such

that f(εnk) = g(δk) for all k ≥ 1 . In particular, it follows that

limk→∞

f(εnk)

g(δk)= 1 . (9.3.27)

Proof. Take any εn ↓ 0 as n → ∞ . Since f(εn) → 0 and f(εn) > 0 we can finda subsequence εnk

↓ 0 such that xnk:= f(εnk

) ↓ 0 as k → ∞ . Since g(1) > 0there is no restriction to assume that xn1 < g(1) . But then by continuity of g andthe fact that xn1 ∈ (g(0), g(1)) there must be δ1 ∈ (0, 1) such that g(δ1) = xn1 .Since xn2 < xn1 it follows that xn2 ∈ (g(0), g(δ1)) and again by continuityof g there must be δ2 ∈ (0, δ1) such that g(δ2) = xn2 . Continuing likewise


by induction we obtain a decreasing sequence δk ∈ (0, 1) such that g(δk) =xnk

for k ≥ 1 . Denoting δ = lim k→∞ δk we see that g(δ) = lim k→∞ g(δk) =lim k→∞ xnk

= 0 . Hence δ must be 0 by (9.3.26). This completes the proof ofLemma 9.6.

Let us continue the proof of Theorem 9.5 in the case when S′(b) = 0 .Take εn ↓ 0 and by Lemma 9.6 choose δk ↓ 0 such that (9.3.27) holds withf(ε) = (S(b+ε)−S(b))/ε and g(δ) = (S(b)−S(b−δ))/δ . Then (9.3.23) reads

G(b+εnk)−G(b)

εnk

≤ V (b+εnk)−V (b)

εnk

≤ f(εnk)

g(δk)G(b−δk)−G(b)

−δk(9.3.28)

for all k ≥ 1 . Letting k → ∞ and using (9.3.27) we see that (V (b + εnk)−

V (b))/εnk→ G′(b) . Since this is true for any εn ↓ 0 it follows that d+V/dx

exists and is equal to dG/dx at b .Similarly, take εn ↓ 0 and by Lemma 9.6 choose δk ↓ 0 such that (9.3.27)

holds with f(ε) = (S(b)−S(b−ε))/ε and g(δ) = (S(b+δ)−S(b))/δ . Then (9.3.24)(with ε and δ traded) reads

G(b+δk)−G(b)δk

f(εnk)

g(δk)≤ V (b−εnk

)−V (b)−εnk

≤ G(b−εnk)−G(b)

−εnk

(9.3.29)

for all k ≥ 1 . Letting k → ∞ and using (9.3.27) we see that (V (b − εnk)−

V (b))/(−εnk) → G′(b) . Since this is true for any εn ↓ 0 it follows that d−V/dx

exists and is equal to dG/dx at b . Taken together with the previous conclusionon d+V/dx this establishes (9.3.22) and the proof of Theorem 9.5 is complete.

7. The question arising naturally from the previous considerations is whetherdifferentiability of the gain function G and regularity of the diffusion process Ximply the smooth fit V ′(b) = G′(b) at the optimal stopping point b .

The negative answer to this question is provided by the following example.

Example 9.7. Let Xt = F (Bt) where

F (x) =

√x if x ∈ [0, 1],

−x2 if x ∈ [−1, 0)(9.3.30)

and B is a standard Brownian motion in (−1, 1) absorbed (killed) at either −1or 1 . Since F is a strictly increasing and continuous function from [−1, 1] onto[−1, 1] , it follows that X is a regular diffusion process in (−1, 1) absorbed (killed)at either −1 or 1 .

Consider the optimal stopping problem (9.3.5) with

G(x) = 1 − x (9.3.31)

for x ∈ (−1, 1) . Set Xxt = F (x+Bt) for x ∈ (−1, 1) and let B be defined on

(Ω,F , P) so that B0 = 0 under P . Since F is increasing (and continuous) it


follows thatLaw(Xx|P) = Law(C |PF (x)) (9.3.32)

where Ct(ω) = ω(t) is the coordinate process (on a canonical space) that isMarkov under the family of probability measures Pc for c ∈ (−1, 1) with Pc(C0 =c) = 1 (note that each c ∈ (−1, 1) corresponds to F (x) for some x ∈ (−1, 1)given and fixed).

In view of (9.3.32) let us consider the auxiliary optimal stopping problem

V (x) = supτ

EG(x+Bτ ) (9.3.33)

where G = GF and the supremum is taken over all stopping times τ of B (upto the time of absorption at −1 or 1 ). Note that

G(x) =

1 −√

x if x ∈ [0, 1],1 + x2 if x ∈ [−1, 0) .

(9.3.34)

Since V is the smallest superharmonic (i.e. concave) function that dominates G(cf. Chapter I), and clearly V (−1) = 2 and V (1) = 0 , it follows that

V (x) = 1 − x (9.3.35)

for x ∈ [−1, 1] . From (9.3.32) we see that V (x) = V (F−1(x)) and since

F−1(x) =

x2 if x ∈ [0, 1] ,

−√|x| if x ∈ [−1, 0) ,(9.3.36)

it follows that

V (x) =

1 − x2 if x ∈ [0, 1] ,

1 +√|x| if x ∈ [−1, 0) .

(9.3.37)

Comparing (9.3.37) with (9.3.31) we see that b = 0 is an optimal stopping point.However, it is evident that the smooth fit V ′(b) = G′(b) fails at b = 0 (see FigureIV.4).

8. Note that the scale function S of X equals F−1 in (9.3.36) above (sinceS(X) = F−1(F (B)) = B is a martingale) so that S′

+(0) = 0 and S′−(0) = +∞ .Note also from (9.3.30) above that X receives a ”strong” push toward (0, 1] anda ”mild” push toward [−1, 0) when at 0 . The two extreme cases of S′

+(0) andS′−(0) are not the only possible ones to ruin the smooth fit. Indeed, if we slightlymodify F in (9.3.30) above by setting

F (x) =

√x if x ∈ [0, 1],

x if x ∈ [−1, 0),(9.3.38)


-1 1

1

2

G(x)x

V(x)x

x

Figure IV.4: The gain function G and the value function V from Example9.7. The smooth fit V ′(b) = G′(b) fails at the optimal stopping pointb = 0 .

then the same analysis as above shows that

V (x) =

1 − x2 if x ∈ [0, 1],1 − x if x ∈ [−1, 0),

(9.3.39)

so that the smooth fit V ′(b) = G′(b) still fails at the optimal stopping pointb = 0 . In this case the scale function S of X equals

F−1(x) =

x2 if x ∈ [0, 1],x if x ∈ [−1, 0),

(9.3.40)

so that S′+(0) = 0 and S′−(0) = 1 .

Moreover, any further speculation that the extreme condition S′+(0) = 0 is

needed to ruin the smooth fit is ruled out by the following modification of F in(9.3.30) above:

F (x) =

−1+

√1+8x

2 if x ∈ [0, 1],x if x ∈ [−1, 0) .

(9.3.41)

Then the same analysis as above shows that

V (x) =

1 − x2+x

2 if x ∈ [0, 1],1 − x if x ∈ [−1, 0),

(9.3.42)


so that the smooth fit V ′(b) = G′(b) still fails at the optimal stopping pointb = 0 . In this case the scale function S of X equals

F−1(x) =

x2+x

2 if x ∈ [0, 1],x if x ∈ [−1, 0),

(9.3.43)

so that S′+(0) = 1/2 and S′

−(0) = 1 .

9. In order to examine what is ”angular” about the diffusion from the pre-ceding example, let us recall that (9.3.3) implies that

Pb(τb−ε <τb+ε) =S(b+ε)−S(b)

S(b+ε)−S(b−ε)(9.3.44)

=(S(b+ε)−S(b))/ε

(S(b+ε)−S(b))/ε + (S(b)−S(b−ε))/ε−→ R

R + L

as ε ↓ 0 whenever S′+(b) =: R and S′−(b) =: L exist (and are assumed to be

different from zero for simplicity). Likewise, one finds that

Pb(τb+ε <τb−ε) =S(b)−S(b−ε)

S(b+ε)−S(b−ε)(9.3.45)

=(S(b)−S(b−ε))/ε

(S(b+ε)−S(b))/ε + (S(b)−S(b−ε))/ε−→ L

R + L

as ε ↓ 0 whenever S′−(b) =: L and S′

+(b) =: R exist (and are assumed to bedifferent from zero for simplicity).

If S is differentiable at b then R = L so that the limit probabilities in(9.3.44) and (9.3.45) are equal to 1/2 . Note that these probabilities correspondto X exiting b infinitesimally to either left or right respectively. On the otherhand, if S is not differentiable at b , then the two limit probabilities R/(R+L)and L/(R+L) are different and this fact alone may ruin the smooth fit at bas Example 9.7 above shows. Thus, regularity of X itself is insufficient for thesmooth fit to hold generally, and X requires this sort of “tuned regularity” instead(recall Theorem 9.5 above).

10. Another way of looking at such diffusions is obtained by means of stochas-tic calculus. The Ito–Tanaka–Meyer formula (page 67) implies that the processXt = F (Bt) solves the integral equation

Xt = X0 +∫ t

0

F ′ F−1(Xs) I(Xs = 0) dBs (9.3.46)

+∫ t

0

12

F ′′ F−1(Xs) I(Xs = 0) ds +12[F ′

+(0) − F ′−(0)

]0t (B)

where 0t (B) is the local time of B at 0 . Setting

At =[F ′

+(0) − F ′−(0)

]0t (B) (9.3.47)

Section 10. The method of time change 165

we see that (9.3.46) reads

dXt = ρ(Xt) dt + σ(Xt) dBt + dAt (9.3.48)

where (At)t≥0 is continuous, increasing (or decreasing), adapted to (Ft)t≥0 andsatisfies ∫ t

0

I(Xs = 0) dAs = 0 (9.3.49)

with A0 = 0 . These conditions usually bear the name of an SDE with reflectionfor (9.3.48). Note however that X is not necessarily non-negative as additionallyrequired from solutions of SDEs with reflection.

Notes. A number of authors have contributed to understanding of the smooth-fit principle by various means. With no aim to review the full history of thesedevelopments, and in addition to the Notes to Subsection 9.1 above, we refer to[180], [145], [23], [146], [3], [37] and [2] (for Levy processes). Further references aregiven in the Notes to Section 25 below.

10. The method of time change

The main goal of this section (following [155]) is to present a deterministic time-change method which enables one to solve some nonlinear optimal stopping prob-lems explicitly. The basic idea is to transform the original (difficult) problem intoa new (easier) problem. The method is firstly described (Subsection 10.1) and thenillustrated through several examples (Subsection 10.2).

10.1. Description of the method

1. To explain the ideas in more detail, let ((Xt)t≥0, Px) be a one-dimensionaltime-homogeneous diffusion associated with the infinitesimal generator

LX = b(x)∂

∂x+ a2(x)

12

∂2

∂x2(10.1.1)

where x → a(x) > 0 and x → b(x) are continuous. Assume moreover that thereexists a standard Brownian motion B = (Bt)t≥0 such that X = (Xt)t≥0 solvesthe stochastic differential equation

dXt = b(Xt) dt + a(Xt) dBt (10.1.2)

with X0 = x under Px . The typical optimal stopping problem which appearsunder consideration below has the value function given by

V∗(t, x) = supτ

Ex

(α(t + τ)Xτ

)(10.1.3)


where the supremum is taken over a class of stopping times τ for X and α is asmooth but nonlinear function. This forces us to take (t, Xt)t≥0 as the underlyingdiffusion in the problem, and thus by general optimal stopping theory (Chapter III)we know that the value function V∗ should solve the following partial differentialequation:

∂V

∂t(t, x) + LXV (t, x) = 0 (10.1.4)

in the domain of continued observation. However, it is generally difficult to finda closed-form solution of the partial differential equation, and the basic idea ofthe time-change method is to transform the original problem into a new optimalstopping problem such that the new value function solves an ordinary differentialequation.

2. To do so one is naturally led to find a deterministic time change t → σt

satisfying the following two conditions:

(i) t → σt is continuous and strictly increasing;

(ii) there exists a one-dimensional time-homogeneous diffusion Z = (Zt)t≥0 withinfinitesimal generator LZ such that α(σt)Xσt = e−rt Zt for some r ∈ R .

From general theory (Chapter III) we know that the new (time-changed) valuefunction

W∗(z) = supτ

Ez

(e−rτZτ

), (10.1.5)

where the supremum is taken over a class of stopping times τ for Z , should solvethe ordinary differential equation

LZW∗(z) = r W∗(z) (10.1.6)

in the domain of continued observation. Note that under condition (i) there is aone-to-one correspondence between the original problem and the new problem, i.e.if τ is a stopping time for Z then στ is a stopping time for X and vice versa.

3. Given the diffusion X = (Xt)t≥0 the crucial point is to find the processZ = (Zt)t≥0 and the time change σt fulfilling conditions (i) and (ii) above. Ito’sformula (page 67) offers an answer to these questions.

Setting Y = (Yt)t≥0 = (β(t)Xt)t≥0 where β = 0 is a smooth function, byIto’s formula we get

Yt = Y0 +∫ t

0

(β′(u)β(u)

Yu + β(u) b

(Yu

β(u)

))du +

∫ t

0

β(u) a

(Yu

β(u)

)dBu (10.1.7)

and hence Y = (Yt)t≥0 has the infinitesimal generator

LY =(

β′(t)β(t)

y + β(t) b

(y

β(t)

))∂

∂y+ β2(t) a2

(y

β(t)

)12

∂2

∂y2. (10.1.8)


The time-changed process Z = (Zt)t≥0 = (Yσt)t≥0 has the infinitesimal generator(see [178, p. 175] and recall Subsection 5.1 above)

LZ =1

ρ(t)LY (10.1.9)

where σt is the time change given by

σt = inf

r > 0 :∫ r

0

ρ(u) du > t

(10.1.10)

for some u → ρ(u) > 0 (to be found) such that σt → ∞ as t → ∞ .The process Z = (Zt)t≥0 and the time change σt will be fulfilling conditions

(i) and (ii) above if the infinitesimal generator LZ does not depend on t . In viewof (10.1.8) this clearly imposes the following conditions on β (and α above)which make the method applicable:

b

(y

β(t)

)= γ(t)G1(y), (10.1.11)

a2

(y

β(t)

)=

γ(t)β(t)

G2(y) (10.1.12)

where γ = γ(t) , G1 = G1(y) and G2 = G2(y) are functions required to exist.

4. In our examples below the diffusion X = (Xt)t≥0 is given as Brownianmotion (Bt+x)t≥0 started at x under Px , and thus its infinitesimal generatoris given by

LX =12

∂2

∂x2. (10.1.13)

By the foregoing observations we shall find a time change σt and a process Z =(Zt)t≥0 satisfying conditions (i) and (ii) above. With the notation introducedabove we see from (10.1.8) that the infinitesimal generator of Y = (Yt)t≥0 in thiscase is given by

LY =β′(t)β(t)

y∂

∂y+ β2(t)

12

∂2

∂y2. (10.1.14)

Observe that conditions (10.1.11) and (10.1.12) are easily realized with γ(t)=β(t),G1 = 0 and G2 = 1 . Thus if β solves the differential equation β′(t)/β(t) =−β2(t)/2 , and we set ρ = β2/2 , then from (10.1.9) we see that LZ does notdepend on t . Noting that β(t) = 1/

√1+t solves this equation, and putting

ρ(t) = 1/2(1+t) , we find that

σt = inf

r > 0 :∫ r

0

ρ(u) du > t

= e2t − 1 . (10.1.15)

Thus the time-changed process Z = (Zt)t≥0 has the infinitesimal generator givenby

LZ = −z∂

∂z+

∂2

∂z2(10.1.16)


and hence Z = (Zt)t≥0 is an Ornstein–Uhlenbeck process. While this fact is wellknown, the technique described may be applied in a similar context involving otherdiffusions (see Example 10.15 below).

5. In the next subsection we shall apply the time-change method describedabove and present solutions to several optimal stopping problems. Apart fromthe time-change arguments just described, the method of proof makes also useof Brownian scaling and the principle of smooth fit in a free-boundary problem.Once the guess is performed, Ito’s calculus is used as a verification tool. The mainemphasis of the section is on the method of proof and its unifying scope.

10.2. Problems and solutions

1. In this subsection we explicitly solve some nonlinear optimal stopping problemsfor a Brownian motion by applying the time-change method described in theprevious subsection (recall also Subsection 5.1 above).

Throughout B = (Bt)t≥0 denotes a standard Brownian motion started atzero under P , and the diffusion X = (Xt)t≥0 is given as the Brownian motion(Bt+x)t≥0 started at x under Px .

Given the time change σt = e2t − 1 from (10.1.15), we know that the time-changed process

Zt = Xσt/√

1 + σt, t ≥ 0, (10.2.1)

is an Ornstein–Uhlenbeck process satisfying

dZt = −Zt dt +√

2 dBt, (10.2.2)

LZ = −z∂

∂z+

∂2

∂z2. (10.2.3)

With this notation we may now enter into the first example.

Example 10.1. Consider the optimal stopping problem with the value function

V∗(t, x) = supτ

Ex

(|Xτ | − c

√t + τ

)(10.2.4)

where the supremum is taken over all stopping times τ for X satisfying Ex√

τ <∞ and c > 0 is given and fixed. We shall solve this problem in five steps ( 1– 5 ).

1. In the first step we shall apply Brownian scaling and note that τ = τ/tis a stopping time for the Brownian motion s → t−1/2Bts . If we now rewrite(10.2.4) as

V∗(t, x) = supτ

E(|Bτ + x| − c

√t + τ

)(10.2.5)

=√

t supτ/t

E(∣∣t−1/2Bt(τ/t) + x/

√t∣∣− c

√1 + τ/t

)


we clearly see thatV∗(t, x) =

√t V∗(1, x/

√t) (10.2.6)

and therefore we only need to look at V∗(1, x) in the sequel. By using (10.2.6) wecan also make the following observation on the optimal stopping boundary for theproblem (10.2.4).

Remark 10.2. In the problem (10.2.4) the gain function equals g(t, x) = |x| −c√

t and the diffusion is identified with(t+r, Xr

). If a point (t0, x0) belongs

to the boundary of the domain of continued observation, i.e. (t0, x0) is an in-stantaneously stopping point ( τ ≡ 0 is an optimal stopping time), then weget from (10.2.6) that V∗(t0, x0) = |x0| − c

√t0 =

√t0 V∗(1, x0/

√t0) . Hence

V∗(1, x0/√

t0) = |x0|/√

t0 − c and therefore the point (1, x0/√

t0) is also in-stantaneously stopping. Set now γ0 = |x0|/

√t0 and note that if (t, x) is any

point satisfying |x|/√t = γ0, then this point is also instantaneously stopping.This offers a heuristic argument that the optimal stopping boundary should be|x| = γ0

√t for some γ0 > 0 to be found.

2. In the second step we shall apply the time change t → σt from (10.1.15)to the problem V∗(1, x) and transform it into a new problem. From (10.2.1) weget

|Xστ | − c√

1 + στ =√

1 + στ

(|Zτ | − c)

= eτ(|Zτ | − c

)(10.2.7)

and the problem to determine V∗(1, x) therefore reduces to computing

V∗(1, x) = W∗(x) (10.2.8)

where W∗ is the value function of the new (time-changed) optimal stopping prob-lem

W∗(z) = supτ

Ez(eτ(|Zτ | − c )

)(10.2.9)

the supremum being taken over all stopping times τ for Z for which Ezeτ < ∞ .

Observe that this problem is one-dimensional (see Subsection 6.2 above).

3. In the third step we shall show how to solve the problem (10.2.9). Fromgeneral optimal stopping theory (Chapter I) we know that the following stoppingtime should be optimal:

τ∗ = inf

t > 0 : |Zt| ≥ z∗

(10.2.10)

where z∗ ≥ 0 is the optimal stopping point to be found. Observe that this guessagrees with Remark 10.2. Note that the domain of continued observation C =(−z∗, z∗) is assumed symmetric around zero since the Ornstein–Uhlenbeck processis symmetric, i.e. the process −Z = (−Zt)t≥0 is also an Ornstein–Uhlenbeckprocess started at −z . By using the same argument we may also argue that thevalue function W∗ should be even.


To compute the value function W∗ for z ∈ (−z∗, z∗) and to determine theoptimal stopping point z∗ , in view of (10.2.9)–(10.2.10) it is natural (Chapter III)to formulate the following system:

LZW (z) = −W (z) for z ∈ (−z∗, z∗), (10.2.11)W (±z∗) = z∗ − c (instantaneous stopping), (10.2.12)W ′(±z∗) = ±1 (smooth fit) (10.2.13)

with LZ in (10.2.3). The system (10.2.11)–(10.2.13) forms a free-boundary prob-lem. The condition (10.2.13) is imposed since we expect that the principle ofsmooth fit should hold.

It is known (see pages 192–193 below) that the equation (10.2.11) admitsthe even solution (10.2.155) and the odd solution (10.2.156) as two linearly inde-pendent solutions. Since the value function should be even, we can forget the oddsolution and from (10.2.155) we see that

W (z) = −AM(− 12 , 1

2 , z2

2 ) (10.2.14)

for some A > 0 to be found.From Figure IV.5 we clearly see that only for c ≥ z∗1 can the two boundary

conditions (10.2.12)–(10.2.13) be fulfilled, where z∗1 is the unique positive root ofM(−1/2 , 1/2 , z2/2) = 0 . Thus by (10.2.12)–(10.2.13) and (10.2.157) when c ≥ z∗1we find that A = z−1

∗ /M(1/2 , 3/2 , z2∗/2) and that z∗ ≤ z∗1 is the unique positive

root of the equation

z−1M(− 12 , 1

2 , z2

2 ) = (c − z)M(12 , 3

2 , z2

2 ) . (10.2.15)

Note that for c < z∗1 the equation (10.2.15) has no solution.In this way we have obtained the following candidate for the value function

W∗ in the problem (10.2.9) when c ≥ z∗1 :

W (z) =

−z−1∗ M(− 1

2 , 12 , z2

2 )/M(12 , 3

2 ,z2∗2 ) if |z| < z∗,

|z| − c if |z| ≥ z∗(10.2.16)

and the following candidate for the optimal stopping time τ∗ when c > z∗1 :

τz∗ = inf t > 0 : |Zt| ≥ z∗ . (10.2.17)

In the proof below we shall see that Ez(eτz∗ ) < ∞ when c > z∗1 (and thusz∗ < z∗1 ). For c = z∗1 (and thus z∗ = z∗1 ) the stopping time τz∗ fails to satisfyEz(eτz∗ ) < ∞ , but clearly τz∗ are approximately optimal if we let c ↓ z∗1 (andhence z∗ ↑ z∗1 ) . For c < z∗1 we have W (z) = ∞ and it is never optimal to stop.

4. To verify that these formulae are correct (with c > z∗1 given and fixed)we shall apply Ito’s formula (page 67) to the process (et W (Zt))t≥0 . For this, note


Figure IV.5: A computer drawing of the solution of the free-boundary problem (10.2.11)–(10.2.13). The solution equals z →−A M(−1/2, 1/2, z2/2) for |z| < z∗ and z → |z| − c for |z| ≥ z∗ .The constant A is chosen (and z∗ is obtained) such that the smooth fitholds at ±z∗ (the first derivative of the solution is continuous at ±z∗ ).

that z → W (z) is C2 everywhere but at ±z∗ . However, since Lebesgue measureof those u for which Zu = ±z∗ is zero, the values W ′′(±z∗) can be chosen inthe sequel arbitrarily. In this way by (10.2.2) we obtain

et W (Zt) = W (z) +∫ t

0

eu(

LZW (Zu) + W (Zu))du + Mt (10.2.18)

where M = (Mt)t≥0 is a continuous local martingale given by

Mt =√

2∫ t

0

eu W ′(Zu) dBu . (10.2.19)

Using that LZW (z) + W (z) ≤ 0 for z = ±z∗ , hence we get

e−t W (Zt) ≤ W (z) + Mt (10.2.20)

for all t . Let τ be any stopping time for Z satisfying Ezeτ < ∞ . Choose a

localization sequence (σn) of bounded stopping times for M . Clearly W (z) ≥|z| − c for all z , and hence from (10.2.20) we find

Ez

(eτ∧σn(|Zτ∧σn | − c)

) ≤ Ez

(eτ∧σn W (Zτ∧σn)

)(10.2.21)

≤ W (z) + EzMτ∧σn = W (z)


for all n ≥ 1 . Letting n → ∞ and using Fatou’s lemma, and then taking supre-mum over all stopping times τ satisfying Eze

τ < ∞ , we obtain

W∗(z) ≤ W (z) . (10.2.22)

Finally, to prove that equality in (10.2.22) is attained, and that the stoppingtime (10.2.17) is optimal, it is enough to verify that

W (z) = Ez

(eτz∗

(|Zτz∗ | − c))

= (z∗− c) Ezeτz∗ . (10.2.23)

However, from general Markov process theory (see Chapter III) we know thatw(z) = Eze

τz∗ solves (10.2.11), and clearly it satisfies w(±z∗) = 1 . Thus (10.2.23)follows immediately from (10.2.16) and definition of z∗ (see also Remark 10.7below).

5. In this way we have established that the formulae (10.2.16) and (10.2.17)are correct. Recalling by (10.2.6) and (10.2.8) that

V∗(t, x) =√

t W∗(x/√

t) (10.2.24)

we have therefore proved the following result.

Theorem 10.3. Let z∗1 denote the unique positive root of M(−1/2 , 1/2 , z2/2) = 0 .The value function of the optimal stopping problem (10.2.4) for c ≥ z∗1 is givenby

V∗(t, x) =

−√

t z−1∗ M(− 12 , 1

2 , x2

2t )/M(12 , 3

2 ,z2∗2 ) if |x|/√t < z∗,

|x| − c√

t if |x|/√t ≥ z∗(10.2.25)

where z∗ is the unique positive root of the equation

z−1M(− 12 , 1

2 , z2

2 ) = (c − z)M(12 , 3

2 , z2

2 ) (10.2.26)

satisfying z∗ ≤ z∗1 . The optimal stopping time in (10.2.4) for c > z∗1 is given by(see Figure IV.6)

τ∗ = inf r > 0 : |Xr| ≥ z∗√

t + r . (10.2.27)

For c = z∗1 the stopping times τ∗ are approximately optimal if we let c ↓ z∗1 . Forc < z∗1 we have V∗(t, x) = ∞ and it is never optimal to stop.

Using√

t + τ ≤ √t +

√τ in (10.2.4) it is easily verified that V∗(t, 0) →

V∗(0, 0) as t ↓ 0 . Hence we see that V∗(0, 0) = 0 with τ∗ ≡ 0 . Note also thatV∗(0, x) = |x| with τ∗ ≡ 0 .

2. Let τ be any stopping time for B satisfying E√

τ < ∞ . Then fromTheorem 10.3 we see that E |Xτ | ≤ c E

√t + τ + V∗(t, 0) for all c > z∗1 . Letting

first t ↓ 0 , and then c ↓ z∗1 , we obtain the following sharp inequality which wasfirst derived by Davis [34].


Figure IV.6: A computer simulation of the optimal stopping time τ∗ inthe problem (10.2.4) for c > z∗

1 as defined in (10.2.27). The process aboveis a standard Brownian motion which at time t starts at x . The optimaltime τ∗ is obtained by stopping the process as soon as it hits the areaabove or below the parabolic boundary r → ±z∗

√r .

Corollary 10.4. Let B = (Bt)t≥0 be a standard Brownian motion started at 0 ,and let τ be any stopping time for B . Then the following inequality is satisfied :

E |Bτ | ≤ z∗1 E√

τ (10.2.28)

with z∗1 being the unique positive root of M(−1/2 , 1/2 , z2/2) = 0 . The constantz∗1 is best possible. The equality is attained through the stopping times

τ∗ = inf

r > 0 : |Br| ≥ z∗√

t + r

(10.2.29)

when t ↓ 0 and c ↓ z∗1 , where z∗ is the unique positive root of the equation

z−1 M(− 12 , 1

2 , z2

2 ) = (c − z)M(12 , 3

2 , z2

2 ) (10.2.30)

satisfying z∗ < z∗1 . (Numerical calculations show that z∗1 = 1.30693 . . . )

3. The optimal stopping problem (10.2.4) can naturally be extended fromthe power 1 to all other p > 0 . For this consider the optimal stopping problemwith the value function

V∗(t, x) = supτ

Ex

(|Xτ |p − c

(t + τ

)p/2)

(10.2.31)


where the supremum is taken over all stopping times τ for X satisfying Exτp/2 <∞ and c > 0 is given and fixed.

Note that the case p = 2 is easily solved directly, since we have

V∗(t, x) = supτ

((1− c)Eτ + x2 − c t

)(10.2.32)

due to E |Bτ |2 = Eτ whenever Eτ < ∞ . Hence we see that V∗(t, x) = +∞ ifc < 1 (and it is never optimal to stop), and V∗(t, x) = x2 − ct if c ≥ 1 (and itis optimal to stop instantly). Thus below we concentrate most on the cases whenp = 2 (although the results formally extend to the case p = 2 by passing to thelimit).

The following extension of Theorem 10.3 and Corollary 10.4 is valid. (Notethat in the second part of the results we make use of parabolic cylinder functionsz → Dp(z) defined in (10.2.158) below.)

Theorem 10.5. (I): For 0 < p < 2 given and fixed, let z∗p denote the unique pos-itive root of M(−p/2 , 1/2 , z2/2) = 0 . The value function of the optimal stoppingproblem (10.2.31) for c ≥ (z∗p)p is given by

V∗(t, x) (10.2.33)

=

−tp/2 zp−2

∗ M(− p2 , 1

2 , x2

2t )/M(1− p2 , 3

2 ,z2∗2 ) if |x|/√t < z∗,

|x|p − c tp/2 if |x|/√t ≥ z∗

where z∗ is the unique positive root of the equation

zp−2 M(− p2 , 1

2 , z2

2 ) = (c − zp)M(1− p2 , 3

2 , z2

2 ) (10.2.34)

satisfying z∗ ≤ z∗p . The optimal stopping time in (10.2.31) for c > (z∗p)p is givenby

τ∗ = inf r > 0 : |Xr| ≥ z∗√

t + r . (10.2.35)

For c = (z∗p)p the stopping times τ∗ are approximately optimal if we let c ↓ (z∗p)p .For c < (z∗p)p we have V∗(t, x) = ∞ and it is never optimal to stop.

(II): For 2 < p < ∞ given and fixed, let zp denote the largest positive rootof Dp(z) = 0 . The value function of the optimal stopping problem (10.2.31) forc ≥ (zp)p is given by

V∗(t, x) =

⎧⎨⎩tp/2 zp−1∗ e(x2/4t)−(z2

∗/4) Dp(|x|/√

t)Dp−1(z∗)

if |x|/√t > z∗,

|x|p − c tp/2 if |x|/√t ≤ z∗(10.2.36)

where z∗ is the unique root of the equation

zp−1 Dp(z) = (zp − c)Dp−1(z) (10.2.37)


satisfying z∗ ≥ zp . The optimal stopping time in (10.2.31) for c > (zp)p is givenby

τ∗ = inf r > 0 : |Xr| ≤ z∗√

t + r . (10.2.38)

For c = (zp)p the stopping times τ∗ are approximately optimal if we let c ↓ (zp)p .For c < (zp)p we have V∗(t, x) = ∞ and it is never optimal to stop.

Proof. The proof is an easy extension of the proof of Theorem 10.3, and we onlypresent a few steps with differences for convenience.

By Brownian scaling we have

V∗(t, x) = supτ

Ex

(|Bτ + x|p − c

(t + τ

)p/2)

(10.2.39)

= tp/2 supτ/t

Ex

(∣∣t−1/2Bt(τ/t) + x/√

t∣∣p − c (1 + τ/t)p/2

)and hence we see that

V∗(t, x) = tp/2 V∗(1, x/√

t) . (10.2.40)

By the time change t → σt from (10.1.15) we find

|Xστ |p − c (1 + στ )p/2 = (1 + στ )p/2(|Zτ |p − c

)= epτ

(|Zτ |p − c)

(10.2.41)


V∗(1, x) = W∗(x) (10.2.42)


W∗(z) = supτ

Ez

(epτ(|Zτ |p − c

))(10.2.43)

the supremum being taken over all stopping times τ for Z for which Ezepτ < ∞ .

To compute W∗ we are naturally led to formulate the following free-boundaryproblem:

LZW (z) = −p W (z) for z ∈ C, (10.2.44)W (z) = |z|p − c for z ∈ ∂C (instantaneous stopping), (10.2.45)

W ′(z) = sign(z) p|z|p−1 for z ∈ ∂C (smooth fit) (10.2.46)

where C is the domain of continued observation. Observe again that W∗ shouldbe even.

In the case 0 0 : |Zt| ≥ z∗

(10.2.47)


Figure IV.7: A computer drawing of the solution of the free-boundaryproblem (10.2.44)–(10.2.46) for positive z when p = 2.5 . The solutionequals z → Bp exp(z2/4) Dp(z) for z > z∗ and z → zp − c for 0 ≤ z ≤z∗ . The solution extends to negative z by mirroring to an even function.The constant Bp is chosen (and z∗ is obtained) such that the smoothfit holds at z∗ (the first derivative of the solution is continuous at z∗ ).A similar picture holds for all other p > 2 which are not even integers.

is optimal. The proof in this case can be carried out along exactly the samelines as above when p = 1 . However, in the case 2 0 : |Zt| ≤ z∗

(10.2.48)

is optimal. The proof in this case requires a small modification of the previousargument. The main difference is that the solution of (10.2.44) used above doesnot have the power of smooth fit (10.2.45)–(10.2.46) any longer. It turns out,however, that the solution z → ez2/4Dp(z) has this power (see Figure IV.7 andFigure IV.8), and once this is understood, the proof is again easily completedalong the same lines as above (see also Remark 10.7 below). Corollary 10.6. Let B = (Bt)t≥0 be a standard Brownian motion started at zero,and let τ be any stopping time for B .

(I): For 0 < p ≤ 2 the following inequality is satisfied :

E |Bτ |p ≤ (z∗p)p Eτp/2 (10.2.49)


Figure IV.8: A computer drawing of the solution of the free-boundaryproblem (10.2.44)–(10.2.46) when p = 4 . The solution equals z →Bp exp(z2/4) Dp(z) for |z| > z∗ and z → |z|p − c for |z| ≤ z∗ . Theconstant Bp is chosen (and z∗ is obtained) such that the smooth fitholds at ±z∗ (the first derivative of the solution is continuous at ±z∗ ).A similar picture holds for all other p > 2 which are even integers.

with z∗p being the unique positive root of M(−p/2 , 1/2 , z2/2) = 0 . The constant(z∗p)p is best possible. The equality is attained through the stopping times

τ∗ = inf

r > 0 : |Br| ≥ z∗√

t + r

(10.2.50)

when t ↓ 0 and c ↓ (z∗p)p , where z∗ is the unique positive root of the equation

zp−2 M(− p2 , 1

2 , z2

2 ) = (c − zp)M(1 − p2 , 3

2 , z2

2 ) (10.2.51)

satisfying z∗ < z∗p .(II): For 2 ≤ p < ∞ the following inequality is satisfied :

E |Bτ |p ≤ (zp)p Eτp/2 (10.2.52)

with zp being the largest positive root of Dp(z) = 0 . The constant (zp)p is bestpossible. The equality is attained through the stopping times

σ∗ = inf r > 0 : |Br+x| ≤ z∗√

r (10.2.53)

when x ↓ 0 and c ↓ (zp)p , where z∗ is the unique root of the equation

zp−1 Dp(z) = (zp − c)Dp−1(z) (10.2.54)

satisfying z∗ > zp .


Remark 10.7. The argument used above to verify (10.2.23) extends to the generalsetting of Theorem 10.5 and leads to the following explicit formulae for 0 0 define the following stopping times:

τa = inf

r > 0 : |Zr| ≥ a, (10.2.55)

γa = inf

r > 0 : |Xr| ≥ a√

t + r

. (10.2.56)

By Brownian scaling and the time change (10.1.15) it is easily verified that

Ex

(γa + t

)p/2 = tp/2 Ex/√

tepτa . (10.2.57)

The argument quoted above for |z| < a then gives

Ezepτa =

M(− p

2 , 12 , z2

2 )/M(− p

2 , 12 , a2

2 ) if 0 < a < z∗p,

∞ if a ≥ z∗p .(10.2.58)

Thus by (10.2.57) for |x| < a√

t we obtain

Ex

(γa + t

)p/2 =

tp/2 M(− p

2 , 12 , x2

2t )/M(− p

2 , 12 , a2

2 ) if 0 < a < z∗p,

∞ if a ≥ z∗p .(10.2.59)

This formula is also derived in [183].

2. For a > 0 define the following stopping times:

τa = inf

r > 0 : Zr ≤ a, (10.2.60)

γa = inf

r > 0 : Xr ≤ a√

t + r

. (10.2.61)

By precisely the same arguments for z > a we get

Ezepτa =

e(z2/4)−(a2/4) Dp(z)/Dp(a) if a > zp,

∞ if a ≤ zp,(10.2.62)

and for x > a√

t we thus obtain

Ex

(γa + t

)p/2 =

tp/2 e(x2/4t)−(a2/4) Dp(x/

√t)/Dp(a) if a > zp,

∞ if a ≤ zp .(10.2.63)

This formula is also derived in [143].



V∗(t, x) = supτ

Ex

( Xτ

t + τ

)(10.2.64)

where the supremum is taken over all stopping times τ for X . This problemwas first solved by Shepp [184] and Taylor [210], and it was later extended byWalker [220] and Van Moerbeke [214]. To compute (10.2.64) we shall use the samearguments as in the proof of Theorem 10.3 above.

1. In the first step we rewrite (10.2.64) as

V∗(t, x) = supτ

E

(Bτ + x

t + τ

)=

1√t

supτ/t

E

(t−1/2Bt(τ/t) + x/

√t

1 + τ/t

)(10.2.65)

and note by Brownian scaling that

V∗(t, x) = 1√tV∗(1, x/

√t) (10.2.66)

so that we only need to look at V∗(1, x) in the sequel. In exactly the same wayas in Remark 10.2 above, from (10.2.66) we can heuristically conclude that theoptimal stopping boundary should be x = γ0

√t for some γ0 > 0 to be found.

2. In the second step we apply the time change t → σt from (10.1.15) tothe problem V∗(1, x) and transform it into a new problem. From (10.2.1) we get

Xστ /(1 + στ ) = Zτ/√

1 + στ = e−τZτ (10.2.67)


V∗(1, x) = W∗(x) (10.2.68)


W∗(z) = supτ

Ez

(e−τZτ

)(10.2.69)

the supremum being taken over all stopping times τ for Z .

3. In the third step we solve the problem (10.2.69). From general optimalstopping theory (Chapter I) we know that the following stopping time should beoptimal:

τ∗ = inf

t > 0 : Zt ≥ z∗

(10.2.70)

where z∗ is the optimal stopping point to be found.


To compute the value function W∗ for z < z∗ and to determine the optimalstopping point z∗ , it is natural (Chapter III) to formulate the following free-boundary problem:

LZW (z) = W (z) for z < z∗, (10.2.71)W (z∗) = z∗ (instantaneous stopping), (10.2.72)W ′(z∗) = 1 (smooth fit) (10.2.73)

with LZ in (10.2.3).The equation (10.2.71) is of the same type as the equation from Example 10.1.

Since the present problem is not symmetrical, we choose its general solution inaccordance with (10.2.160)–(10.2.161), i.e.

W (z) = Aez2/4D−1(z) + B ez2/4D−1(−z) (10.2.74)

where A and B are unknown constants.To determine A and B the following observation is crucial. Letting z → −∞

above, we see by (10.2.163) that ez2/4D−1(z) → ∞ and ez2/4D−1(−z) → 0 .Hence we find that A > 0 would contradict the clear fact that z → W∗(z) isincreasing, while A < 0 would contradict the fact that W∗(z) ≥ z (by observingthat ez2/4D−1(z) converges to ∞ faster than a polynomial). Therefore we musthave A = 0 . Moreover, from (10.2.163) we easily find that

ez2/4 D−1(−z) = ez2/2

∫ z

−∞e−u2/2 du (10.2.75)

and hence W ′(z) = z W (z) + B . The boundary condition (10.2.73) implies that1 = W ′(z∗) = z∗ W (z∗) + B = z2∗ + B , and hence we obtain B = 1 − z2∗ (seeFigure IV.9). Setting this into (10.2.72), we find that z∗ is the root of the equation

z = (1− z2) ez2/2

∫ z

−∞e−u2/2 du . (10.2.76)

In this way we have obtained the following candidate for the value functionW∗ :

W (z) =

⎧⎨⎩(1− z2∗) ez2/2

∫ z

−∞e−u2/2 du if z < z∗,

z if z ≥ z∗,(10.2.77)

and the following candidate for the optimal stopping time:

τz∗ = inf

t > 0 : Zt ≥ z∗

. (10.2.78)

4. To verify that these formulae are correct, we can apply Ito’s formula(page 67) to (e−t W (Zt))t≥0 , and in exactly the same way as in the proof ofTheorem 10.3 above we can conclude

W∗(z) ≤ W (z) . (10.2.79)


Figure IV.9: A computer drawing of the solution of the free-boundary problem (10.2.71)–(10.2.73). The solution equals z →B exp(z2/4) D−1(−z) for z < z∗ and z → z for z ≥ z∗ . The con-stant B is chosen (and z∗ is obtained) such that the smooth fit holds atz∗ (the first derivative of the solution is continuous at z∗ ).

To prove that equality is attained at τz∗ from (10.2.78), it is enough to show that

W (z) = Ez

(e−τz∗Zτz∗

)= z∗ Ez

(e−τz∗

). (10.2.80)

However, from general Markov process theory (Chapter III) we know that w(z) =Eze

−τz∗ solves (10.2.71), and clearly it satisfies w(z∗) = 1 and w(−∞) = 0 .Thus (10.2.80) follows from (10.2.77).

5. In this way we have established that formulae (10.2.77) and (10.2.78)are correct. Recalling by (10.2.66) and (10.2.68) that

V∗(t, x) = 1√tW∗(x/

√t) (10.2.81)

we have therefore proved the following result.

Theorem 10.9. The value function of the optimal stopping problem (10.2.64) isgiven by

V∗(t, x) =

⎧⎪⎨⎪⎩1√t(1− z2

∗) ex2/2t

∫ x/√

t

−∞e−u2/2 du if x/

√t < z∗,

x/t if x/√

t ≥ z∗

(10.2.82)


Figure IV.10: A computer simulation of the optimal stopping time τ∗in the problem (10.2.64) as defined in (10.2.84). The process above is astandard Brownian motion which at time t starts at x . The optimaltime τ∗ is obtained by stopping this process as soon as it hits the areaabove the parabolic boundary r → z∗

√r .


z = (1 − z2) ez2/2

∫ z

−∞e−u2/2 du . (10.2.83)

The optimal stopping time in 10.2.64 is given by (see Figure IV.10)

τ∗ = inf

r > 0 : Xr ≥ z∗√

t + r

. (10.2.84)

(Numerical calculations show that z∗ = 0.83992 . . . .)

4. Since the state space of the process X = (Xt)t≥0 is R the most naturalway to extend the problem (10.2.64) is to take X = (Xt)t≥0 to the power of anodd integer (such that the state space again is R ). Consider the optimal stoppingproblem with the value function

V∗(t, x) = supτ

Ex

(X2n−1

τ

(t + τ)q

)(10.2.85)

where the supremum is taken over all stopping times τ for X , and n ≥ 1 andq > 0 are given and fixed. This problem was solved by Walker [220] in the casen = 1 and q > 1/2 . We may now further extend Theorem 10.9 as follows.


Theorem 10.10. Let n ≥ 1 and q > 0 be taken to satisfy q > n − 12 . Then the

value function of the optimal stopping problem (10.2.85) is given by

V∗(t, x) (10.2.86)

=

⎧⎪⎨⎪⎩z2n−1∗ tn−q−1/2 e(x2/4t)−(z2

∗/4)

×D2(n−q)−1(−x/√

t)/D2(n−q)−1(−z∗) if x/

√t < z∗,

x2n−1/tq if x/√

t ≥ z∗


(2n−1)D2(n−q)−1(−z) = z(2(q−n) + 1

)D2(n−q−1)(−z) . (10.2.87)

The optimal stopping time in (10.2.85) is given by

τ∗ = inf

r > 0 : Xr ≥ z∗√

t + r

. (10.2.88)

(Note that in the case q ≤ n− 1/2 we have V∗(t, x) = ∞ and it is never optimalto stop.)

Proof. The proof will only be sketched, since the arguments are the same as forthe proof of Theorem 10.9. By Brownian scaling and the time change we find

V∗(t, x) = tn−q−1/2 W∗(x/√

t) (10.2.89)


W∗(z) = supτ

Ez

(e(2(n−q)−1)τ Z2n−1

τ

)(10.2.90)

the supremum being taken over all stopping times τ for Z .Again the optimal stopping time should be of the form

τ∗ = inf t > 0 : Zt ≥ z∗ (10.2.91)

and therefore the value function W∗ and the optimal stopping point z∗ shouldsolve the following free-boundary problem:

LZW (z) =(1−2(n− q)

)W (z) for 4z < z∗, (10.2.92)

W (z∗) = z2n−1∗ (instantaneous stopping), (10.2.93)

W ′(z∗) = (2n−1) z2(n−1)∗ (smooth fit). (10.2.94)

Arguing as in the proof of Theorem 10.9 we find that the following solution of(10.2.92) should be taken into consideration:

W (z) = Aez2/4 D2(n−q)−1(−z) (10.2.95)


where A is an unknown constant. The two boundary conditions (10.2.93)–(10.2.94) with (10.2.162) imply that A = z2n−1∗ e−z2

∗/4/D2(n−q)−1(−z∗) where z∗is the root of the equation

(2n − 1)D2(n−q)−1(−z) = z (2(q − n) + 1)D2(n−q−1)(−z) . (10.2.96)

Thus the candidate guessed for W∗ is

W (z) =

⎧⎨⎩z2n−1∗ e(z2/4)−(z2∗/4)

D2(n−q)−1(−z)D2(n−q)−1(−z∗)

if z < z∗,

z2n−1 if z ≥ z∗(10.2.97)

and the optimal stopping time is given by (10.2.91). By applying Ito’s formula(page 67) as in the proof of Theorem 10.9 one can verify that these formulae arecorrect. Finally, inserting this back into (10.2.89) one obtains the result.

Remark 10.11. By exactly the same arguments as in Remark 10.7 above, we canextend the verification of (10.2.80) to the general setting of Theorem 10.10, andthis leads to the following explicit formulae for 0 0 define the following stopping times:

τa = inf r > 0 : Zr ≥ a , (10.2.98)

γa = inf r > 0 : Xr ≥ a√

t + r . (10.2.99)

Then for z < a we get

Eze−pτa = e(z2/4)−(a2/4) D−p(−z)

D−p(−a)(10.2.100)

and for x < a√

t we thus obtain

Ex(γa + t)−p/2 = t−p/2 e(x2/4t)−(a2/4) D−p(−x/√

t)D−p(−a)

. (10.2.101)


V∗(t, x) = supτ

Ex

( |Xτ |t + τ

)(10.2.102)

where the supremum is taken over all stopping times τ for X . This problem(for the reflected Brownian motion |X | = (|Xt|)t≥0 ) is a natural extension of theproblem (10.2.64) and can be solved likewise.

By Brownian scaling and a time change we find

V∗(t, x) = 1√tW∗(x/

√t) (10.2.103)



W∗(z) = supτ

Ez

(e−τ |Zτ |

)(10.2.104)

the supremum being taken over all stopping times for Z .The problem (10.2.104) is symmetrical (recall the discussion about (10.2.10)

above), and therefore the following stopping time should be optimal:

τ∗ = inf t > 0 : |Zt| ≥ z∗ . (10.2.105)

Thus it is natural (Chapter III) to formulate the following free-boundary problem:

LZW (z) = W (z) for z ∈ (−z∗, z∗), (10.2.106)W (±z∗) = |z∗| (instantaneous stopping), (10.2.107)W ′(±z∗) = ±1 (smooth fit). (10.2.108)

From the proof of Theorem 10.3 we know that the equation (10.2.106) ad-mits an even and an odd solution which are linearly independent. Since the valuefunction should be even, we can forget the odd solution, and therefore we musthave

W (z) = AM(12 , 1

2 , z2

2 ) (10.2.109)

for some A > 0 to be found. Note that M(1/2 , 1/2 , z2/2) = exp(z2/2) (see pages192–193 below). The two boundary conditions (10.2.107) and (10.2.108) implythat A = 1/

√e and z∗ = 1 , and in this way we obtain the following candidate

for the value function:W (z) = e(z2/2)−(1/2) (10.2.110)

for z ∈ (−1, 1) , and the following candidate for the optimal stopping time:

τ = inf t > 0 : |Zt| ≥ 1 . (10.2.111)

By applying Ito’s formula (as in Example 10.8) one can prove that these formulaeare correct. Inserting this back into (10.2.103) we obtain the following result.

Theorem 10.13. The value function of the optimal stopping problem (10.2.102) isgiven by

V∗(t, x) =

1√te(x2/2t)−(1/2) if |x| <

√t,

|x|/t if |x| ≥ √t .

(10.2.112)


τ∗ = inf r > 0 : |Xr| ≥√

t + r . (10.2.113)


As in Example 10.8 above, we can further extend (10.2.102) by consideringthe optimal stopping problem with the value function

V∗(t, x) = supτ

Ex

( |Xτ |p(t + τ)q

)(10.2.114)

where the supremum is taken over all finite stopping times τ for X , and p , q > 0are given and fixed. The arguments used to solve the problem (10.2.102) can berepeated, and in this way we obtain the following result (see [138]).

Theorem 10.14. Let p , q > 0 be taken to satisfy q > p/2 . Then the value functionof the optimal stopping problem (10.2.114) is given by

V∗(t, x) (10.2.115)

=

zp∗ tp/2−q M(q− p

2 , 12 , x2

2t )/M(q− p2 , 1

2 ,z2∗2 ) if |x|/√t < z∗,

|x|p/tq if |x|/√t ≥ z∗


p M(q− p2 , 1

2 , z2

2 ) = z2 (2q−p)M(q+1− p2 , 3

2 , z2

2 ) . (10.2.116)


τ∗ = inf r > 0 : Xr ≥ z∗√

t + r . (10.2.117)

(Note that in the case q ≤ p/2 we have V∗(t, x) = ∞ and it is never optimal tostop.)

Example 10.15. In this example we indicate how the problem and the results inExample 10.1 and Example 10.12 above can be extended from reflected Brownianmotion to Bessel processes of arbitrary dimension α ≥ 0 . To avoid the computa-tional complexity which arises, we shall only indicate the essential steps towardssolution.

1. The case α > 1 . The Bessel process of dimension α > 1 is a unique(non-negative) strong solution of the stochastic differential equation

dXt =α−12Xt

dt + dBt (10.2.118)

satisfying X0 = x for some x ≥ 0 . The boundary point 0 is instantaneouslyreflecting if α < 2, and is an entrance boundary point if α ≥ 2 . (When α ∈N = 1, 2 . . . the process X = (Xt)t≥0 may be realized as the radial part of theα -dimensional Brownian motion.)

In the notation of Subsection 10.1 let us consider the process Y = (Yt)t≥0 =(β(t)Xt)t≥0 and note that b(x) = (α− 1)/2x and a(x) = 1 . Thus conditions(10.1.11) and (10.1.12) may be realized with γ(t) = β(t) , G1(y) = (α − 1)/2y


and G2(y) = 1 . Noting that β(t) = 1/√

1+t solves β′(t)/β(t) = −β2(t)/2 andsetting ρ = β2/2 , we see from (10.1.9) that

LZ =(−z +

α−1z

)∂

∂z+

∂2

∂z2(10.2.119)

where Z = (Zt)t≥0 = (Yσt)t≥0 with σt = e2t − 1 . Thus Z = (Zt)t≥0 solves theequation

dZt =(−Zt +

α−1Zt

)dt +

√2 dBt. (10.2.120)

Observe that the diffusion Z = (Zt)t≥0 may be seen as the Euclidean velocityof the α -dimensional Brownian motion whenever α ∈ N , and thus may be in-terpreted as the Euclidean velocity of the Bessel process X = (Xt)t≥0 of anydimension α > 1 .

The Bessel process X = (Xt)t≥0 of any dimension α ≥ 0 satisfies theBrownian scaling property Law

((c−1Xc2t)t≥0 | Px/c

)= Law

((Xt)t≥0 | Px

)for all

c > 0 and all x . Thus the initial arguments used in Example 10.1 and Ex-ample 10.12 can be repeated, and the crucial point in the formulation of thecorresponding free-boundary problem is the following analogue of the equations(10.2.11) and (10.2.106):

LZW (z) = ρ W (z) (10.2.121)

where ρ ∈ R . In comparison with the equation (10.2.151) this reads as follows:

y′′(x) −(x − α−1

x

)y′(x) − ρ y(x) = 0 (10.2.122)

where ρ ∈ R . By substituting y(x) = x−(α−1)/2 exp(x2/4)u(x) the equation(10.2.122) reduces to the following equation:

u′′(x) −(

x2

4 +(ρ − α

2

)+ α−1

2

(α−1

2 − 1)

1x2

)u(x) = 0 . (10.2.123)

The unpleasant term in this equation is 1/x2 , and the general solution is notimmediately found in the list of special functions in [1]. Motivated by our consid-erations below when 0 ≤ α ≤ 1, we may substitute y(x2) = y(x) and observethat the equation (10.2.122) is equivalent to

4z y ′′(z) + 2(α− z) y ′(z) − ρ y(z) = 0 (10.2.124)

where z = x2 . This equation now can be reduced to Whittaker’s equation (see[1]) as described in (10.2.131) and (10.2.132) below. The general solution of Whit-taker’s equation is given by Whittaker’s functions which are expressed in termsof Kummer’s functions. This establishes a basic fact about the extension of thefree-boundary problem from the reflected Brownian motion to the Bessel processof the dimension α > 1 . The problem then can be solved in exactly the same


manner as before. It is interesting to observe that if the dimension α of theBessel process X = (Xt)t≥0 equals 3, then the equation (10.2.123) is of the form(10.2.152), and thus the optimal stopping problem is solved immediately by usingthe corresponding closed form solution given in Example 10.1 and Example 10.12above.

2. The case 0 ≤ α ≤ 1 . The Bessel process of dimension 0 ≤ α ≤ 1 doesnot solve a stochastic differential equation in the sense of (10.2.118), and thereforeit is convenient to look at the squared Bessel process X = (Xt)t≥0 which is aunique (non-negative) strong solution of the stochastic differential equation

dXt = α dt + 2√

Xt dBt (10.2.125)

satisfying X0 = x for some x ≥ 0 . (This is true for all α ≥ 0 .) The Besselprocess X = (Xt)t≥0 is then defined as the square root of X = (Xt)t≥0 . Thus

Xt =√

Xt. (10.2.126)

The boundary point 0 is an instantaneously reflecting boundary point if 0 < α ≤1 , and is a trap if α = 0 . (The Bessel process X = (Xt)t≥0 may be realized as areflected Brownian motion when α = 1 .)

In the notation of Subsection 10.1 let us consider the process Y = (Yt)t≥0 =(β(t)Xt)t≥0 and note that b(x) = α and a(x) = 2

√x . Thus conditions (10.1.11)

and (10.1.12) may be realized with γ(t) = 1 , G1(y) = α and G2(y) = 4y . Notingthat β(t) = 1/(1+t) solves β′(t)/β(t) = −β(t) and setting ρ = β/2, we see from(10.1.9) that

LZ = 2(−z + α

) ∂

∂z+ 4z

∂2

∂z2(10.2.127)

where Z = (Zt)t≥0 = (Yσt)t≥0 with σt = e2t − 1 . Thus Z = (Zt)t≥0 solves theequation

dZt = 2(−Zt + α

)dt + 2

√2Zt dBt . (10.2.128)

It is interesting to observe that

Zt = Yσt =Xσt

1+σt=(

Xσt√1+σt

)2

(10.2.129)

and thus the process(√

Zt

)t≥0

may be seen as the Euclidean velocity of theα -dimensional Brownian motion for α ∈ [0, 1] .

This enables us to reformulate the initial problem about X = (Xt)t≥0 interms of X = (Xt)t≥0 and then after Brownian scaling and time change t → σt interms of the diffusion Z = (Zt)t≥0 . The pleasant fact is hidden in the formulationof the corresponding free-boundary problem for Z = (Zt)t≥0 :

LZW = ρ W (10.2.130)


which in comparison with the equation (10.2.151) reads as follows:

4x y′′(x) + 2(α−x) y′(x) − ρ y(x) = 0 . (10.2.131)

Observe that this equation is of the same type as equation (10.2.124). By substi-tuting y(x) = x−α/4 exp(x/4)u(x) equation (10.2.131) reduces to

u′′(x) +(− 1

16+

14

(ρ +

α

2

) 1x

+α

4

(1 − α

4

) 1x2

)u(x) = 0 (10.2.132)

which may be recognized as a Whittaker’s equation (see [1]). The general solutionof Whittaker’s equation is given by Whittaker’s functions which are expressed interms of Kummer’s functions. This again establishes a basic fact about extensionof the free-boundary problem from the reflected Brownian motion to the Besselprocess of dimension 0 ≤ α < 1 . The problem then can be solved in exactly thesame manner as before. Note also that the arguments about the passage to thesquared Bessel process just presented are valid for all α ≥ 0 . When α > 1 it is amatter of taste which method to choose.

Example 10.16. In this example we show how to solve some path-dependent optimalstopping problems (i.e. problems with the gain function depending on the entirepath of the underlying process up to the time of observation). For comparisonwith general theory recall Section 6 above.

Given an Ornstein–Uhlenbeck process Z = (Zt)t≥0 satisfying (10.2.2),started at z under Pz , consider the optimal stopping problem with the valuefunction

W∗(z) = supτ

Ez

(∫ τ

0

e−uZu du

)(10.2.133)

where the supremum is taken over all stopping time τ for Z . This problemis motivated by the fact that the integral appearing above may be viewed as ameasure of the accumulated gain (up to the time of observation) which is assumedproportional to the velocity of the Brownian particle being discounted. We willfirst verify by Ito’s formula (page 67) that this problem is in fact equivalent tothe one-dimensional problem (10.2.69). Then by using the time change σt weshall show that these problems are also equivalent to yet another path-dependentoptimal stopping problem which is given in (10.2.140) below.

1. Applying Ito’s formula (page 67) to the process (e−tZt)t≥0 , we find byusing (10.2.2) that

e−tZt = z + Mt − 2∫ t

0

e−uZu du (10.2.134)


Mt =√

2∫ t

0

e−u dBu . (10.2.135)


If τ is a bounded stopping time for Z , then by the optional sampling theorem(page 60) we get

Ez

(∫ τ

0

e−uZu du)

=12

(z + Ez

(e−τ (−Zτ )

)). (10.2.136)

Taking the supremum over all bounded stopping times τ for Z , and using that−Z = (−Zt)t≥0 is an Ornstein–Uhlenbeck process starting from −z under Pz ,we obtain

W∗(z) =12(z + W∗(−z)

)(10.2.137)

where W∗ is the value function from (10.2.69). The explicit expression for W∗is given in (10.2.77), and inserting it in (10.2.137), we immediately obtain thefollowing result.

Corollary 10.17. The value function of the optimal stopping problem (10.2.133) isgiven by

W∗(z) =

⎧⎨⎩ 12

(z+(1−z2

∗) ez2/2

∫ ∞

z

e−u2/2 du)

if z > −z∗,

0 if z ≤ −z∗(10.2.138)

where z∗ > 0 is the unique root of (10.2.83). The optimal stopping time in(10.2.133) is given by

τ∗ = inf t > 0 : Zt ≤ −z∗ . (10.2.139)

2. Given the Brownian motion Xt = Bt + x started at x under Px ,consider the optimal stopping problem with the value function

V∗(t, x) = supτ

Ex

(∫ τ

0

Xu

(t+u)2du

)(10.2.140)

where the supremum is taken over all stopping times τ for X . It is easily verifiedby Brownian scaling that we have

V∗(t, x) =1√tV∗(1, x/

√t) . (10.2.141)

Moreover, by time change (10.1.15) we get∫ στ

0

Xu

(1+u)2du =

∫ τ

0

Xσu

(1+σu)2dσu (10.2.142)

= 2∫ τ

0

e2u(1+σu)−3/2Zu du = 2∫ τ

0

e−uZu du


V∗(1, x) = W∗(x) (10.2.143)


where W∗ is given by (10.2.133). From (10.2.141) and (10.2.143) we thus obtainthe following result as an immediate consequence of Corollary 10.17.

Corollary 10.18. The value function of the optimal stopping problem (10.2.140) isgiven by

V∗(t, x) =

⎧⎨⎩xt + 1√

t(1− z2

∗) ex2/2t

∫ ∞

x/√

t

e−u2/2 du if x/√

t > −z∗,

0 if x/√

t ≤ −z∗(10.2.144)

where z∗ > 0 is the unique root of (10.2.83). The optimal stopping time in(10.2.140) is given by

τ∗ = inf r > 0 : Xr ≤ −z∗√

t + r . (10.2.145)

3. The optimal stopping problem (10.2.133) can be naturally extended byconsidering the optimal stopping problem with the value function

W∗(z) = supτ

Ez

(∫ τ

0

e−puHen(Zu) du)

(10.2.146)

where the supremum is taken over all stopping times τ for Z and x → Hen(x)is the Hermite polynomial given by (10.2.166), with p > 0 given and fixed. Thecrucial fact is that x → Hen(x) solves the differential equation (10.2.151), and byIto’s formula (page 67) and (10.2.2) this implies

e−ptHen(Zt) = Hen(z) (10.2.147)

+ Mt +∫ t

0

e−pu(LZ

(Hen

)(Zu) − pHen(Zu)

)du

= Hen(z) + Mt − (n+p)∫ t

0

e−puHen(Zu) du


Mt =√

2∫ t

0

e−pu(Hen)′(Zu) du . (10.2.148)

Again as above we find that

W∗(z) = 1n+p

(Hen(z) + W∗(z)

)(10.2.149)

with W∗ being the value function of the optimal stopping problem

W∗(z) = supτ

Ez

(e−pτ

(−Hen(Zu)))

(10.2.150)


where the supremum is taken over all stopping times τ for Z . This problem isone-dimensional and can be solved by the method used in Example 10.1.

4. Observe that the problem (10.2.146) with the arguments just presentedcan be extended from the Hermite polynomial to any solution of the differentialequation (10.2.151).

5. Auxiliary results. In the examples above we need the general solution ofthe second-order differential equation

y′′(x) − x y′(x) − ρ y(x) = 0 (10.2.151)

where ρ ∈ R . By substituting y(x) = exp(x2/4)u(x) the equation (10.2.151)reduces to

u′′(x) −(

x2

4+(ρ − 1

2

))u(x) = 0 . (10.2.152)

The general solution of (10.2.152) is well known, and in the text above we makeuse of the following two pairs of linearly independent solutions (see [1]).

1. The Kummer confluent hypergeometric function is defined by

M(a, b, x) = 1 +a

bx +

a(a + 1)b(b + 1)

x2

2!+ · · · . (10.2.153)

Two linearly independent solutions of (10.2.152) can be expressed as

u1(x) = e−x2/4 M(ρ2 , 1

2 , x2

2 ) & u2(x) = x e−x2/4 M(ρ2 + 1

2 , 32 , x2

2 ) (10.2.154)

and therefore two linearly independent solutions of (10.2.151) are given by

y1(x) = M(ρ2 , 1

2 , x2

2 ), (10.2.155)

y2(x) = xM(ρ2 + 1

2 , 32 , x2

2 ) . (10.2.156)

Observe that y1 is even and y2 is odd. Note also that

M ′(a, b, x) = ab M(a + 1, b + 1, x) . (10.2.157)

2. The parabolic cylinder function is defined by

Dν(x) = A1 e−x2/4 M(− ν2 , 1

2 , x2

2 ) + A2 x e−x2/4 M(− ν2 + 1

2 , 32 , x2

2 ) (10.2.158)

where A1 = 2ν/2π−1/2 cos(νπ/2)Γ((1 + ν)/2) and A2 = 2(1+ν)/2π−1/2 sin(νπ/2)Γ(1 + ν/2) . Two linearly independent solutions of (10.2.152) can be expressed as

u1(x) = D−ρ(x) & u2(x) = D−ρ(−x) (10.2.159)

Section 11. The method of space-change 193

and therefore two linearly independent solutions of (10.2.151) are given by

y1(x) = ex2/4 D−ρ(x), (10.2.160)

y2(x) = ex2/4 D−ρ(−x) (10.2.161)

whenever −ρ /∈ N ∪ 0 . Note that y1 and y2 are not symmetric around zerounless −ρ ∈ N ∪ 0 . Note also that

d

dx

(ex2/4Dν(x)

)= ν ex2/4Dν−1(x) . (10.2.162)

Moreover, the following integral representation is valid:

Dν(x) =e−x2/4

Γ(−ν)

∫ ∞

0

u−ν−1 e−xu−u2/2 du (10.2.163)

whenever ν < 0 .

3. To identify zero points of the solutions above, it is useful to note that

M(−n , 12 , x2

2 ) = He2n(x)/He2n(0), (10.2.164)

ex2/4Dn(x) = Hen(x) (10.2.165)

where x → Hen(x) is the Hermite polynomial

Hen(x) = (−1)nex2/2 dn

dxn

(e−x2/2

)(10.2.166)

for n ≥ 0 . For more information on the facts presented in this part we refer to [1].

11. The method of space change

In this section we adopt the setting and notation from Section 8 above. Given twostate spaces E1 and E2 , any measurable function C : E1 → E2 is called a spacechange. It turns out that such space changes sometimes prove useful in solvingoptimal stopping problems. In this section we will discuss two simple examples ofthis type. It is important to notice (and keep in mind) that any change of spacecan be performed either on the process (probabilistic transformation) or on theequation of the infinitesimal generator (analytic transformation). The two trans-formations stand in one-to-one correspondence to each other and yield equivalentconclusions.


To illustrate two examples of space change, let us assume that X = (Xt)t≥0 is aone-dimensional diffusion process solving




V (x) = supτ

ExG(Xτ ) (11.1.2)

where the supremum is taken over all stopping times τ of X and X0 = x underPx with x ∈ R .

1. Change of scale. Given a strictly increasing smooth function C : R → R ,set

Zt = C(Xt) (11.1.3)

and note that we can write

G(Xt) = G(C−1 C(Xt)

)= (G C−1)(Zt) = G(Zt) (11.1.4)

for t ≥ 0 where we denote

G(z) = G(C−1(z)) (11.1.5)

for z ∈ R (in the image of C ). By Ito’s formula (page 67) we get

C(Xt) = C(X0) +∫ t

0

(LXC)(Xs) ds +∫ t

0

C′(Xs)σ(Xs) dBs (11.1.6)

or equivalently

Zt = Z0 +∫ t

0

(LXC)(C−1(Zs)) ds +∫ t

0

C′(C−1(Zs))σ(C−1(Zs)) dBs (11.1.7)

for t ≥ 0 upon recalling that LXC = ρ C′+(σ2/2)C′′ . From (11.1.4) and (11.1.7)we see that the problem (11.1.2) is equivalent to the following problem:

V (z) = supτ

EzG(Zτ ) (11.1.8)

where Z = (Zt)t≥0 is a new one-dimensional diffusion process solving

dZt = ρ(Zt) dt + σ(Zt) dBt (11.1.9)

with Z0 = z under Pz and:

ρ = (LXC) C−1, (11.1.10)

σ = (C′σ) C−1. (11.1.11)

For some C the process Z may be simpler than the initial process X andthis in turn may lead to a solution of the problem (11.1.8). This solution is thenreadily transformed back to a solution of the initial problem (11.1.2) using (11.1.3).

Section 11. The method of space-change 195

The best known example of such a function C is the scale function S : R →R of X solving

LXS = 0. (11.1.12)

This yields the following explicit expression:

S(x) =∫ x

.

exp(−∫ y

.

2ρ(z)σ2(z)

dz

)dy (11.1.13)

which is determined uniquely up to an affine transformation ( S = aS + b is alsoa solution to (11.1.12) when a, b ∈ R ). From (11.1.7) one sees that Z = S(X)is a continuous (local) martingale (since the first integral term vanishes). Themartingale property may then prove helpful in the search for a solution to (11.1.8).Moreover, yet another step may be to time change Z and reduce the setting to astandard Brownian motion as indicated in (8.1.3)–(8.1.5).

2. Change of variables. Assuming that G is smooth (e.g. C2 ) we find byIto’s formula (page 67) that

G(Xt) = G(X0) +∫ t

0

(LXG)(Xs) ds +∫ t

0

G′(Xs)σ(Xs) dBs (11.1.14)

where Mt :=∫ t

0 G′(Xs)σ(Xs) dBs is a continuous (local) martingale for t ≥ 0 .By the optional sampling theorem (page 60) upon localization if needed, we mayconclude that ExMτ = 0 for all stopping times τ satisfying Ex

√τ < ∞ , given

that G′ and σ satisfy certain integrability conditions (e.g. both being bounded).In this case it follows from (11.1.14) that

ExG(Xτ ) = G(x) + Ex

(∫ τ

0

(LXG)(Xs) ds

)(11.1.15)

for all stopping times τ of X satisfying Ex√

τ < ∞ . Setting

L = LXG (11.1.16)

we see that the Mayer formulated problem (11.1.2) is equivalent to the Lagrangeformulated problem

V (x) = supτ

Ex

(∫ τ

0

L(Xt) dt

)(11.1.17)

where the supremum is taken over all stopping times τ of X . Moreover, if weare given G + M instead of G in (11.1.2), where M : R → R is a measurablefunction satisfying the usual integrability condition, then (11.1.2) is equivalent tothe Bolza formulated problem

V (x) = supτ

Ex

(M(Xτ ) +

∫ τ

0

L(Xt) dt

)(11.1.18)

where the supremum is taken as in (11.1.17) above.


It should be noted that the underlying Markov process X in (11.1.2) trans-forms into the underlying Markov process (X, I) in (11.1.18) where It =∫ t

0L(Xs) ds for t ≥ 0 . The latter process is more complicated and frequently,

when (11.1.18) is to be solved, one applies the preceding transformation to reducethis problem to problem (11.1.2). One simple example of this type will be givenin the next subsection. It should also be noted however that when we are givenN(Iτ ) instead of Iτ =

∫ τ

0 L(Xt) dt in (11.1.18), where N is a nonlinear function(e.g. N(x) =

√x ), then the preceding transformation is generally not applicable

(cf. Section 20 below). While in the former case ( N(x) = x ) we speak of linearproblems , in the latter case we often speak of nonlinear problems.


Let us illustrate the preceding transformation (change of variables) with one simpleexample.

Example 11.1. Consider the optimal stopping problem

V = supτ

E(|Bτ | − τ

)(11.2.1)

where the supremum is taken over all stopping times τ of the standard Brownianmotion B satisfying Eτ < ∞ . By Ito’s formula (page 67) applied to F (Bt) = B2

t ,and the optional sampling theorem (page 60), we know that

Eτ = EB2τ (11.2.2)

whenever Eτ < ∞ . It follows that problem (11.2.1) is equivalent to the problem

V = supτ

E(|Bτ | − |Bτ |2

)(11.2.3)

where the supremum is taken as in (11.2.1). Setting Z = |Bτ | with τ as above,we clearly have

E(|Bτ | − |Bτ |2

)=∫ ∞

0

(z − z2) dPZ(z). (11.2.4)

Observing that the function z → z−z2 has a unique maximum on [0,∞) attainedat z∗ = 1

2 , we see that the supremum over all Z in (11.2.4) is attained at Z ≡ 12 .

Recalling that Z = |Bτ | we see that

τ∗ = inf t ≥ 0 : |Bt| = 1/2 (11.2.5)

is an optimal stopping time in (11.2.3). It follows that τ∗ is an optimal stoppingtime in the initial problem (11.2.1), and we have V = 1

2 − 14 = 1

4 as is seenfrom (11.2.4).

Further examples of this kind will be studied in Section 16 below.

Section 12. The method of measure-change 197

12. The method of measure change

In this section we will adopt the setting and notation from Section 8 above. Thebasic idea of the method of measure change is to reduce the dimension of theproblem by replacing the initial probability measure with a new probability mea-sure which preserves the Markovian setting. Such a replacement is most often notpossible but in some cases it works.


To illustrate the method in a general setting, let us assume that X is a one-dimensional diffusion process solving



V = sup0≤τ≤T

EG(Zτ ) (12.1.2)

where the supremum is taken over all stopping times τ of Z and G is a measur-able function satisfying needed regularity conditions. Recall that Z = (I, X, S)is a three-dimensional (strong) Markov process where I is the integral process ofX and S is the maximum process of X (see (6.0.2) and (6.0.3)).

Introduce the exponential martingale

Et = exp(∫ t

0

Hs dBs − 12

∫ t

0

H2s ds

)(12.1.3)

for t ≥ 0 where H is a suitable process making (12.1.3) well defined (and sat-isfying e.g. the Novikov condition E exp

(12

∫ T

0 H2s ds

)< ∞ which implies the

martingale property). Rewrite the expectation in (12.1.2) as follows (when possi-ble):

EG(Zτ ) = E

(Eτ

G(Zτ )Eτ

)= E

(G(Zτ )Eτ

)= E G(Yτ ) (12.1.4)

where the symbol E denotes the expectation under a new probabilities measureP given by

dP = ET dP; (12.1.5)

the symbol G denotes a new gain function, and Y is a (strong) Markov process.Clearly, finding H which makes the latter possible is the key issue which makesthe method applicable or not.

When this is possible (see Example 12.1 below to see how G and Y can bechosen as suggested) it follows that problem (12.1.2) is equivalent to the problem

V = sup0≤τ≤T

E G(Yτ ) (12.1.6)


in the sense that having a solution to (12.1.6) we can reconstruct the correspond-ing solution to (12.1.2) using (12.1.5), and vice versa. The advantage of problem(12.1.6) over problem (12.1.2) is that the former is often only one-dimensionalwhile the latter (i.e. the initial problem) may be two- or three-dimensional (recallour discussion in Subsection 6.2).


Let us illustrate the preceding discussion by one example.

Example 12.1. Consider the optimal stopping problem

V = supτ

E(e−λτ (Iτ − Xτ )

)(12.2.1)

where X is a geometric Brownian motion solving

dXt = ρXt dt + σXt dBt (12.2.2)

with X0 = 1 and I is the integral process of X given by

It =∫ t

0

Xs ds (12.2.3)

for t ≥ 0 . In (12.2.1) and (12.2.2) we assume that λ > 0 , ρ ∈ R , σ > 0 and Bis a standard Brownian motion.

Recall that the unique (strong) solution to (12.2.2) is given by

Xt = exp(σBt + (ρ−σ2/2)t

)= eρtEt (12.2.4)

where Et = exp(σBt− (σ2/2)t

)is an exponential martingale for t ≥ 0 . It follows

that (12.2.1) can be rewritten as follows:

V = supτ

E(e−λτ (Iτ − Xτ )

)= sup

τE(Xτe−λτ (Iτ /Xτ − 1)

)(12.2.5)

= supτ

E(e−(λ−ρ)τEτ (Iτ/Xτ − 1)

)= sup

τE(e−rτ (Yτ − 1)

)where dP = ET dP , we set r = λ − ρ , and Yt = It/Xt for t ≥ 0 .

It turns out that Y is a (strong) Markov process. To verify the Markovproperty note that for

Y yt =

y + It

Xt=

1Xt

(y +

∫ t

0

Xs ds

)(12.2.6)

Section 13. Optimal stopping of the maximum process 199

with y ∈ R we have

Y yt+h =

y +∫ t

0Xs ds +

∫ t+h

tXs ds

exp(σ(Bt+h −Bt) + σBt + ρ(t +h− t) + ρt

) (12.2.7)

=1

exp(σBh + ρh)

[1

Xt

(y +

∫ t

0

Xs ds)

+∫ t+h

t

exp(σ(Bs −Bt) + ρ(s− t)

)ds

]where ρ = ρ − σ2/2 and Bh = Bt+h − Bt is a standard Brownian motionindependent from FX

t for h ≥ 0 and t ≥ 0 . (The latter conclusion makes useof stationary independent increments of B .) From the final expression in (12.2.7)upon recalling (12.2.6) it is evident that Y is a (strong) Markov process. Moreover,using Ito’s formula (page 67) it is easily checked that Y solves

dYt =(1 + (σ2 −ρ)

)Yt dt + σYt dBt (12.2.8)

where B = −B is also a standard Brownian motion. It follows that the infinites-imal generator of Y is given by

LY =(1 + (σ2 −ρ)

) ∂

∂y+

σ2y2

2∂2

∂y2(12.2.9)

and the problem (12.2.5) can be treated by standard one-dimensional techniques(at least when the horizon is infinite).

Further examples of this kind (involving the maximum process too) will bestudied in Sections 26 and 27.

13. Optimal stopping of the maximum process

13.1. Formulation of the problem

Let X = (Xt)t≥0 be a one-dimensional time-homogeneous diffusion process as-sociated with the infinitesimal generator

LX = ρ(x)∂

∂x+

σ2(x)2

∂2

∂x2(13.1.1)

where the drift coefficient x → ρ(x) and the diffusion coefficient x → σ(x) > 0are continuous. Assume moreover that there exists a standard Brownian motionB = (Bt)t≥0 defined on (Ω,F , P) such that X solves the stochastic differentialequation



with X0 = x under Px := Law(X | P, X0 = x) for x ∈ R . The state space of Xis assumed to be R .

With X we associate the maximum process

St =(

max0≤r≤t

Xr

)∨ s (13.1.3)

started at s ≥ x under Px,s := Law(X, S | P, X0 = x, S0 = s) . The main objectiveof this section is to present the solution to the optimal stopping problem with thevalue function

V∗(x, s) = supτ

Ex,s

(Sτ −

∫ τ

0

c(Xt) dt

)(13.1.4)

where the supremum is taken over stopping times τ of X satisfying

Ex,s

(∫ τ

0

c(Xt) dt

)< ∞, (13.1.5)

and the cost function x → c(x) > 0 is continuous.

1. To state and prove the initial observation about (13.1.4), and for furtherreference, we need to recall a few general facts about one-dimensional diffusions(recall Subsection 4.5 and see e.g. [178, p. 270–303] for further details).

The scale function of X is given by

L(x) =∫ x

exp(−∫ y 2ρ(z)

σ2(z)dz

)dy (13.1.6)

for x ∈ R . Throughout we denote

τx = inf t > 0 : Xt = x (13.1.7)

and set τx,y = τx ∧ τy . Then we have

Px

(Xτa,b

= a)

=L(b)−L(x)L(b)−L(a)

, (13.1.8)

Px

(Xτa,b

= b)

=L(x)−L(a)L(b)−L(a)

(13.1.9)

whenever a ≤ x ≤ b .

The speed measure of X is given by

m(dx) =2 dx

L′(x)σ2(x). (13.1.10)

The Green function of X on [a, b] is defined by

Ga,b(x, y) =

⎧⎪⎪⎨⎪⎪⎩(L(b)−L(x))(L(y)−L(a))

(L(b)−L(a))if a ≤ y ≤ x,

(L(b)−L(y))(L(x)−L(a))(L(b)−L(a))

if x ≤ y ≤ b.(13.1.11)


If f : R → R is a measurable function, then

Ex

(∫ τa,b

0

f(Xt) dt

)=∫ b

a

f(y)Ga,b(x, y)m(dy). (13.1.12)

2. Due to the specific form of the optimal stopping problem (13.1.4), thefollowing observation is nearly evident (see [45, p. 237–238]).

Proposition 13.1. The process Xt = (Xt, St) cannot be optimally stopped on thediagonal of R2 .

Proof. Fix x ∈ R , and set ln = x− 1/n and rn = x +1/n . Denoting τn = τln,rn

it will be enough to show that

Ex,x

(Sτn −

∫ τn

0

c(Xt) dt

)> x (13.1.13)

for n ≥ 1 large enough.

For this, note first by the strong Markov property and (13.1.8)–(13.1.9) that

Ex,x(Sτn) ≥ xPx(Xτn = ln) + rnPx(Xτn = rn) (13.1.14)

= xL(rn)−L(x)L(rn)−L(ln)

+ rnL(x)−L(ln)L(rn)−L(ln)

= x + (rn −x)L(x)−L(ln)L(rn)−L(ln)

= x + (rn −x)L′(ξn)(x− ln)L′(ηn)(rn − ln)

≥ x +K

n

since L ∈ C1 . On the other hand K1 := supln≤z≤rnc(z) < ∞ . Thus by (13.1.10)–

(13.1.12) we get

Ex,x

(∫ τn

0

c(Xt) dt

)≤ K1Exτn = 2K1

∫ rn

ln

Ga,b(x, y)dy

σ2(y)L′(y)(13.1.15)

≤ K2

(∫ x

ln

(L(y)−L(ln)

)dy +

∫ rn

x

(L(rn)−L(y)

)dy

)≤ K3

((x− ln)2 + (rn −x)2

)=

2K3

n2

since σ is continuous and L ∈ C1 . Combining (13.1.14) and (13.1.15) we clearlyobtain (13.1.13) for n ≥ 1 large enough. The proof is complete.

13.2. Solution to the problem

In the setting of (13.1.1)–(13.1.3) consider the optimal stopping problem (13.1.4)where the supremum is taken over all stopping times τ of X satisfying (13.1.5).


Our main aim in this subsection is to present the solution to this problem (Theorem13.2). We begin our exposition with a few observations on the underlying structureof (13.1.4) with a view to the Markovian theory of optimal stopping (Chapter I).

1. Note that Xt = (Xt, St) is a two-dimensional Markov process with thestate space D = (x, s) ∈ R2 : x ≤ s , which can change (increase) in the secondcoordinate only after hitting the diagonal x = s in R2 . Off the diagonal, the pro-cess X = (Xt)t≥0 changes only in the first coordinate and may be identified withX . Due to its form and behaviour at the diagonal, we claim that the infinitesimalgenerator of X may thus be formally described as follows:

LX = LX for x < s, (13.2.1)∂

∂s= 0 at x = s (13.2.2)

with LX as in (13.1.1). This means that the infinitesimal generator of X is actingon a space of C2 -functions f on D satisfying (∂f/∂s)(s, s) = 0 . Observe thatwe do not tend to specify the domain of LX precisely, but will only verify thatif f : D → R is a C2 -function which belongs to the domain, then (∂f/∂s)(s, s)must be zero.

To see this, we shall apply Ito’s formula (page 67) to the process f(Xt, St)and take the expectation under Ps,s . By the optional sampling theorem (page60) being applied to the continuous local martingale which appears in this process(localized if needed), we obtain

Es,sf(Xt, St) − f(s, s)t

= Es,s

(1t

∫ t

0

(LXf)(Xr, Sr) dr

)(13.2.3)

+ Es,s

(1t

∫ t

0

∂f

∂s(Xr, Sr) dSr

)−→ LXf(s, s) +

∂f

∂s(s, s)

(limt↓0

Es,s(St − s)t

)as t ↓ 0 . Due to σ > 0 , we have t−1Es,s(St − s) → ∞ as t ↓ 0 , and thereforethe limit above is infinite, unless (∂f/∂s)(s, s) = 0 . This completes the claim (seealso [45, p. 238–239]).

2. The problem (13.1.4) can be considered as a standard (i.e. of type (11.1.2))optimal stopping problem for a d -dimensional Markov process by introducing thefunctional

At = a +∫ t

0

c(Xr) dr (13.2.4)

with a ≥ 0 given and fixed, and noting that Zt = (At, Xt, St) is a Markovprocess which starts at (a, x, s) under P . Its infinitesimal generator is obtained


by adding c(x) (∂/∂a) to the infinitesimal generator of X , which combined with(13.2.1) leads to the formal description

LZ = c(x)∂

∂a+ LX in x < s, (13.2.5)

∂

∂s= 0 at x = s

with LX as in (13.1.1). Given Z = (Zt)t≥0 , introduce the gain function G(a, x, s)= s−a , note that the value function (13.1.4) viewed in terms of the general theoryought to be defined as

V∗(a, x, s) = supτ

EG(Zτ ) (13.2.6)

where the supremum is taken over all stopping times τ of Z satisfying EAτ < ∞ ,and observe that

V∗(a, x, s) = V∗(x, s) − a (13.2.7)

where V∗(x, s) is defined in (13.1.4). This identity is the main reason that weabandon the general formulation (13.2.6) and simplify it to the form (13.1.4), andthat we speak of optimal stopping for the process Xt = (Xt, St) rather than theprocess Zt = (At, Xt, St) .

Let us point out that the contents of this paragraph are used in the sequelmerely to clarify the result and method in terms of the general theory (recallSection 6 above).

3. From now on our main aim will be to show that the problem (13.1.4)reduces to the problem of solving a first-order nonlinear differential equation (forthe optimal stopping boundary). To derive this equation we shall first try to get afeeling for the points in the state space (x, s) ∈ R2 : x ≤ s at which the processXt = (Xt, St) can be optimally stopped (recall Figure 1 on page xviii above).

When on the horizontal level s , the process Xt = (Xt, St) stays at the samelevel until it hits the diagonal x = s in R2 . During that time X does not change(increase) in the second coordinate. Due to the strictly positive cost in (13.1.4),it is clear that we should not let the process X run too much to the left, since itcould be “too expensive” to get back to the diagonal in order to offset the “cost”spent to travel all that way. More specifically, given s there should exist a pointg∗(s) ≤ s such that if the process (X, S) reaches the point (g∗(s), s) we shouldstop it instantly. In other words, the stopping time

τ∗ = inf t>0 : Xt ≤ g∗(St) (13.2.8)

should be optimal for the problem (13.1.4). For this reason we call s → g∗(s)an optimal stopping boundary, and our aim will be to prove its existence and tocharacterize it. Observe by Proposition 13.1 that we must have g∗(s) < s for alls , and that V∗(x, s) = s for all x ≤ g∗(s) .


4. To compute the value function V∗(x, s) for g∗(s) < x ≤ s , and to findthe optimal stopping boundary s → g∗(s) , we are led (recall Section 6 above) toformulate the following system:

(LXV )(x, s) = c(x) for g(s) < x < s with s fixed, (13.2.9)∂V

∂s(x, s)

∣∣∣x=s−

= 0 (normal reflection), (13.2.10)

V (x, s)∣∣x=g(s)+

= s (instantaneous stopping), (13.2.11)

∂V

∂x(x, s)

∣∣∣x=g(s)+

= 0 (smooth fit) (13.2.12)

with LX as in (13.1.1). Note that (13.2.9)–(13.2.10) are in accordance with thegeneral theory (Section 6) upon using (13.2.5) and (13.2.7) above: the infinitesimalgenerator of the process being applied to the value function must be zero in thecontinuation set. The condition (13.2.11) is evident. The condition (13.2.12) is notpart of the general theory; it is imposed since we believe that in the “smooth”setting of the problem (13.1.4) the principle of smooth fit should hold (recallSection 6 above). This belief will be vindicated after the fact, when we showin Theorem 13.2.1, that the solution of the system (13.2.9)–(13.2.12) leads tothe value function of (13.1.4). The system (13.2.9)–(13.2.12) constitutes a free-boundary problem (see Chapter III above). It was derived for the first time byDubins, Shepp and Shiryaev [45] in the case of Bessel processes.

5. To solve the system (13.2.9)–(13.2.12) we shall consider a stopping timeof the form

τg = inf t>0 : Xt ≤ g(St) (13.2.13)

and the map

Vg(x, s) = Ex,s

(Sτg −

∫ τg

0

c(Xt) dt

)(13.2.14)

associated with it, where s → g(s) is a given function such that both Ex,sSτg

and Ex,s(∫ τg

0c(Xt) dt) are finite. Set Vg(s) := Vg(s, s) for all s . Considering

τg(s),s = inf t > 0 : Xt /∈ (g(s), s) and using the strong Markov property of Xat τg(s),s , by (13.1.8)–(13.1.12) we find

Vg(x, s) = sL(s)−L(x)

L(s)−L(g(s))+ Vg(s)

L(x)−L(g(s))L(s)−L(g(s))

(13.2.15)

−∫ s

g(s)

Gg(s),s(x, y) c(y)m(dy)

for all g(s) < x < s .


In order to determine Vg(s) , we shall rewrite (13.2.15) as follows:

Vg(s) − s (13.2.16)

=L(s)−L(g(s))L(x)−L(g(s))

((Vg(x, s) − s

)+∫ s

g(s)

Gg(s),s(x, y) c(y)m(dy))

and then divide and multiply through by x− g(s) to obtain

limx↓g(s)

Vg(x, s) − s

L(x)−L(g(s))=

1L′(g(s))

∂Vg

∂x(x, s)

∣∣∣x=g(s)+

. (13.2.17)

It is easily seen by (13.2.12) that

limx↓g(s)

L(s)−L(g(s))L(x)−L(g(s))

∫ s

g(s)

Gg(s),s(x, y) c(y)m(dy) (13.2.18)

=∫ s

g(s)

(L(s)−L(y)

)c(y)m(dy).

Thus, if the condition of smooth fit

∂Vg

∂x(x, s)

∣∣∣x=g(s)+

= 0 (13.2.19)

is satisfied, we see from (13.2.16)–(13.2.18) that the following identity holds:

Vg(s) = s +∫ s

g(s)

(L(s)−L(y)

)c(y)m(dy). (13.2.20)

Inserting this into (13.2.15), and using (13.1.11)–(13.1.12), we get

Vg(x, s) = s +∫ x

g(s)

(L(x)−L(y)

)c(y)m(dy) (13.2.21)

for all g(s) ≤ x ≤ s .If we now forget the origin of Vg(x, s) in (13.2.14), and consider it purely

as defined by (13.2.21), then it is straightforward to verify that (x, s) → Vg(x, s)solves the system (13.2.9)–(13.2.12) in the region g(s) < x < s if and only ifthe C1 -function s → g(s) solves the following first-order nonlinear differentialequation:

g′(s) =σ2(g(s))L′(g(s))

2 c(g(s)) [L(s)−L(g(s))]. (13.2.22)

Thus, to each solution s → g(s) of the equation (13.2.22) corresponds a function(x, s) → Vg(x, s) defined by (13.2.21) which solves the system (13.2.9)–(13.2.12)in the region g(s) < x < s , and coincides with the expectation in (13.2.14)whenever Ex,sSτg and Ex,s

∫ τg

0 c(Xt) dt are finite (the latter is easily verified


by Ito’s formula). We shall use this fact in the proof of Theorem 13.2 below uponapproximating the selected solution of (13.2.22) by solutions which hit the diagonalin R2 .

6. Observe that among all possible functions s → g(s) , only those whichsatisfy (13.2.22) lead to the smooth-fit property (13.2.19) for Vg(x, s) of (13.2.14),and vice versa. Thus the differential equation (13.2.22) is obtained by the principleof smooth fit in the problem (13.1.4). The fundamental question to be answeredis how to choose the optimal stopping boundary s → g∗(s) among all admissiblecandidates which solve (13.2.22).

Before passing to answer this question let us also observe from (13.2.21) that

∂Vg

∂x(x, s) = L′(x)

∫ x

g(s)

c(y)m(dy), (13.2.23)

V ′g(s) = L′(s)

∫ s

g(s)

c(y)m(dy). (13.2.24)

These equations show that, in addition to the continuity of the derivative ofVg(x, s) along the vertical line across g(s) in (13.2.19), we have obtained thecontinuity of Vg(x, s) along the vertical line and the diagonal in R2 across thepoint where they meet. In fact, we see that the latter condition is equivalent tothe former, and thus may be used as an alternative way of looking at the principleof smooth fit in this problem.

7. In view of the analysis of (13.2.8), we assign a constant value to Vg(x, s)at all x < g(s) . The following properties of the solution Vg(x, s) obtained arethen straightforward:

Vg(x, s) = s for x ≤ g(s), (13.2.25)x → Vg(x, s) is (strictly) increasing on [ g(s), s], (13.2.26)

(x, s) → Vg(x, s) is C2 outside (g(s), s) : s ∈ R, (13.2.27)

x → Vg(x, s) is C1 at g(s). (13.2.28)

Let us also make the following observations:

g → Vg(x, s) is (strictly) decreasing. (13.2.29)The function (a, x, s) → Vg(x, s) − a is superharmonic for theMarkov process Zt = (At, Xt, St) (with respect to stopping timesτ satisfying (13.1.5)).

(13.2.30)

The property (13.2.29) is evident from (13.2.21), whereas (13.2.30) is derived inthe proof of Theorem 13.2 (see (13.2.38) below).

8. Combining (13.2.7) and (13.2.29)–(13.2.30) with the superharmonic char-acterization of the value function from the Markovian theory (see Theorem 2.4


and Theorem 2.7), and recalling the result of Proposition 13.1, we are led to thefollowing maximality principle for determining the optimal stopping boundary (wesay that s → g∗(s) is an optimal stopping boundary for the problem (13.1.4), ifthe stopping time τ∗ defined in (13.2.8) is optimal for this problem).

The Maximality Principle. The optimal stopping boundary s → g∗(s) for the prob-lem (13.1.4) is the maximal solution of the differential equation (13.2.22) satisfyingg∗(s) < s for all s .

This principle is equivalent to the superharmonic characterization of the valuefunction (for the process Zt = (At, Xt, St) ), and may be viewed as its alternative(analytic) description. The proof of its validity is given in the next theorem, themain result of the subsection. (For simplicity of terminology we shall say that afunction g = g(s) is an admissible function if g(s) < s for all s .)

Theorem 13.2. (Optimal stopping of the maximum process) In the setting of(13.1.1)–(13.1.3) consider the optimal stopping problem (13.1.4) where the supre-mum is taken over all stopping times τ of X satisfying (13.1.5).

(I): Let s → g∗(s) denote the maximal admissible solution of (13.2.22) when-ever such a solution exists (see Figure IV.11). Then we have:

1. The value function is finite and is given by

V∗(x, s) = s +∫ x

g∗(s)

(L(x)−L(y)

)c(y)m(dy) (13.2.31)

for g∗(s) ≤ x ≤ s and V∗(x, s) = s for x ≤ g∗(s) .

2. The stopping time

τ∗ = inf t > 0 : Xt ≤ g∗(St) (13.2.32)

is optimal for the problem (13.1.4) whenever it satisfies (13.1.5); otherwise it is“approximately” optimal in the sense described in the proof below.

3. If there exists an optimal stopping time σ in (13.1.4) satisfying (13.1.5),then Px,s(τ∗ ≤ σ) = 1 for all (x, s), and τ∗ is an optimal stopping time for(13.1.4) as well.

(II): If there is no (maximal) admissible solution of (13.2.22), then V∗(x, s) =+∞ for all (x, s), and there is no optimal stopping time.

Proof. (I): Let s → g(s) be any solution of (13.2.22) satisfying g(s) < s forall s . Then, as indicated above, the function Vg(x, s) defined by (13.2.21) solvesthe system (13.2.9)–(13.2.12) in the region g(s) < x < s . Due to (13.2.27) and(13.2.28), Ito’s formula (page 67) can be applied to the process Vg(Xt, St) , and


s

x

C

x = ss g*(s)

Figure IV.11: A computer drawing of solutions of the differential equation(13.2.22) in the case when ρ ≡ 0 , σ ≡ 1 (thus L(x) = x ) and c ≡1/2 . The bold line s → g∗(s) is the maximal admissible solution. (Inthis particular case s → g∗(s) is a linear function.) By the maximalityprinciple proved below, this solution is the optimal stopping boundary(the stopping time τ∗ from (13.2.8) is optimal for the problem (13.1.4)).

in this way by (13.1.1)–(13.1.2) we get

Vg(Xt, St) = Vg(x, s) +∫ t

0

∂Vg

∂x(Xr, Sr) dXr (13.2.33)

+∫ t

0

∂Vg

∂s(Xr, Sr) dSr +

12

∫ t

0

∂2Vg

∂x2(Xr, Sr) d

⟨X, X

⟩r

= Vg(x, s) +∫ t

0

σ(Xr)∂Vg

∂x(Xr, Sr) dBr +

∫ t

0

(LXVg)(Xr , Sr) dr

where the integral with respect to dSr is zero, since the increment ∆Sr outsidethe diagonal in R2 equals zero, while at the diagonal we have (13.2.10).

The process M = (Mt)t≥0 defined by

Mt =∫ t

0

σ(Xr)∂Vg

∂x(Xr, Sr) dBr (13.2.34)

is a continuous local martingale. Introducing the increasing process

Pt =∫ t

0

c(Xr) 1(Xr≤g(Sr)) dr (13.2.35)


and using the fact that the set of all t for which Xt is either g(St) or St is ofLebesgue measure zero, the identity (13.2.33) can be rewritten as

Vg(Xt, St) −∫ t

0

c(Xr) dr = Vg(x, s) + Mt − Pt (13.2.36)

by means of (13.2.9) with (13.2.25). From this representation we see that theprocess Vg(Xt, St) −

∫ t

0 c(Xr) dr is a local supermartingale.

Let τ be any stopping time of X satisfying (13.1.5). Choose a localizationsequence (σn)n≥1 of bounded stopping times for M . By means of (13.2.25) and(13.2.26) we see that Vg(x, s) ≥ s for all (x, s) , so that from (13.2.36) it followsthat

Ex,s

(Sτ∧σn −

∫ τ∧σn

0

c(Xt) dt

)(13.2.37)

≤ Ex,s

(Vg(Xτ∧σn , Sτ∧σn) −

∫ τ∧σn

0

c(Xt) dt

)≤ Vg(x, s) + Ex,sMτ∧σn = Vg(x, s).

Letting n → ∞ , and using Fatou’s lemma with (13.1.5), we get

Ex,s

(Sτ −

∫ τ

0

c(Xt) dt

)≤ Vg(x, s). (13.2.38)

This proves (13.2.30). Taking the supremum over all such τ , and then the infimumover all such g , by means of (13.2.29) we may conclude

V∗(x, s) ≤ infg

Vg(x, s) = Vg∗(x, s) (13.2.39)

for all (x, s) . From these considerations it clearly follows that the only possiblecandidate for the optimal stopping boundary is the maximal solution s → g∗(s)of (13.2.22).

To prove that we have the equality in (13.2.39), and that the value functionV∗(x, s) is given by (13.2.31), assume first that the stopping time τ∗ defined by(13.2.32) satisfies (13.1.5). Then, as pointed out when deriving (13.2.21), we have

Vg∗(x, s) = Ex,s

(Sτg∗ −

∫ τg∗

0

c(Xt) dt

)(13.2.40)

so that Vg∗(x, s) = V∗(x, s) in (13.2.39) and τ∗ is an optimal stopping time. Theexplicit expression given in (13.2.31) is obtained by (13.2.21).

Assume now that τ∗ fails to satisfy (13.1.5). Let (gn)n≥1 be a decreasingsequence of solutions of (13.2.22) satisfying gn(s) ↓ g∗(s) as n → ∞ for all s .Note that each such solution must hit the diagonal in R2 , so the stopping times


τgn defined as in (13.2.13) must satisfy (13.1.5). Moreover, since Sτgnis bounded

by a constant, we see that Vgn(x, s) defined as in (13.2.14) is given by (13.2.21)with g = gn for n ≥ 1 . By letting n → ∞ we get

Vg∗(x, s) = limn→∞Vgn(x, s) = lim

n→∞ Ex,s

(Sτgn

−∫ τgn

0

c(Xt) dt

). (13.2.41)

This shows that the equality in (13.2.39) is attained through the sequence ofstopping times (τgn)n≥1 , and the explicit expression in (13.2.31) is easily obtainedas already indicated above.

To prove the final (uniqueness) statement, assume that σ is an optimalstopping time in (13.1.4) satisfying (13.1.5). Suppose that Px,s(σ<τ∗) > 0 . Notethat τ∗ can be written in the form

τ∗ = inf t>0 : V∗(Xt, St) = St (13.2.42)

so that Sσ < V∗(Xσ, Sσ) on σ < τ∗ , and thus

Ex,s

(Sσ −

∫ σ

0

c(Xt) dt

)< Ex,s

(V∗(Xσ, Sσ) −

∫ σ

0

c(Xt) dt

)(13.2.43)

≤ V∗(x, s)

where the latter inequality is derived as in (13.2.38), since the process V∗(Xt, St)−∫ t

0 c(Xr) dr is a local supermartingale. The strict inequality in (13.2.43) shows thatPx,s(σ<τ∗) > 0 fails, so we must have Px,s(τ∗≤σ) = 1 for all (x, s) .

To prove the optimality of τ∗ in such a case, it is enough to note that if σsatisfies (13.1.5) then τ∗ must satisfy it as well. Therefore (13.2.40) is satisfied,and thus τ∗ is optimal. A straightforward argument can also be given by using thelocal supermartingale property of the process V∗(Xt, St) −

∫ t

0 c(Xr) dr . Indeed,since Px,s(τ∗≤σ) = 1 , we get

V∗(x, s) = Ex,s

(Sσ −

∫ σ

0

c(Xt) dt

)(13.2.44)

≤ Ex,s

(V∗(Xσ, Sσ) −

∫ σ

0

c(Xt) dt

)≤ Ex,s

(V∗(Xτ∗ , Sτ∗) −

∫ τ∗

0

c(Xt) dt

)= Ex,s

(Sτ∗ −

∫ τ∗

0

c(Xt) dt

)so τ∗ is optimal for (13.1.4). The proof of the first part of the theorem is complete.

(II): Let (gn)n≥1 be a decreasing sequence of solutions of (13.2.22) whichsatisfy gn(0) = −n for n ≥ 1 . Then each gn must hit the diagonal in R2 atsome sn > 0 for which we have sn ↑ ∞ when n → ∞ . Since there is no solutionof (13.2.22) which is less than s for all s , we must have gn(s) ↓ −∞ as n → ∞


for all s . Let τgn denote the stopping time defined by (13.2.13) with g = gn .Then τgn satisfies (13.1.5), and since Sτgn

≤ s∨sn , we see that Vgn(x, s) , definedby (13.2.14) with g = gn , is given as in (13.2.21):

Vgn(x, s) = s +∫ x

gn(s)

(L(x)−L(y)

)c(y)m(dy) (13.2.45)

for all gn(s) ≤ x ≤ s . Letting n → ∞ in (13.2.45), we see that the integral

I :=∫ x

−∞

(L(x)−L(y)

)c(y)m(dy) (13.2.46)

plays a crucial role in the proof (independently of the given x and s ).

Assume first that I = +∞ (this is the case whenever c(y) ≥ ε > 0 for ally , and −∞ is a natural boundary point for X , see paragraph 11 below). Thenfrom (13.2.45) we clearly get

V∗(x, s) ≥ limn→∞Vgn(x, s) = +∞ (13.2.47)

so the value function must be infinite.

On the other hand, if I < ∞ , then (13.1.11)–(13.1.12) imply

Ex,s

(∫ τs

0

c(Xt) dt

)≤∫ s

−∞

(L(s)−L(y)

)c(y)m(dy) < ∞ (13.2.48)

where τs = inf t > 0 : Xt = s for s ≥ s . Thus, if we let the process (Xt, St)first hit (s, s) , and then the boundary (gn(s), s) : s ∈ R with n → ∞ , thenby (13.2.45) (with x = s = s ) we see that the value function equals at least s .More precisely, if the process (Xt, St) starts at (x, s) , consider the stopping timesτn = τs + τgn θτs

for n ≥ 1 . Then by (13.2.48) we see that each τn satisfies(13.1.5), and by the strong Markov property of X we easily get

V∗(x, s) ≥ lim supn→∞

Ex,s

(Sτn −

∫ τn

0

c(Xt) dt

)≥ s. (13.2.49)

By letting s ↑ ∞ , we again find V∗(x, s) = +∞ . The proof of the theorem iscomplete.

9. On the equation (13.2.22). Theorem 13.2 shows that the optimal stoppingproblem (13.1.4) reduces to the problem of solving the first-order nonlinear differ-ential equation (13.2.22). If this equation has a maximal admissible solution, thenthis solution is an optimal stopping boundary. We may note that this equation isof the following normal form:

y′ =F (y)

G(x)−G(y)(13.2.50)


for x > y , where y → F (y) is strictly positive, and x → G(x) is strictlyincreasing. To the best of our knowledge the equation (13.2.50) has not beenstudied before in full generality, and in view of the result proved above we wantto point out the need for its investigation. It turns out that its treatment dependsheavily on the behaviour of the map G .

(i): If the process X is in natural scale, that is L(x) = x for all x , we cancompletely characterize and describe the maximal admissible solution of (13.2.22).This can be done in terms of equation (13.2.50) with G(x) = x and F (y) =σ2(y)/2c(y) as follows. Note that by passing to the inverse z → y−1(z) , equation(13.2.50) in this case can be rewritten as

(y−1

)′(z) − 1F (z)

y−1(z) = − z

F (z). (13.2.51)

This is a first-order linear equation and its general solution is given by

y−1α (z) = exp

(∫ z

0

dy

F (y)

)(α −

∫ z

0

y

F (y)exp

(−∫ y

0

du

F (u)

)dy

), (13.2.52)

where α is a constant. Hence we see that, with G(x) = x , the necessary andsufficient condition for equation (13.2.50) to have a maximal admissible solution,is that

α∗ := supz∈R

(z exp

(−∫ z

0

dy

F (y)

)(13.2.53)

+∫ z

0

y

F (y)exp

(−∫ y

0

du

F (u)

)dy

)< ∞,

and that this supremum is not attained at any z ∈ R . In this case the maximaladmissible solution x → y∗(x) of (13.2.50) can be expressed explicitly throughits inverse z → y−1

α∗ (z) given by (13.2.52).

Note also when L(x) = G(x) = x2 sgn (x) that the same argument trans-forms (13.2.50) into a Riccati equation, which then can be further transformed intoa linear homogeneous equation of second order by means of standard techniques.The trick of passing to the inverse in (13.2.22) is further used in [160] where a nat-ural connection between the result of the present subsection and the Azema–Yorsolution of the Skorokhod-embedding problem [6] is described.

(ii): If the process X is not in natural scale, then the treatment of (13.2.50)is much harder, due to the lack of closed form solutions. In such cases it is possibleto prove (or disprove) the existence of the maximal admissible solution by usingPicard’s method of successive approximations. The idea is to use Picard’s theoremlocally, step by step, and in this way show the existence of some global solutionwhich is admissible. Then, by passing to the equivalent integral equation and usinga monotone convergence theorem, one can argue that this implies the existence of


the maximal admissible solution. This technique is described in detail in Section3 of [81] in the case of G(x) = xp and F (y) = yp+1 when p > 1 . It is alsoseen there that during the construction one obtains tight bounds on the maximalsolution which makes it possible to compute it numerically as accurate as desired(see [81] for details). In this process it is desirable to have a local existence anduniqueness of the solution, and these are provided by the following general facts.

From the general theory (Picard’s method) we know that if the directionfield (x, y) → f(x, y) := F (y)/(G(x)−G(y)) is (locally) continuous and (locally)Lipschitz in the second variable, then the equation (13.2.50) admits (locally) aunique solution. For instance, this will be so if along a (local) continuity of (x, y) →f(x, y) , we have a (local) continuity of (x, y) → (∂f/∂y)(x, y) . In particular, upondifferentiating over y in f(x, y) we see that (13.2.22) admits (locally) a uniquesolution whenever the map y → σ2(y)L′(y)/c(y) is (locally) C1 . It is also possibleto prove that the equation (13.2.50) admits (locally) a solution, if only the (local)continuity of the direction field (x, y) → F (y)/(G(x)−G(y)) is verified. However,such a solution may fail to be (locally) unique.

Instead of entering further into such abstract considerations here, we shallrather confine ourselves to some concrete examples with applications in ChapterV below.

10. We have proved in Theorem 13.2 that τ∗ is optimal for (13.1.4) wheneverit satisfies (13.1.5). In Example 18.7 we will exhibit a stopping time τ∗ whichfails to satisfy (13.1.5), but nevertheless its value function is given by (13.2.31)as proved above. In this case τ∗ is “approximately” optimal in the sense that(13.2.41) holds with τgn↑τ∗ as n → ∞ .

11. Other state spaces. The result of Theorem 13.2 extends to diffusions withother state spaces in R . In view of many applications, we will indicate such anextension for non-negative diffusions.

In the setting of (13.1.1)–(13.1.3) assume that the diffusion X is non-negative, consider the optimal stopping problem (13.1.4) where the supremumis taken over all stopping times τ of X satisfying (13.1.5), and note that theresult of Proposition 13.1 extends to this case provided that the diagonal is takenin (0,∞)2 . In this context it is natural to assume that σ(x) > 0 for x > 0 , andσ(0) may be equal 0 . Similarly, we shall see that the case of strictly positive costfunction c differs from the case when c is strictly positive only on (0,∞) . Inany case, both x → σ(x) and x → c(x) are assumed continuous on [0,∞) .

In addition to the infinitesimal characteristics from (13.1.1) which govern Xin (0,∞) , we must specify the boundary behaviour of X at 0 . For this we shallconsider the cases when 0 is a natural, exit, regular (instantaneously reflecting),and entrance boundary point (see [109, p. 226–250]).


The relevant fact in the case when 0 is either a natural or exit boundarypoint is that ∫ s

0

(L(s)−L(y)

)c(y)m(dy) = +∞ (13.2.54)

for all s > 0 whenever c(0) > 0 . In view of (13.2.31) this shows that for themaximal solution of (13.2.22) we must have 0 < g∗(s) < s for all s > 0 unlessV∗(s, s) = +∞ . If c(0) = 0 , then the integral in (13.2.54) can be finite, and wecannot state a similar claim; but from our method used below it will be clear howto handle such a case too, and therefore the details in this direction will be omittedfor simplicity.

The relevant fact in the case when 0 is either a regular (instantaneouslyreflecting) or entrance boundary point is that

E0,s

(∫ τs∗

0

c(Xt) dt

)=∫ s∗

0

(L(s∗)−L(y)

)c(y)m(dy) (13.2.55)

for all s∗ ≥ s > 0 where τs∗ = inf t > 0 : Xt = s∗ . In view of (13.2.31) thisshows that it is never optimal to stop at (0, s) . Therefore, if the maximal solutionof (13.2.22) satisfies g∗(s∗) = 0 for some s∗ > 0 with g∗(s) > 0 for all s > s∗ ,then τ∗ = inf t > 0 : Xt ≤ g∗(St) is to be the optimal stopping time, since Xdoes not take negative values. If moreover c(0) = 0 , then the value of m(0)does not play any role, and all regular behaviour (from absorption m(0) = +∞ ,over sticky barrier phenomenon 0 < m(0) < +∞ , to instantaneous reflectionm(0) = 0 ) can be treated in the same way.

For simplicity in the next result we will assume that c(0) > 0 if 0 is eithera natural (attracting or unattainable) or an exit boundary point, and will onlyconsider the instantaneously-reflecting regular case. The remaining cases can betreated similarly.

Corollary 13.3. (Optimal stopping for non-negative diffusions) In the setting of(13.1.1)–(13.1.3) assume that the diffusion X is non-negative, and that 0 is anatural, exit, instantaneously-reflecting regular, or entrance boundary point. Con-sider the optimal stopping problem (13.1.4) where the supremum is taken over allstopping times τ of X satisfying (13.1.5).

(I): Let s → g∗(s) denote the maximal admissible solution of (13.2.22) in thefollowing sense (whenever such a solution exists — see Figure IV.12): There existsa point s∗ ≥ 0 (with s∗ = 0 if 0 is either a natural or an exit boundary point)such that g∗(s∗) = 0 and g∗(s) > 0 for all s > s∗; the map s → g∗(s) solves(13.2.22) for s > s∗ and is admissible (i.e. g∗(s) < s for all s > s∗ ); the maps → g∗(s) is the maximal solution satisfying these two properties (the comparisonof two maps is taken pointwise wherever they are both strictly positive). Then wehave:


1. The value function is finite and for s ≥ s∗ is given by

V∗(x, s) = s +∫ x

g∗(s)

(L(x)−L(y)

)c(y)m(dy) (13.2.56)

for g∗(s) ≤ x ≤ s with V∗(x, s) = s for 0 ≤ x ≤ g∗(s) , and for s ≤ s∗ (when0 is either an instantaneously-reflecting regular or an entrance boundary point) isgiven by

V∗(x, s) = s∗ +∫ x

0

(L(x)−L(y)

)c(y)m(dy) (13.2.57)

for 0 ≤ x ≤ s .

2. The stopping time

τ∗ = inf t>0 : St ≥ s∗, Xt ≤ g∗(St) (13.2.58)

is optimal for the problem (13.1.4) whenever it satisfies (13.1.5); otherwise, it is“approximately” optimal.

3. If there exists an optimal stopping time σ in (13.1.4) satisfying (13.1.5),then Px,s(τ∗ ≤ σ) = 1 for all (x, s) , and τ∗ is an optimal stopping time for(13.1.4) as well.

(II): If there is no (maximal) solution of (13.2.22) in the sense of (I) above,then V∗(x, s) = +∞ for all (x, s) , and there is no optimal stopping time.

Proof. With only minor changes the proof can be carried out in exactly the sameway as the proof of Theorem 13.2 upon using the additional facts about (13.2.54)and (13.2.55) stated above, and the details will be omitted. Note, however, thatin the case when 0 is either an instantaneously-reflecting regular or an entranceboundary point, the strong Markov property of X at τs∗ = inf t>0 : Xt = s∗ gives

V∗(x, s) = s∗ +∫ s∗

0

(L(s∗)−L(y)

)c(y)m(dy) − Ex,s

(∫ τs∗

0

c(Xt) dt

)(13.2.59)

for all 0 ≤ x ≤ s ≤ s∗ . Hence formula (13.2.57) follows by applying (13.1.11)+(13.1.12) to the last term in (13.2.59). (In the instantaneous reflecting case onecan make use of τs∗,s∗ after extending L to R− by setting L(x) := −L(−x) forx < 0 ). The proof is complete.

12. The “discounted” problem. One is often more interested in the discountedversion of the optimal stopping problem (13.1.4). Such a problem can be reducedto the initial problem (13.1.4) by changing the underlying diffusion process.

Given a continuous function x → λ(x) ≥ 0 called the discounting rate, inthe setting of (13.1.1)–(13.1.3) introduce the functional

Λ(t) =∫ t

0

λ(Xr) dr, (13.2.60)


s

x

C

x = ss g

*(s)

(0,0)

Figure IV.12: A computer drawing of solutions of the differential equation(13.2.22) in the case when X is a geometric Brownian motion from Ex-ample 18.9 with ρ = −1 , σ2 = 2 (thus ∆ = 2 ) and c = 50 . The boldline s → g∗(s) is the maximal admissible solution. (In this particular casethere is no closed formula for s → g∗(s) , but it is proved that s → g∗(s)satisfies (18.4.15).)

and consider the optimal stopping problem with the value function

V∗(x, s) = supτ

Ex,s

(e−Λ(τ)Sτ −

∫ τ

0

e−Λ(t)c(Xt) dt

), (13.2.61)

where the supremum is taken over all stopping times τ of X for which the integralhas finite expectation, and the cost function x → c(x) > 0 is continuous.

The standard argument (Subsection 5.4) shows that the problem (13.2.61) isequivalent to the problem

V∗(x, s) = supτ

Ex,s

(Sτ −

∫ τ

0

c(Xt) dt

)(13.2.62)

where X = (Xt)t≥0 is a diffusion process which corresponds to the “killing” ofthe sample paths of X at the “rate” λ(X) . The infinitesimal generator of X isgiven by

L eX = −λ(x) + ρ(x)∂

∂x+

σ2(x)2

∂2

∂x2. (13.2.63)


We conjecture that the maximality principle proved above also holds for thisproblem (see [185] and [151]). The main technical difficulty in a general treatmentof this problem is the fact that the infinitesimal generator L eX has the term−λ(x) , so that L eX = 0 may have no simple solution. Nonetheless, it is clear thatthe corresponding system (13.2.9)–(13.2.12) must be valid, and this system definesthe (maximal) boundary s → g∗(s) implicitly.

13. The “Markovian” cost problem. Yet another class of optimal stoppingproblems (Mayer instead of Lagrange formulated) reduces to the problem (13.1.4).Suppose that in the setting of (13.1.1)–(13.1.3) we are given a smooth functionx → D(x) , and consider the optimal stopping problem with the value function

V∗(x, s) = supτ

Ex,s

(Sτ − D(Xτ )

)(13.2.64)

where the supremum is taken over a class of stopping times τ of X . Then a vari-ant of Ito’s formula (page 67) applied to D(Xt) , the optional sampling theorem(page 60) applied to the continuous local martingale Mt =

∫ t

0 D′(Xs)σ(Xs) dBs

localized if necessary, and uniform integrability conditions enable one to conclude

Ex,sD(Xτ ) = D(x) + Ex,s

(∫ τ

0

(LXD

)(Xs) ds

). (13.2.65)

Hence we see that the problem (13.2.64) reduces to the problem (13.1.4) withx → c(x) replaced by x → (LXD)(x) whenever non-negative. The conditionsassumed above to make such a transfer possible are not restrictive in general (seeSection 19 below).

Notes. Our main aim in this section (following [159]) is to present the solutionto a problem of optimal stopping for the maximum process associated with a one-dimensional time-homogeneous diffusion. The solution found has a large numberof applications, and may be viewed as the cornerstone in a general treatment ofthe maximum process.

In the setting of (13.1.1)–(13.1.3) we consider the optimal stopping problem(13.1.4), where the supremum is taken over all stopping times τ satisfying (13.1.5),and the cost function c is positive and continuous. The main result of the sectionis presented in Theorem 13.2, where it is proved that this problem has a solution(the value function is finite and there is an optimal stopping strategy) if and onlyif the maximality principle holds, i.e. the first-order nonlinear differential equation(13.2.22) has a maximal admissible solution (see Figures IV.11 and IV.12). Themaximal admissible solution is proved to be an optimal stopping boundary, i.e.the stopping time (13.2.32) is optimal, and the value function is given explicitlyby (13.2.31). Moreover, this stopping time is shown to be pointwise the smallestpossible optimal stopping time. If there is no such maximal admissible solutionof (13.2.22), the value function is proved to be infinite and there is no optimal


stopping time. The examples given in Chapter V below are aimed to illustratesome applications of the result proved.

The optimal stopping problem (13.1.4) has been considered in some specialcases earlier. Jacka [103] treats the case of reflected Brownian motion, while Du-bins, Shepp and Shiryaev [45] treat the case of Bessel processes. In these papersthe problem was solved effectively by guessing the nature of the optimal stoppingboundary and making use of the principle of smooth fit. The same is true for the“discounted” problem (13.2.61) with c ≡ 0 in the case of geometric Brownian mo-tion which in the framework of option pricing theory (Russian option) was solvedby Shepp and Shiryaev in [185] (see also [186] and [79]). For the first time a strongneed for additional arguments was felt in [81], where the problem (13.1.4) for ge-ometric Brownian motion was considered with the cost function c(x) ≡ c > 0 .There, by use of Picard’s method of successive approximations, it was proved thatthe maximal admissible solution of (13.2.22) is an optimal stopping boundary,and since this solution could not be expressed in closed form, it really showedthe full power of the method. Such nontrivial solutions were also obtained in [45]by a method which relies on estimates of the value function obtained a priori.Motivated by similar ideas, sufficient conditions for the maximality principle tohold for general diffusions are given in [82]. The method of proof used there relieson a transfinite induction argument. In order to solve the problem in general, thefundamental question was how to relate the maximality principle to the superhar-monic characterization of the value function, which is the key result in the generaltheory (recall Theorems 2.4 and 2.7 above).

The most interesting point in our solution of the optimal stopping problem(13.1.4) relies on the fact that we have described this connection, and actuallyproved that the maximality principle is equivalent to the superharmonic char-acterization of the value function (for a three-dimensional process). The crucialobservations in this direction are (13.2.29) and (13.2.30), which show that theonly possible optimal stopping boundary is the maximal admissible solution (see(13.2.39) in the proof of Theorem 13.2). In the next step of proving that the max-imal solution is indeed an optimal stopping boundary, it was crucial to make useof so-called “bad-good” solutions of (13.2.22), “bad” in the sense that they hit thediagonal in R2 , and “good” in the sense that they are not too large (see FiguresIV.11 and IV.12). These “bad-good” solutions are used to approximate the max-imal solution in a desired manner, see the proof of Theorem 13.2 (starting from(13.2.41) onwards), and this turns out to be the key argument in completing theproof.

Our methodology adopts and extends earlier results of Dubins, Shepp andShiryaev [45], and is, in fact, quite standard in the business of solving particularoptimal stopping problems: (i) one tries to guess the nature of the optimal stop-ping boundary as a member of a “reasonable” family; (ii) computes the expectedreward; (iii) maximizes this over the family; (iv) and then tries to argue that theresulting stopping time is optimal in general. This process is often facilitated by“ad hoc” principles, as the “principle of smooth fit” for instance. This procedure

Section 14. Nonlinear integral equations 219

is used effectively in this section too, as opposed to results from the general theoryof optimal stopping (Chapter I). It should be clear, however, that the maximalityprinciple of the present section should rather be seen as a convenient reformula-tion of the basic principle on a superharmonic characterization from the generaltheory, than a new principle on its own (see also [154] for a related result).

For results on discounted problems see [151] and for similar optimal stoppingproblems of Poisson processes see [119].

14. Nonlinear integral equations

This section is devoted to nonlinear integral equations which play a prominent rolein problems of optimal stopping (Subsection 14.1) and the first passage problem(Subsection 14.2). The two avenues are by no means independent and the purposeof this section is to highlight this fact without drawing parallels explicitly.

14.1. The free-boundary equation

In this subsection we will briefly indicate how the local time-space calculus (cf.Subsection 3.5) naturally leads to nonlinear integral equations which characterizethe optimal stopping boundary within an admissible class of functions.

For simplicity of exposition, let us assume that X is a one-dimensionaldiffusion process solving




Et,xG(t+τ, Xt+τ ) (14.1.2)

where Xt = x under Pt,x and τ is a stopping time of X . Assuming furtherthat G is smooth we know (cf. (8.2.2)–(8.2.4)) that (14.1.2) leads to the followingfree-boundary problem:

Vt + LXV = 0 in C, (14.1.3)V = G in D, (14.1.4)Vx = Gx at ∂C (14.1.5)

where C = V > G is the continuation set, D = V = G is the stopping set,and the stopping time τD = inf s ∈ [0, T −t] : (t+s, Xt+s) ∈ D is optimal in(14.1.2) under Pt,x .

For simplicity of exposition, let us further assume that

C =

(t, x) ∈ [0, T ]× R : x > b(t), (14.1.6)

D =

(t, x) ∈ [0, T ]× R : x ≤ b(t)

(14.1.7)


where b : [0, T ] → R is a continuous function of bounded variation. Then bis the optimal stopping boundary and the problem reduces to determining Vand b . Thus (14.1.3)–(14.1.5) may be viewed as a system of equations for the twounknowns V and b .

Generally, if one is given to solve a system of two equations with two un-knowns, a natural approach is to use the first equation in order to express thefirst unknown in terms of the second unknown, insert the resulting expression inthe second equation, and consider the resulting equation in order to determinethe second unknown. Quite similarly, this methodology extends to the system(14.1.3)–(14.1.5) as follows.

Assuming that sufficient conditions stated in Subsection 3.5 are satisfied, letus apply the change-of-variable formula (3.5.9) to V (t+s, Xt+s) under Pt,x . Thisyields:

V (t+s, Xt+s) = V (t, x) (14.1.8)

+∫ s

0

(Vt+LXV )(t+u, Xt+u) I(Xt+u = b(t+u)

)du

+∫ s

0

Vx(t+u, Xt+u)σ(Xt+u) I(Xt+u = b(t+u)

)dBt+u

+∫ s

0

(Vx(s, Xs+) − Vx(s, Xs−)

)I(Xt+u = b(t+u)

)db

t+u(X)

for s ∈ [0, T−t] where LXV = ρVx + (σ2/2)Vxx . Due to the smooth-fit condition(14.1.5) we see that the final integral in (14.1.8) must be zero. Moreover, sinceVt + LXV = 0 in C by (14.1.3), and Vt + LXV = Gt + LXG in D by (14.1.4),we see that (14.1.8) reads as follows:

V (t+s, Xt+s) = V (t, x) (14.1.9)

+∫ s

0

(Gt + LXG)(t + u, Xt+u) I(Xt+u < b(t + u)

)du + Ms

where Ms :=∫ s

0Vx(t+u, Xt+u)σ(Xt+u) I

((Xt+u = b(t+u)

)dBt+u is a continuous

(local) martingale for s ∈ [0, T − t] . The identity (14.1.9) may be viewed as anexplicit semimartingale decomposition of the value function composed with theprocess ( i.e. V (t+s, Xt+s) for s ∈ [0, T−t] ) under Pt,x .

Setting s = T − t in (14.1.9), taking Et,x on both sides, and using thatEt,x(MT−t) = 0 (whenever fulfilled), we get

Et,xG(T, XT ) = V (t, x) (14.1.10)

+∫ T−t

0

Et,x

[(Gt + LXG)(t+u, Xt+u) I

(Xt+u < b(t+u)

)]du

for all t ∈ [0, T ] and all x ∈ R . When x > b(t) then (14.1.10) is an equationcontaining both unknowns b and V . On the other hand, when x ≤ b(t) then


V (t, x) = G(t, x) is a known value so that (14.1.10) is an equation for b only. Inparticular, if we insert x = b(t) in (14.1.10) and use (14.1.4), we see that (14.1.10)becomes

Et,b(t)G(T, XT ) = G(t, b(t)) (14.1.11)

+∫ T−t

0

Et,x

[(Gt + LXG)(t+u, Xt+u) I

(Xt+u < b(t+u)

)]du

for t ∈ [0, T ] . This is a nonlinear integral equation for b that we call the free-boundary equation.

We will study specific examples of the free-boundary equation (14.1.11) inChapters VI–VIII below. It will be shown there that this equation characterizesthe optimal stopping boundary within an admissible class of functions. This factis far from being obvious at first glance and its establishment has led to thedevelopment of the local time-space calculus reviewed briefly in Subsection 3.5.On closer inspection it is instructive to note that the structure of the free-boundaryequation (14.1.11) is rather similar to the structure of the first-passage equationtreated in the following section.

14.2. The first-passage equation

1. Let B = (Bt)t≥0 be a standard Brownian motion started at zero, let g :(0,∞) → R be a continuous function satisfying g(0+) ≥ 0 , let

τ = inf t>0 : Bt≥g(t) (14.2.1)

be the first-passage time of B over g , and let F denote the distribution functionof τ .

The first-passage problem seeks to determine F when g is given. The inversefirst-passage problem seeks to determine g when F is given. Both the process Band the boundary g in these formulations may be more general, and our choice ofBrownian motion is primarily motivated by the tractability of the exposition. Thefacts to be presented below can be extended to more general Markov processesand boundaries (such as two-sided ones) and the time may also be discrete.

2. Chapman–Kolmogorov equations of Volterra type. It will be convenientto divide our discussion into two parts depending on if the time set T of theMarkov process X = (Xt)t∈T is either discrete (finite or countable) or continuous(uncountable). The state space E of the process may be assumed to be a subsetof R .

1. Discrete time and space. Recall that (Xn)n≥0 is a (time-homogeneous)Markov process if the following condition is satisfied:

Ex(H θk | Fk) = EXk(H) (14.2.2)


for all (bounded) measurable H and all k and x . (Recall that X0 = x underPx , and that Xn θk = Xn+k .)

Then the Chapman–Kolmogorov equation (see (4.1.20)) holds:

Px(Xn = z) =∑y∈E

Py(Xn−k = z) Px(Xk = y) (14.2.3)

for x , z in E and 1 < k < n given and fixed, which is seen as follows:

Px(Xn = z) =∑y∈E

Px(Xn = z, Xk = y) (14.2.4)

=∑y∈E

Ex

(I(Xk = y)Ex

(I(Xn−k = z) θk | Fk

))=∑y∈E

Ex

(I(Xk = y)EXk

I(Xn−k = z))

=∑y∈E

Px(Xk = y)Py(Xn−k = z)

upon using (14.2.2) with Y = I(Xn−k = z) .

A geometric interpretation of the Chapman–Kolmogorov equation (14.2.3)is illustrated in Figure IV.13 (note that the vertical line passing through k isgiven and fixed). Although for (14.2.3) we only considered the time-homogeneousMarkov property (14.2.2) for simplicity, it should be noted that a more generalMarkov process creates essentially the same picture.

Imagine now on Figure IV.13 that the vertical line passing through k beginsto move continuously and eventually transforms into a new curve still separatingx from z as shown in Figure IV.14. The question then arises naturally how theChapman–Kolmogorov equation (14.2.3) extends to this case.

An evident answer to this question is stated in the following Theorem 14.1.This fact is then extended to the case of continuous time and space in Theorem14.2 below.

Theorem 14.1. Let X = (Xn)n≥0 be a Markov process (taking values in a count-able set E ), let x and z be given and fixed in E , let g : N → E be a functionseparating x and z relative to X (i.e. if X0 = x and Xn = z for some n ≥ 1,then there exists 1 ≤ k ≤ n such that Xk = g(k) ), and let

τ = inf k≥1 : Xk = g(k) (14.2.5)

be the first-passage time of X over g . Then the following sum equation holds :

Px(Xn = z) =n∑

k=1

P(Xn = z |Xk = g(k)

)Px(τ = k). (14.2.6)


x

z

k n

g

Figure IV.13: A symbolic drawing of the Chapman–Kolmogorov equation(14.2.3). The arrows indicate a time evolution of the sample paths of theprocess. The vertical line at k represents the state space of the process.The equations (14.2.12) have a similar interpretation.

Moreover, if the Markov process X is time-homogeneous, then (14.2.6) readsas follows :

Px(Xn = z) =n∑

k=1

Pg(k)(Xn−k = z)Px(τ = k). (14.2.7)

Proof. Since g separates x and z relative to X , we have

Px(Xn = z) =n∑

k=1

Px(Xn = z, τ = k). (14.2.8)

On the other hand, by the Markov property:

Px(Xn = z | Fk) = PXk(Xn = z) (14.2.9)

and the fact that τ = k ∈ Fk , we easily find

Px(Xn = z, τ = k) = P(Xn = z |Xk = g(k)

)Px(τ = k). (14.2.10)

Inserting this into (14.2.8) we obtain (14.2.6). The time-homogeneous simplifica-tion (14.2.7) follows then immediately, and the proof is complete.


x

z

g

k n

Figure IV.14: A symbolic drawing of the integral equation (14.2.6)–(14.2.7). The arrows indicate a time evolution of the sample paths ofthe process. The vertical line at k has been transformed into a time-dependent boundary g . The equations (14.2.17)–(14.2.18) have a similarinterpretation.

The equations (14.2.6) and (14.2.7) extend to the case when the state spaceS is uncountable. In this case the relation “ = z ” in (14.2.6) and (14.2.7) canbe replaced by “∈ G ” where G is any measurable set that is “separated” fromthe initial point x relative to X in the sense described above. The extensions of(14.2.6) and (14.2.7) obtained in this way will be omitted.

2. Continuous time and space. A passage from the discrete to the continuouscase introduces some technical complications (e.g. regular conditional probabilitiesare needed) which we set aside in the sequel (see Subsection 4.3).

A process (Xt)t≥0 is called a Markov process (in a wide sense) if the followingcondition is satisfied:

P(Xt∈G | Fs) = P(Xt∈G |Xs) (14.2.11)

for all measurable G and all s < t (recall (4.1.2)). Then the Chapman–Kolmo-gorov equation (see (4.3.2)) holds:

P (s, x; t, A) =∫

E

P (s, x; u, dy)P (u, y; t, A) (0 ≤ s < u < t) (14.2.12)

where P (s, x; t, A) = P(Xt ∈ A |Xs = x) and s < u < t are given and fixed.

Kolmogorov [111] called (14.2.12) ‘the fundamental equation’, noted that(under a desired Markovian interpretation) it is satisfied if the state space E


is finite or countable (the ‘total probability law’), and in the case when E isuncountable took it as a “new axiom”.

If Xt under Xs = x has a density function f satisfying

P (s, x; t, A) =∫

A

f(s, x; t, y) dy (14.2.13)

for all measurable sets A , then the equations (14.2.12) reduce to

f(s, x; t, z) =∫

E

f(s, x; u, dy) f(u, y; t, z) dy (14.2.14)

for x and z in E and s < u < t given and fixed.

In [111] Kolmogorov proved that under some additional conditions f satis-fies certain differential equations of parabolic type (the forward and the backwardequation — see (4.3.7) and (4.3.8)). Note that in [112] Kolmogorov mentionedthat this integral equation was studied by Smoluchowski [204], and in a footnotehe acknowledged that these differential equations for certain particular cases wereintroduced by Fokker [68] and Planck [172] independently of the Smoluchowski in-tegral equation. (The Smoluchowski integral equation [204] is a time-homogeneousversion of (14.2.14). The Bachelier–Einstein equation (cf. [7], [56])

f(t+s, z) =∫

E

f(t, z−x) f(s, x) dx (14.2.15)

is a space-time homogeneous version of the Smoluchowski equation.)

Without going into further details on these facts, we will only note that theinterpretation of the Chapman–Kolmogorov equation (14.2.3) described above bymeans of Figure IV.13 carries over to the general case of the equation (14.2.12),and the same is true for the question raised above by means of Figure IV.14. Thefollowing theorem extends the result of Theorem 14.1 on this matter.

Theorem 14.2. (cf. Schrodinger [182] and Fortet [69, p. 217]) Let X = (Xt)t≥0

be a strong Markov process with continuous sample paths started at x , let g :(0,∞) → R be a continuous function satisfying g(0+) ≥ x , let

τ = inf t> 0 : Xt ≥ g(t) (14.2.16)

be the first-passage time of X over g , and let F denote the distribution functionof τ .

Then the following integral equation holds :

Px(Xt ∈ G) =∫ t

0

P(Xt ∈ G |Xs = g(s)

)F (ds) (14.2.17)

for each measurable set G contained in [ g(t),∞) .


Moreover, if the Markov process X is time-homogeneous, then (14.2.17)reads as follows :

Px(Xt ∈ G) =∫ t

0

Pg(s)

(Xt−s ∈ G

)F (ds). (14.2.18)

for each measurable set G contained in [ g(t),∞) .

Proof. The key argument in the proof is to apply a strong Markov property at timeτ (see (4.3.27) and (4.3.28)). This can be done informally (with G ⊆ [ g(t),∞)given and fixed) as follows:

Px(Xt ∈ G) = Px(Xt ∈ G, τ ≤ t) = Ex

(I (τ ≤ t)Ex

(I (Xt ∈ G) | τ)) (14.2.19)

=∫ t

0

Ex

(I(Xt ∈ G) | τ = s

)F (ds)

=∫ t

0

P(Xt ∈ G |Xs = g(s)

)F (ds)

which is (14.2.17). In the last identity above we used that for s ≤ t we have

Ex

(I(Xt ∈ G) | τ = s

)= P

(Xt ∈ G |Xs = g(s)

)(14.2.20)

which formally requires a precise argument. This is what we do in the rest of theproof.

For this, recall that if Z = (Zt)t≥0 is a strong Markov process then

Ez(H θσ | Fσ) = EZσ(H) (14.2.21)

for all (bounded) measurable H and all stopping times σ .

For our proof we choose Zt = (t, Xt) and define

σ = inf t > 0 : Zt /∈ C , (14.2.22)β = inf t > 0 : Zt /∈ C ∪ D (14.2.23)

where C = (s, y) : 0 < s < t, y < g(s) and D = (s, y) : 0 < s < t, y ≥ g(s) ,so that C ∪ D = (s, y) : 0 < s < t . Thus β = t under P(0,x) i.e. Px , andmoreover β = σ+βθσ since both σ and β are hitting times of the process Z toclosed (open) sets, the second set being contained in the first one, so that σ ≤ β .(See (7.0.7) and (4.1.25) above.)

Setting F (s, y) = 1G(y) and H = F (Zβ) , we thus see that H θσ =F (Zβ) θσ = F (Zσ+βσ) = F (Zβ) = H , which by means of (14.2.21) implies that

Ez(F (Zβ) | Fσ) = EZσF (Zβ). (14.2.24)


In the special case z = (0, x) this reads

E(0,x)

(I (Xt ∈ G) | Fσ

)= E(σ,g(σ))I (Xt ∈ G) (14.2.25)

where Fσ on the left-hand side can be replaced by σ since the right-hand sidedefines a measurable function of σ . It follows then immediately from such modified(14.2.25) that

E(0,x)

(I(Xt ∈ G) | σ = s

)= E(s,g(s))

(I(Xt ∈ G)

)(14.2.26)

and since σ = τ∧t we see that (14.2.26) implies (14.2.20) for s ≤ t . Thus the finalstep in (14.2.19) is justified and therefore (14.2.17) is proved as well. The time-homogeneous simplification (14.2.18) is a direct consequence of (14.2.17), and theproof of the theorem is complete.

The proof of Theorem 14.2 just presented is not the only possible one. Theproof of Theorem 14.3 given below can easily be transformed into a proof ofTheorem 14.2. Yet another quick proof can be given by applying the strong Markovproperty of the process (t, Xt) to establish (14.2.25) (multiplied by I(τ ≤ t) onboth sides) with σ = τ ∧ t on the left-hand side and σ = τ on the right-handside. The right-hand side then easily transforms to the right-hand side of (14.2.17)thus proving the latter.

In order to examine the scope of the equations (14.2.17) in a clearer manner,we will leave the realm of a general Markov process in the sequel, and considerthe case of a standard Brownian motion instead. The facts and methodology pre-sented below extend to the case of more general Markov processes (or boundaries)although some of the formulae may be less explicit.

3. The master equation. The following notation will be used throughout:

ϕ(x) =1√2π

e−x2/2, Φ(x) =∫ x

−∞ϕ(z) dz, Ψ(x) = 1−Φ(x) (14.2.27)

for x ∈ R . We begin this paragraph by recalling the result of Theorem 14.2. Thus,let g : (0,∞) → R be a continuous function satisfying g(0+) ≥ 0 , and let Fdenote the distribution function of τ from (14.2.16).

If specialized to the case of standard Brownian motion (Bt)t≥0 started atzero, the equation (14.2.18) with G = [ g(t),∞) reads as follows:

Ψ(

g(t)√t

)=∫ t

0

Ψ(

g(t)− g(s)√t− s

)F (ds) (14.2.28)

where the scaling property Bt ∼√

tB1 of B is used, as well as that (z+Bt)t≥0

defines a standard Brownian motion started at z whenever z ∈ R .1. Derivation. It turns out that the equation (14.2.28) is just one in the

sequence of equations that can be derived from a single master equation from


Theorem 14.3 below. This master equation can be obtained by taking G = [z,∞)in (14.2.18) with z ≥ g(t) . We now present yet another proof of this derivation.

Theorem 14.3. (The Master Equation) Let B = (Bt)t≥0 be a standard Brownianmotion started at zero, let g : 〈0,∞〉 → R be a continuous function satisfyingg(0+) ≥ 0 , let

τ = inf t > 0 : Bt ≥ g(t) (14.2.29)

be the first-passage time of B over g , and let F denote the distribution functionof τ .

Then the following integral equation (called the Master Equation) holds :

Ψ(

z√t

)=∫ t

0

Ψ(

z− g(s)√t− s

)F (ds) (14.2.30)

for all z ≥ g(t) where t > 0 .

Proof. We will make use of the strong Markov property of the process Zt = (t, Bt)at time τ . This makes the present argument close to the argument used in theproof of Theorem 14.2.

For each t > 0 let z(t) from [ g(t),∞) be given and fixed. Setting f(t, x) =I(x ≥ z(t)) and H =

∫∞0

e−λsf(Zs) ds by the strong Markov property (of theprocess Z ) given in (14.2.21) with σ = τ , and the scaling property of B , wefind: ∫ ∞

0

e−λtP0

(Bt ≥ z(t)

)dt = E0

(∫ ∞

0

e−λtf(Zt) dt

)(14.2.31)

= E0

(E0

(∫ ∞

τ

e−λtf(Zt) dt∣∣∣Fτ

))= E0

(E0

(∫ ∞

0

e−λ(τ+s)f(Zτ+s) ds∣∣∣Fτ

))= E0

(e−λτE0(H θτ | Fτ )

)= E0(e−λτEZτ H)

=∫ ∞

0

e−λtE(t,g(t))

(∫ ∞

0

e−λsf(Zs) ds

)F (dt)

=∫ ∞

0

e−λt

∫ ∞

0

e−λs P0

(g(t)+Bs ≥ z(t+s)

)ds F (dt)

=∫ ∞

0

e−λt

∫ ∞

0

e−λs Ψ(

z(t+s)− g(t)√s

)ds F (dt)

=∫ ∞

0

e−λt

∫ ∞

t

e−λ(r−t) Ψ(

z(r)− g(t)√r− t

)dr F (dt)

=∫ ∞

0

e−λr

∫ r

0

Ψ(

z(r)− g(t)√r− t

)F (dt) dr


for all λ > 0 . By the uniqueness theorem for Laplace transform it follows that

P0

(Bt ≥ z(t)

)=∫ t

0

Ψ(

z(t)− g(s)√t− s

)F (ds) (14.2.32)

which is seen equivalent to (14.2.30) by the scaling property of B . The proof iscomplete.

2. Constant and linear boundaries. It will be shown in Theorem 14.4 belowthat when g is C1 on (0,∞) then there exists a continuous density f = F ′ ofτ . The equation (14.2.28) then becomes

Ψ(

g(t)√t

)=∫ t

0

Ψ(


)f(s) ds (14.2.33)

for t > 0 . This is a linear Volterra integral equation of the first kind in f if g isknown (it is a nonlinear equation in g if f is known). Its kernel

K(t, s) = Ψ(


)(14.2.34)

is nonsingular in the sense that the mapping (s, t) → K(t, s) for 0 ≤ s < t isbounded.

If g(t) ≡ c with c ∈ R , then (14.2.28) or (14.2.33) reads as follows:

P(τ ≤ t) = 2 P(Bt ≥ c) (14.2.35)

and this is the reflection principle (see (4.4.19)).

If g(t) = a + bt with b ∈ R and a > 0 , then (14.2.33) reads as follows:

Ψ(

g(t)√t

)=∫ t

0

Ψ(b√

t− s)f(s) ds (14.2.36)

where we see that the kernel K(t, s) is a function of the difference t − s andthus of a convolution type. Standard Laplace transform techniques therefore canbe applied to solve the equation (14.2.36) yielding the following explicit formula:

f(t) =a

t3/2ϕ

(a + bt√

t

)(14.2.37)

(see (4.4.31)).

The case of more general boundaries g will be treated using classic theoryof integral equations in Theorem 14.7 below.

3. Numerical calculation. The fact that the kernel (14.2.34) of the equation(14.2.33) is nonsingular in the sense explained above makes this equation especially


attractive to numerical calculations of f if g is given. This can be done using thesimple idea of Volterra (dating back to 1896).

Setting tj = jh for j = 0, 1, . . . , n where h = t/n and n ≥ 1 is given andfixed, we see that the following approximation of the equation (14.2.33) is valid(when g is C1 for instance):

n∑j=1

K(t, tj) f(tj)h = b(t) (14.2.38)

where we set b(t) = Ψ(g(t)/√

t) . In particular, applying this to each t = ti yields

i∑j=1

K(ti, tj) f(tj)h = b(ti) (14.2.39)

for i = 1, . . . , n . Setting

aij = 2K(ti, tj), xj = f(tj), bi = 2b(ti)/h (14.2.40)

we see that the system (14.2.39) reads as follows:

i∑j=1

aijxj = bi (i = 1, . . . , n) (14.2.41)

the simplicity of which is obvious (cf. [149]).

4. Remarks. It follows from (14.2.37) that for τ in (14.2.29) with g(t) = a+btwe have

P(τ < ∞) = e−2αβ (14.2.42)

whenever b ≥ 0 and a > 0 . This shows that F in (14.2.30) does not have to bea proper distribution function but generally satisfies F (+∞) ∈ (0, 1] .

On the other hand, recall that Blumenthal’s 0–1 law implies that P(τ = 0)is either 0 or 1 for τ in (14.2.29) and a continuous function g : (0,∞) → R . IfP(τ = 0) = 0 then g is said to be an upper function for B , and if P(τ = 0) = 1then g is said to be a lower function for B . Kolmogorov’s test (see e.g. [100,pp. 33–35]) gives sufficient conditions on g to be an upper or lower function. Itfollows by Kolmogorov’s test that

√2 t log log 1/t is a lower function for B , and√

(2+ε) t log log 1/t is an upper function for B for every ε > 0 .

4. The existence of a continuous first-passage density. The equation (14.2.33)is a Volterra integral equation of the first kind. These equations are generallyknown to be difficult to deal with directly, and there are two standard ways ofreducing them to Volterra integral equations of the second kind. The first methodconsists of differentiating both sides in (14.2.33) with respect to t , and the second


method (Theorem 14.7) makes use of an integration by parts in (14.2.33) (see e.g.[92, pp. 40–41]). Our focus in this paragraph is on the first method.

Being led by this objective we now present a simple proof of the fact that Fis C1 when g is C1 (compare the arguments given below with those given in[207, p. 323] or [65, p. 322]).

Theorem 14.4. Let B = (Bt)t≥0 be a standard Brownian motion started at zero,let g : (0,∞) → R be an upper function for B , and let τ in (14.2.29) be thefirst-passage time of B over g .

If g is continuously differentiable on (0,∞) then τ has a continuous den-sity f . Moreover, the following identity is satisfied:

∂

∂tΨ(

g(t)√t

)=

12

f(t) +∫ t

0

∂

∂tΨ(


)f(s) ds (14.2.43)

for all t > 0 .

Proof. 1. Setting G(t) = Ψ(g(t)/√

t) and K(t, s) = Ψ((g(t)− g(s))/√

t− s) for0 ≤ s < t we see that (14.2.28) (i.e. (14.2.30) with z = g(t) ) reads as follows:

G(t) =∫ t

0

K(t, s)F (ds) (14.2.44)

for all t > 0 . Note that K(t, t−) = ψ(0) = 1/2 for every t > 0 since(g(t)− g(s))/

√t− s → 0 as s ↑ t for g that is C1 on (0,∞) . Note also that

∂

∂tK(t, s) =

1√t− s

(12

g(t)− g(s)t− s

− g′(t))

ϕ

(g(t)− g(s)√

t− s

)(14.2.45)

for 0 < s < t . Hence we see that (∂K/∂t)(t, t−) is not finite (whenever g′(t) =0 ), and we thus proceed as follows.

2. Using (14.2.44) we find by Fubini’s theorem that

limε↓0

∫ t2

t1

(∫ t−ε

0

∂

∂tK(t, s)F (ds)

)dt (14.2.46)

= limε↓0

(∫ t2−ε

0

K(t2, s)F (ds)

−∫ t1−ε

0

K(t1, s)F (ds) −∫ t2−ε

t1−ε

K(s+ε, s)F (ds))

= G(t2)−G(t1) − 12(F (t2)−F (t1)

)for 0 < t1 ≤ t ≤ t2 < ∞ . On the other hand, we see from (14.2.45) that∣∣∣ ∫ t−ε

0

∂

∂tK(t, s)F (ds)

∣∣∣ ≤ C

∫ t

0

F (ds)√t− s

(14.2.47)


for all t ∈ [t1, t2] and ε > 0 , while again by Fubini’s theorem it is easily verifiedthat ∫ t2

t1

(∫ t

0

F (ds)√t− s

)dt < ∞. (14.2.48)

We may thus by the dominated convergence theorem (applied twice) interchangethe first limit and the first integral in (14.2.46) yielding∫ t2

t1

(∫ t

0

∂

∂tK(t, s)F (ds)

)dt = G(t2) − G(t1) − 1

2(F (t2)−F (t1)

)(14.2.49)

at least for those t ∈ [t1, t2] for which∫ t

0

F (ds)√t− s

< ∞. (14.2.50)

It follows from (14.2.48) that the set of all t > 0 for which (14.2.50) fails is ofLebesgue measure zero.

3. To verify (14.2.50) for all t > 0 we may note that a standard rule on thedifferentiation under an integral sign can be applied in (14.2.30), and this yieldsthe following equation:

1√tϕ

(z√t

)=∫ t

0

1√t− s

ϕ

(z− g(s)√

t− s

)F (ds) (14.2.51)

for all z > g(t) with t > 0 upon differentiating in (14.2.30) with respect to z .By Fatou’s lemma hence we get∫ t

0

1√t− s

ϕ

(g(t)− g(s)√

t− s

)F (ds) (14.2.52)

=∫ t

0

lim infz↓g(t)

1√t− s

ϕ

(z− g(s)√

t− s

)F (ds)

≤ lim infz↓g(t)

∫ t

0

1√t− s

ϕ

(z− g(s)√

t− s

)F (ds) =

1√tϕ

(g(t)√

t

)< ∞

for all t > 0 . Now for s < t close to t we know that ϕ((g(t)− g(s))/√

t− s) in(14.2.52) is close to 1/

√2π > 0 , and this easily establishes (14.2.50) for all t > 0 .

4. Returning to (14.2.49) it is easily seen using (14.2.45) that t →∫ t

0(∂K/∂t) (t, s)F (ds) is right-continuous at t ∈ (t1, t2) if we have∫ tn

t

F (ds)√tn − s

→ 0 (14.2.53)


for tn ↓ t as n → ∞ . To check (14.2.53) we first note that by passing to thelimit for z ↓ g(t) in (14.2.51), and using (14.2.50) with the dominated conver-gence theorem, we obtain (14.2.56) below for all t > 0 . Noting that (s, t) →ϕ((g(t)− g(s))/

√t− s) attains its strictly positive minimum c > 0 over 0 < t1 ≤

t ≤ t2 and 0 ≤ s < t , we may write∫ tn

t

F (ds)√tn − s

≤ 1c

∫ tn

t

1√tn − s

ϕ

(g(tn)− g(s)√

tn − s

)F (ds) (14.2.54)

=1c

(1√tn

ϕ

(g(tn)√

tn

)−∫ t

0

1√tn − s

ϕ

(g(tn)− g(s)√

tn − s

)F (ds)

)where the final expression tends to zero as n → ∞ by means of (14.2.56) be-low and using (14.2.50) with the dominated convergence theorem. Thus (14.2.53)holds and therefore t → ∫ t

0 (∂K/∂t)(t, s)F (ds) is right-continuous. It can be sim-ilarly verified that this mapping is left-continuous at each t ∈ (t1, t2) and thuscontinuous on (0,∞) .

5. Dividing finally by t2 − t1 in (14.2.49) and then letting t2 − t1 → 0 , weobtain

F ′(t) = 2(

G′(t) −∫ t

0

∂

∂tK(t, s)F (ds)

)(14.2.55)

for all t > 0 . Since the right-hand side of (14.2.55) defines a continuous functionof t > 0 , it follows that f = F ′ is continuous on (0,∞) , and the proof iscomplete.

5. Derivation of known equations. In the previous proof we saw that themaster equation (14.2.30) can be once differentiated with respect to z implyingthe equation (14.2.51), and that in (14.2.51) one can pass to the limit for z ↓ g(t)obtaining the following equation:

1√tϕ

(g(t)√

t

)=∫ t

0

1√t− s

ϕ

(g(t)− g(s)√

t− s

)F (ds) (14.2.56)

for all t > 0 .

The purpose of this paragraph is to show how the equations (14.2.43) and(14.2.56) yield some known equations studied previously by a number of authors.

1. We assume throughout that the hypotheses of Theorem 14.4 are ful-filled (and that t > 0 is given and fixed). Rewriting (14.2.43) more explicitly bycomputing derivatives on both sides gives(

12

g(t)t3/2

− g′(t)√t

)ϕ

(g(t)√

t

)=

12

f(t) (14.2.57)

+∫ t

0

(12

g(t)− g(s)(t− s)3/2

− g′(t)√t− s

)ϕ

(g(t)− g(s)√

t− s

)f(s) ds.


Recognizing now the identity (14.2.56) multiplied by g′(t) within (14.2.57), andmultiplying the remaining part of the identity (14.2.57) by 2 , we get

g(t)t3/2

ϕ

(g(t)√

t

)= f(t) +

∫ t

0

g(t)− g(s)(t− s)3/2

ϕ

(g(t)− g(s)√

t− s

)f(s) ds. (14.2.58)

This equation has been derived and studied by Ricciardi et al. [175] using othermeans. Moreover, the same argument shows that the factor 1/2 can be removedfrom (14.2.57) yielding(

g(t)t3/2

− g′(t)√t

)ϕ

(g(t)√

t

)= f(t) (14.2.59)

+∫ t

0

(g(t)− g(s)(t− s)3/2

− g′(t)√t− s

)ϕ

(g(t)− g(s)√

t− s

)f(s) ds.

This equation has been derived independently by Ferebee [65] and Durbin [48].Ferebee’s derivation is, set aside technical points, the same as the one presentedhere. Williams [49] presents yet another derivation of this equation (assuming thatf exists). [Multiplying both sides of (14.2.33) by 2r(t) and both sides of (14.2.56)by 2(k(t)+g′(t)) , and adding the resulting two equations to the equation (14.2.58),we obtain the equation (14.2.10)+(14.2.30) in Buonocore et al. [24] derived byother means.]

2. With a view to the inverse problem (of finding g if f is given) itis of interest to produce as many nonequivalent equations linking g to f aspossible. (Recall that (14.2.33) is a nonlinear equation in g if f is known, andnonlinear equations are marked by a nonuniqueness of solutions.) For this reasonit is tempting to derive additional equations to the one given in (14.2.56) startingwith the master equation (14.2.30) and proceeding similarly to (14.2.51) above.

A standard rule on the differentiation under an integral sign can be induc-tively applied to (14.2.30), and this gives the following equations:

1tn/2

ϕ(n−1)

(z√t

)=∫ t

0

1(t−s)n/2

ϕ(n−1)

(z− g(s)√

t− s

)F (ds) (14.2.60)

for all z > g(t) and all n ≥ 1 where t > 0 . Recall that

ϕ(n)(x) = (−1)nhn(x)ϕ(x) (14.2.61)

for x ∈ R and n ≥ 1 where hn is a Hermite polynomial of degree n for n ≥ 1 .

Noting that ϕ′(x) = −xϕ(x) and recalling (14.2.58) we see that a passageto the limit for z ↓ g(t) in (14.2.60) is not straightforward when n ≥ 2 butcomplicated. For this reason we will not pursue it in further detail here.

3. The Chapman–Kolmogorov equation (14.2.12) is known to admit a re-duction to the forward and backward equation (see [111] and Subsection 4.3) which


are partial differential equations of parabolic type. No such derivation or reductionis generally possible in the entire-past dependent case of the equation (14.2.17) or(14.2.18), and the same is true for the master equation (14.2.30) in particular. Weshowed above how the differentiation with respect to z in the master equation(14.2.30) leads to the density equation (14.2.56), which together with the distri-bution equation (14.2.28) yields known equations (14.2.58) and (14.2.59). It wasalso indicated above that no further derivative with respect to z can be taken inthe master equation (14.2.30) so that the passage to the limit for z ↓ g(t) in theresulting equation becomes straightforward.

6. Derivation of new equations. Expanding on the previous facts a bit furtherwe now note that it is possible to proceed in a reverse order and integrate themaster equation (14.2.30) with respect to z as many times as we please. Thisyields a whole spectrum of new nonequivalent equations, which taken togetherwith (14.2.28) and (14.2.56), may play a fundamental role in the inverse problem(see page 240).

Theorem 14.5. Let B = (Bt)t≥0 be a standard Brownian motion started at zero,let g : (0,∞) → R be a continuous function satisfying g(0+) ≥ 0 , let τ in(14.2.29) be the first-passage time of B over g , and let F denote the distributionfunction of τ .

Then the following system of integral equations is satisfied :

tn/2Hn

(g(t)√

t

)=∫ t

0

(t− s)n/2Hn

(g(t)− g(s)√

t− s

)F (ds) (14.2.62)

for t > 0 and n = −1, 0, 1, . . . , where we set

Hn(x) =∫ ∞

x

Hn−1(z) dz (14.2.63)

with H−1 = ϕ being the standard normal density from (14.2.27).

Remark 14.6. For n = −1 the equation (14.2.62) is the density equation (14.2.56).For n = 0 the equation (14.2.62) is the distribution equation (14.2.28). All equa-tions in (14.2.62) for n = −1 are nonsingular (in the sense that their kernels arebounded over the set of all (s, t) satisfying 0 ≤ s < t ≤ T ).

Proof. Let t > 0 be given and fixed. Integrating (14.2.30) we get∫ ∞

z

Ψ(

z′√t

)dz′ =

∫ t

0

∫ ∞

z

Ψ(

z′− g(s)√t− s

)dz′ F (ds) (14.2.64)

for all z ≥ g(t) by means of Fubini’s theorem. Substituting u = z′/√

t andv = (z′ − g(s))/

√t− s we can rewrite (14.2.64) as follows:

√t

∫ ∞

z/√

t

Ψ(u) du =∫ t

0

√t− s

∫ ∞

(z−g(s))/√

t−s

Ψ(v) dv F (ds) (14.2.65)


which is the same as the following identity:

√t H1

(z√t

)=∫ t

0

√t− s H1

(z− g(s)√

t− s

)F (ds) (14.2.66)

for all z ≥ g(t) upon using that H1 is defined by (14.2.63) above with n = 1 .

Integrating (14.2.66) as (14.2.30) prior to (14.2.64) above, and proceedingsimilarly by induction, we get

tn/2Hn

(z√t

)=∫ t

0

(t− s)n/2Hn

(z−g(s)√

t− s

)F (ds) (14.2.67)

for all z ≥ g(t) and all n ≥ 1 . (This equation was also established earlier forn = 0 in (14.2.30) and for n = −1 in (14.2.51).) Setting z = g(t) in (14.2.67)above we obtain (14.2.62) for all n ≥ 1 . (Using that Ψ(x) ≤ √

2/π ϕ(x) for allx > 0 it is easily verified by induction that all integrals appearing in (14.2.62)–(14.2.67) are finite.) As the equation (14.2.62) was also proved earlier for n = 0in (14.2.28) and for n = −1 in (14.2.56) above, we see that the system (14.2.62)holds for all n ≥ −1 , and the proof of the theorem is complete.

In view of our considerations in paragraph 5 above it is interesting to establishthe analogues of the equations (14.2.58) and (14.2.59) in the case of other equationsin (14.2.62).

For this, fix n ≥ 1 and t > 0 in the sequel, and note that taking a derivativewith respect to t in (14.2.62) gives

n

2tn/2−1Hn

(g(t)√

t

)+ tn/2H ′

n

(g(t)√

t

)(g′(t)√

t− g(t)

2t3/2

)(14.2.68)

=∫ t

0

(n

2(t− s)n/2−1Hn

(g(t)− g(s)√

t− s

)

+ (t− s)n/2H ′n

(g(t)− g(s)√

t− s

)(g′(t)√t− s

− g(t)− g(s)2(t− s)3/2

))F (ds).

Recognizing now the identity (14.2.62) (with n−1 instead of n using that H ′n =

Hn−1 ) multiplied by g′(t) within (14.2.68), and multiplying the remaining partof the identity (14.2.68) by 2 , we get

tn/2−1

(nHn

(g(t)√

t

)− g(t)√

tHn−1

(g(t)√

t

))(14.2.69)

=∫ t

0

(t− s)n/2−1

(nHn

(g(t)− g(s)√

t− s

)

− g(t)− g(s)√t− s

Hn−1

(g(t)− g(s)√

t− s

))F (ds).


Moreover, the same argument shows that the factor 1/2 can be removed from(14.2.68) yielding

tn/2

(n

tHn

(g(t)√

t

)−(

g(t)t3/2

− g′(t)√t

)Hn−1

(g(t)√

t

))(14.2.70)

=∫ t

0

(t− s)n/2

(n

(t− s)Hn

(g(t)− g(s)√

t− s

)

−(

g(t)− g(s)(t− s)3/2

− g′(t)√t− s

)Hn−1

(g(t)− g(s)√

t− s

))F (ds).

Each of the equations (14.2.69) and (14.2.70) is contained in the system (14.2.62).No equation of the system (14.2.62) is equivalent to another equation from thesame system but itself.

7. A closed expression for the first-passage distribution. In this paragraphwe briefly tackle the problem of finding F when g is given using classic theory oflinear integral equations (see e.g. [92]). The key tool in this approach is the fixed-point theorem for contractive mappings, which states that a mapping T : X → X ,where (X, d) is a complete metric space, satisfying

d(T (x), T (y)) ≤ β d(x, y) (14.2.71)

for all x, y ∈ X with some β ∈ (0, 1) has a unique fixed point in X , i.e. thereexists a unique point x0 ∈ X such that T (x0) = x0 .

Using this principle and some of its ramifications developed within the theoryof integral equations, the papers [148] and [175] present explicit expressions for Fin terms of g in the case when X is taken to be a Hilbert space L2 . These resultswill here be complemented by describing a narrow class of boundaries g that allowX to be the Banach space B(R+) of all bounded functions h : R+ → R equippedwith the sup-norm

‖h‖∞ = supt≥0

|h(t)|. (14.2.72)

While examples from this class range from a constant to a square-root boundary,the approach itself is marked by simplicity of the argument.

Theorem 14.7. Let B = (Bt)t≥0 be a standard Brownian motion started at zero,let g : R+ → R be a continuous function satisfying g(0) > 0 , let τ in (14.2.29)be the first-passage time of B over g , and let F denote the distribution functionof τ .

Assume, moreover, that g is C1 on (0,∞) , increasing, concave, and thatit satisfies

g(t) ≤ g(0) + c√

t (14.2.73)


for all t ≥ 0 with some c > 0 . Then we have

F (t) = h(t) +∞∑

n=1

(∫ t

0

Kn(t, s)h(s) ds

)(14.2.74)

where the series converges uniformly over all t ≥ 0 , and we set

h(t) = 2Ψ(

g(t)√t

), (14.2.75)

K1(t, s) =1√t− s

(2 g′(s) − g(t)− g(s)

t− s

)ϕ

(g(t)− g(s)√

t− s

), (14.2.76)

Kn+1(t, s) =∫ t

s

K1(t, r)Kn(r, s) dr (14.2.77)

for 0 ≤ s < t and n ≥ 1 .

Moreover, introducing the function

R(t, s) =∞∑

n=1

Kn(t, s) (14.2.78)

for 0 ≤ s < t , the following representation is valid :

F (t) = h(t) +∫ t

0

R(t, s)h(s) ds (14.2.79)

for all t > 0 .

Proof. Setting u = Ψ((g(t)− g(s))/

√t− s

)and v = F (s) in the integral equation

(14.2.28) and using the integration by parts formula, we obtain

Ψ(

g(t)√t

)=

12

F (t) −∫ t

0

∂

∂sΨ(


)F (s) ds (14.2.80)

for each t > 0 that is given and fixed in the sequel. Using the notation of (14.2.75)and (14.2.76) above we can rewrite (14.2.80) as follows:

F (t) −∫ t

0

K1(t, s)F (s) ds = h(t). (14.2.81)

Introduce a mapping T on B(R+) by setting

(T (G))(t) = h(t) +∫ t

0

K1(t, s)G(s) ds (14.2.82)

for G ∈ B(R+) . Then (14.2.81) reads as follows:

T (F ) = F (14.2.83)

and the problem reduces to solving (14.2.83) for F in B(R+) .


In view of the fixed-point theorem quoted above, we need to verify that Tis a contraction from B(R+) into itself with respect to the sup-norm (14.2.72).For this, note that

‖T (G1)−T (G2)‖∞ = supt≥0

|(T (G1 −G2))(t)| (14.2.84)

= supt≥0

∣∣∣ ∫ t

0

K1(t, s)(G1(s)−G2(s)

)ds∣∣∣

≤(

supt≥0

∫ t

0

|K1(t, s)| ds

)‖G1 −G2‖∞.

Since s → g(s) is concave and increasing, it is easily verified that s →(g(t)− g(s))/

√t− s is decreasing and thus s → Ψ

((g(t)− g(s))/

√t− s

)is in-

creasing on (0, t) . It implies that

β := supt≥0

∫ t

0

∣∣K1(t, s)∣∣ ds = sup

t≥0

∫ t

0

∣∣∣∣ 2 ∂

∂sΨ(


)∣∣∣∣ ds (14.2.85)

= supt≥0

∫ t

0

2∂

∂sΨ(


)ds

= supt≥0

2(

12− Ψ

(g(t)− g(0)√

t

))≤ 1 − 2Ψ(c) < 1

using the hypothesis (14.2.73). This shows that T is a contraction from the Banachspace B(R+) into itself, and thus by the fixed-point theorem there exists a uniqueF0 in B(R+) satisfying (14.2.83). Since the distribution function F of τ belongsto B(R+) and satisfies (14.2.83), it follows that F0 must be equal to F .

Moreover, the representation (14.2.74) follows from (14.2.81) and the well-known formula for the resolvent of the integral operator K = T − h associatedwith the kernel K1 : (

I −K)−1 =

∞∑n=0

Kn (14.2.86)

upon using Fubini’s theorem to justify that Kn+1 in (14.2.77) is the kernel of theintegral operator Kn+1 for n ≥ 1 . Likewise, the final claim about (14.2.78) and(14.2.79) follows by the Fubini–Tonelli theorem since all kernels in (14.2.76) and(14.2.77) are non-negative, and so are all maps s → Kn(t, s)h(s) in (14.2.74) aswell. This completes the proof.

Leaving aside the question on usefulness of the multiple-integral series rep-resentation (14.2.74), it is an interesting mathematical question to find a similarexpression for F in terms of g that would not require additional hypotheseson g such as (14.2.73) for instance. In this regard especially those g satisfyingg(0+) = 0 seem problematic as they lead to singular (or weakly singular) kernelsgenerating the integral operators that turn out to be noncontractive.


8. The inverse problem. In this paragraph we will reformulate the inverseproblem of finding g when F is given using the result of Theorem 14.5. Recallfrom there that g and F solve

tn/2Hn

(g(t)√

t

)=∫ t

0

(t− s)n/2Hn

(g(t)− g(s)√

t− s

)F (ds) (14.2.87)

for t > 0 and n ≥ −1 where Hn(x) =∫∞

xHn−1(z) dz with H−1 = ϕ . Then the

inverse problem reduces to answer the following three questions:

Question 8.1. Does there exist a (continuous) solution t → g(t) of thesystem (14.2.87)?

Question 8.2. Is this solution unique?

Question 8.3. Does the (unique) solution t → g(t) solve the inverse first-passage problem i.e. is the distribution function of τ from(14.2.29) equal to F?

It may be noted that each equation in g of the system (14.2.87) is a nonlinearVolterra integral equation of the second kind. Nonlinear equations are known tolead to nonunique solutions, so it is hoped that the totality of countably manyequations could counterbalance this deficiency.

Perhaps the main example one should have in mind is when F has a contin-uous density f . Note that in this case f(0+) can be strictly positive (and finite).Some information on possible behaviour of g at zero for such f can be found in[162] (see also [207] for closely related results).

Notes. The first-passage problem has a long history and a large numberof applications. Yet explicit solutions to the first-passage problem (for Brownianmotion) are known only in a limited number of special cases including linear orquadratic g . The law of τ is also known for a square-root boundary g but onlyin the form of a Laplace transform (which appears intractable to inversion). Theinverse problem seems even harder. For example, it is not known if there exists aboundary g for which τ is exponentially distributed (cf. [162]).

One way to tackle the problem is to derive an equation which links g and F .Motivated by this fact many authors have studied integral equations in connectionwith the first-passage problem (see e.g. [182], [205], [69], [201], [149], [65], [175], [48],[124]) under various hypotheses and levels of rigor. The main aim of this section(following [161]) is to present a unifying approach to the integral equations arisingin the first-passage problem that is done in a rigorous fashion and with minimaltools.

The approach naturally leads to a system of integral equations for g and F(paragraph 6) in which the first two equations contain the previously known ones


(paragraph 5). These equations are derived from a single master equation (The-orem 14.3) that can be viewed as a Chapman–Kolmogorov equation of Volterratype (see Theorem 14.2). The initial idea in the derivation of the master equationgoes back to Schrodinger [182]. The master equation cannot be reduced to a par-tial differential equation of forward or backward type (cf. [111]). A key technicaldetail needed to connect the second equation of the system to known methodsleads to a simple proof of the fact that F has a continuous density when g iscontinuously differentiable (Theorem 14.4). The problem of finding F when g isgiven is tackled using classic theory of linear integral equations (Theorem 14.7).The inverse problem is reduced to solving a system of nonlinear Volterra integralequations of the second kind (see (14.2.87)). General theory of such systems seemsfar from being complete at present.

Chapter V.

Optimal stopping in stochastic analysis

The aim of this chapter is to study a number of optimal stopping problems whichare closely related to sharp inequalities arising in stochastic analysis. We will beginby giving a general overview of the methodology which will be applied throughout.

15. Review of problems

In stochastic analysis one often deals with a “complicated” random variable Xc

and tries to say something about its properties in terms of a “simple” randomvariable Xs . For example, if B = (Bt)t≥0 is a standard Brownian motion andwe consider Xc = sup0≤t≤τ |Bt|2 for a stopping time τ of B , then Xc may bea complicated random variable and, for example, it may be nontrivial to computeEXc or even say something about its exact size. On the other hand, recallingthat EXc = E max0≤t≤τ |Bt|2 ≤ 4 Eτ by Doob’s inequality (and the optionalsampling theorem) and setting Xs = τ , we see that although Xs may not bethat simple at all, it may be possible to say something about its expectationEXs = Eτ (e.g. show that it is finite) and in this way get some informationabout the “complicated” quantity EXc = E max0≤t≤τ |Bt|2 (i.e. conclude thatit is finite). Even a more appealing choice in terms of simplicity for Xs is |Bτ |2when the inequality EXc = E max0≤t≤τ |Bt|2 ≤ 4E |Bτ |2 = 4EXs provides arather strong conclusion on the size of the maximum of |Bt|2 over all t ∈ [0, τ ]in terms of the terminal value |Bτ |2 .

This sort of reasoning is the main motivation for the present chapter. Itturns out that optimal stopping techniques prove very helpful in deriving sharpinequalities of the preceding type.

244 Chapter V. Optimal stopping in stochastic analysis

To describe this in more detail, still keeping it very general, let us assumethat X = (Xt)t≥0 is a Markov process, and for given functions L and K let

It =∫ t

0

L(Xs) ds, (15.0.1)

St = max0≤s≤t

K(Xs) (15.0.2)

be the integral process associated with L(X) and the maximum process associatedwith K(X) for t ≥ 0 . Given functions F and G we may and will consider thefollowing optimal stopping problem:

V (c) = supτ

E[F (Iτ , Xτ , Sτ ) − c G(Iτ , Xτ , Sτ )

](15.0.3)

where the supremum is taken over a class of stopping/Markov times τ , and c > 0is a given and fixed constant.

It follows from (15.0.3) that

EF (Iτ , Xτ , Sτ ) ≤ V (c) + c EG(Iτ , Xτ , Sτ ) (15.0.4)

for all stopping times τ and all c > 0 . Hence

EF (Iτ , Xτ , Sτ ) ≤ infc>0

(V (c) + c EG(Iτ , Xτ , Sτ )

)(15.0.5)

:= H(EG(Iτ , Xτ , Sτ )

)for all stopping times τ . In this way we have produced a function H which has thepower of providing a sharp estimate of EF (Iτ , Xτ , Sτ ) in terms of EG(Iτ , Xτ , Sτ ).Note that when supremum in (15.0.3) is attained, then equality in (15.0.4) isattained, so that whenever this is true for all c > 0 , it is in particular true forc∗ > 0 at which the infimum in (15.0.5) is attained (or approximately attained),demonstrating that (15.0.5) is indeed a sharp inequality as claimed.

In what follows we will study a number of specific examples of the optimalstopping problem (15.0.3). Normally the functions F and G (as well as L andK ) take a simple form. For example, in the case of Doob’s inequality above wehave X = B , K(x) = x2 , L(x) ≡ 1 , F (a, x, s) = s and G(a, x, s) = a . WhenG (or F to the same effect) is a nonlinear function of a (e.g. G(a, x, s) =

√a )

we speak of nonlinear problems. Note that such problems are also studied inSection 10 above (see also Section 20 below).

16. Wald inequalities

The aim of this section (following [78]) is to present the solution to a class ofWald type optimal stopping problems for Brownian motion, and from this deducesome sharp inequalities, which give bounds on the expectation of functionals ofrandomly stopped Brownian motion in terms of the expectation of the stoppingtime.

Section 16. Wald inequalities 245


Let B = (Bt)t≥0 be a standard Brownian motion defined on a probability space(Ω,F , P) . In this section we solve all optimal stopping problems of the followingform: Maximize the expectation

E(G(|Bτ |) − cτ

)(16.1.1)

over all stopping times τ of B with Eτ < ∞ , where the measurable functionG : R+ → R satisfies G(|x|) ≤ c|x|2+ d for all x ∈ R with some d ∈ R , andc > 0 is given and fixed.

It will be shown below (Theorem 16.1) that the (approximately) optimalstopping time is the first hitting time of the reflecting Brownian motion |B| =(|Bt|)t≥0 to the set of all (approximate) maximum points of the function x →G(|x|) − cx2 on R . This leads to some sharp inequalities which will be discussedbelow.


In this subsection we present the solution to the optimal stopping problem (16.1.1).For simplicity, we will only consider the case where G(|x|) = |x|p for 0 < p ≤ 2 ,and it will be clear from the proof below that the case of a general function G(satisfying the boundedness condition) could be treated analogously.

1. Thus, if B = (Bt)t≥0 is a standard Brownian motion, then the problemunder consideration is the following: Maximize the expectation

E(|Bτ |p − cτ

)(16.2.1)

over all stopping times τ of B with Eτ <∞ , where 0 0 aregiven and fixed.

Firstly, it should be noted that in the case p = 2 , we find by the classicalWald identity (see (3.2.6)) for Brownian motion ( E |Bτ |2 = Eτ ) that the expres-sion in (16.2.1) equals (1− c)Eτ . Thus, taking τ ≡ n or 0 for n ≥ 1 , dependingon whether 0 < c < 1 or 1 < c < ∞ , we see that the supremum equals +∞ or0 respectively. If c = 1 , then the supremum equals 0 , and any stopping time τof B with Eτ <∞ is optimal. These facts solve the problem (16.2.1) in the casep = 2 . The solution in the general case 0 0 be given and fixed.Consider the optimal stopping problem

supτ

E(|Bτ |p − cτ

)(16.2.2)


where the supremum is taken over all stopping times τ of B with Eτ <∞ . Thenthe optimal stopping time in (16.2.2) (the one at which the supremum is attained)is given by

τ∗p,c = inf

t > 0 : |Bt| =

( p

2c

)1/(2−p)

. (16.2.3)

Moreover, for all stopping times τ of B with Eτ <∞ we have

E(|Bτ |p − cτ

) ≤ (2−p

2

)( p

2c

)p/(2−p)

. (16.2.4)

The upper bound in (16.2.4) is best possible.

Proof. Given 0 0 , denote

Vτ (p, c) = E(|Bτ |p − cτ

)(16.2.5)

whenever τ is a stopping time of B with Eτ <∞ . Then by Wald’s identity forBrownian motion it follows that the expression in (16.2.5) may be equivalentlywritten in the following form:

Vτ (p, c) =∫ ∞

−∞

( |x|p − cx2)dPBτ (x) (16.2.6)

whenever τ is a stopping time of B with Eτ < ∞ . Our next step is to maximizethe function x → D(x) = |x|p − cx2 over R . For this, note that D(−x) = D(x)for all x ∈ R , and therefore it is enough to consider D(x) for x > 0 . We haveD′(x) = pxp−1 − 2cx for x > 0 , and hence we see that D attains its maximalvalue at the point ±(p/2c)1/(2−p) . Thus it is clear from (16.2.6) that the optimalstopping time in (16.2.2) is to be defined by (16.2.3). This completes the first partof the proof.

Finally, inserting τ∗ = τ∗p,c from (16.2.3) into (16.2.6), we easily find that

Vτ∗(p, c) = D(( p

2c

)1/(2−p))=(2 − p

2

)( p

2c

)p/(2−p)

. (16.2.7)

This establishes (16.2.4) with the last statement of the theorem, and the proof iscomplete.

Remark 16.2. The preceding proof shows that the solution to the problem (16.1.1)in the case of a general function G (satisfying the boundedness condition) could befound by using exactly the same method: The (approximately) optimal stoppingtime is the first hitting time of the reflecting Brownian motion |B| = (|Bt|)t≥0

to the set of all (approximate) maximum points of the function x → D(x) =G(|x|) − cx2 on R . Here “approximate” stands to cover the case (in an obviousmanner) when D does not attain its least upper bound on the real line.


2. In the remainder of this subsection we will explore some consequences ofthe inequality (16.2.4) in more detail. For this, let a stopping time τ of B withEτ < ∞ and 0 0

(cEτ +

(2 − p

2

)( p

2c

)p/(2−p))

. (16.2.8)

It is elementary to compute that this infimum equals (Eτ)p/2 . In this way weobtain

E |Bτ |p ≤ (Eτ)p/2 (0 < p ≤ 2) (16.2.9)

with the constant 1 being best possible in all the inequalities. (Observe that thisalso follows by Wald’s identity and Jensen’s inequality in a straightforward way.)

Next let us consider the case 2 < p < ∞ . Thus we shall look at −Vτ (p, c)instead of Vτ (p, c) in (16.2.5) and (16.2.6). By the same argument as for (16.2.6)we obtain

−Vτ (p, c) = E(cτ − |Bτ |p

)=∫ ∞

−∞

(cx2− |x|p) dPBτ (x) (16.2.10)

where 2 < p < ∞ . The same calculation as in the proof of Theorem 16.1 showsthat the function x → −D(x) = cx2− |x|p attains its maximal value over R atthe point ±(p/2c)1/(2−p) . Thus as in the proof of Theorem 16.1 we find

E(cτ −|Bτ |p

) ≤ (p − 22

)( p

2c

)p/(2−p)

. (16.2.11)

From this inequality we get

supc>0

(cEτ +

(2 − p

2

)( p

2c

)p/(2−p))

≤ E |Bτ |p. (16.2.12)

The same calculation as for the proof of (16.2.9) shows that this supremum equals(Eτ)p/2 . Thus as above for (16.2.9) we obtain

(Eτ)p/2 ≤ E |Bτ |p (2 ≤ p < ∞) (16.2.13)

with the constant 1 being best possible in all the inequalities. (Observe again thatthis also follows by Wald’s identity and Jensen’s inequality in a straightforwardway.)

3. The previous calculations together with the conclusions (16.2.9) and(16.2.13) indicate that the inequality (16.2.4)+(16.2.8) provide sharp estimateswhich are otherwise obtainable by a different method that relies upon convexityand Jensen’s inequality (see Remark 16.4 below). This leads precisely to the mainobservation: The previous procedure can be repeated for any measurable map G


satisfying the boundedness condition. In this way we obtain a sharp estimate ofthe form

EG(|Bτ |

) ≤ γG(Eτ) (16.2.14)

where γG is a function to be found (by maximizing and minimizing certain realvalued functions of real variables). We formulate this more precisely in the nextcorollary.

Corollary 16.3. Let B=(Bt)t≥0 be standard Brownian motion, and let G : R→R

be a measurable map. Then for any stopping time τ of B the following inequalityholds:

EG(|Bτ |

) ≤ infc>0

(cEτ + sup

x∈R

(G(|x|)− cx2

))(16.2.15)

and is sharp whenever the right-hand side is finite. Similarly, if H : R → R is ameasurable map, then for any stopping time τ of B with Eτ <∞ the followinginequality holds:

supc>0

(cEτ + inf

x∈R

(H(|x|)− cx2

)) ≤ EH(|Bτ |) (16.2.16)

and is sharp whenever the left-hand side is finite.

Proof. It follows from the proof of Theorem 16.1 as indicated in Remark 16.2 andthe lines above following it (or just straightforwardly by using Wald’s identity).It should be noted that the boundedness condition on the maps G and H iscontained in the nontriviality of the conclusions.

Remark 16.4. If we set H(x) = G(√

x) for x ≥ 0 , then

supx∈R

(G(|x|)− cx2

)= − inf

x≥0

(cx−H(x)

)=: −H(c) (16.2.17)

where H denotes the concave conjugate of H . Similarly, we have

infc>0

(cEτ + sup

x∈R

(G(|x|)− cx2

))= inf

(cEτ − H(c)

)= ˜

H(Eτ). (16.2.18)

Thus (16.2.15) reads as

EH(|Bτ |2) ≤ ˜H(Eτ). (16.2.19)

Moreover, since ˜H is the (smallest) concave function which dominates H , it is

clear from a simple comparison that (16.2.19) also follows by Jensen’s inequality.This provides an alternative way of looking at (16.2.15) and clarifies (16.2.8)–(16.2.9). (A similar remark may be directed to (16.2.16) with (16.2.12)–(16.2.13).)Note that (16.2.19) gets the form

EG(|Bτ |

) ≤ G(√

Eτ)

(16.2.20)

whenever x → G(√

x) is concave on R+ .


Remark 16.5. By using the standard time-change method (see Subsection 5.1),one can generalize and extend the inequalities (16.2.15) and (16.2.16) to cover thecase of all continuous local martingales. Let M = (Mt)t≥0 be a continuous localmartingale with the quadratic variation process 〈M〉 = (〈M〉t)t≥0 (see (3.3.6))such that M0 = 0 , and let G , H : R+ → R be measurable functions. Then forany t > 0 for which E 〈M〉t < ∞ the following inequalities hold:

EG(|Mt|) ≤ infc>0

(cE 〈M〉t + sup

x∈R

(G(|x|)− cx2

)), (16.2.21)

supc>0

(cE 〈M〉t + inf

x∈R

(H(|x|)− cx2

)) ≤ EH(|Mt|) (16.2.22)

and are sharp whenever the right-hand side in (16.2.21) and the left-hand side in(16.2.22) are finite.

To prove the sharpness of (16.2.21) and (16.2.22) for any given and fixedt > 0 , consider Mt = Bαt+τβ

with α > 0 and τβ being the first hitting time ofthe reflecting Brownian motion |B| = (|Bt|)t≥0 to some β > 0 . Letting α → ∞and using (integrability) properties of τβ (in the context of Corollary 16.3), bythe Burkholder-Davis-Gundy inequalities (see (C5) on page 63) and uniform inte-grability arguments one ends up with the inequalities (16.2.15) and (16.2.16) foroptimal τ = τβ , at least in the case when G allows that the limiting proceduresrequired can be performed (the case of general G can then follow by approxi-mation). Thus the sharpness of (16.2.21)–(16.2.22) follows from the sharpness of(16.2.15)–(16.2.16).

16.3. Applications

As an application of the methodology exposed above, we will present a simpleproof of the Dubins–Jacka–Schwarz–Shepp–Shiryaev (square-root-of-two) maxi-mal inequality for randomly stopped Brownian motion, which was first derived in[44] and independently in [103], and then proved by an entirely different methodin [45]. We will begin by stating two inequalities to be proved (the second onebeing the “square-root-of-two” inequality).

Let B = (Bt)t≥0 be a standard Brownian motion, and let τ be a stoppingtime of B with Eτ <∞ . Then the following inequalities are sharp:

E(

max0≤t≤τ

Bt

)≤

√Eτ, (16.3.1)

E(

max0≤t≤τ

|Bt|)≤ √

2√

Eτ. (16.3.2)

1. We shall first deduce these inequalities by our method, and then showtheir sharpness by exhibiting the optimal stopping times (at which the equalitiesare attained). Our approach to the problem of establishing (16.3.1) is motivated


by the fact that the process (max0≤s≤t Bs −Bt)t≥0 is equally distributed as thereflecting Brownian motion process (|Bt|)t≥0 for which we have found optimalbound (16.2.4) (from where by (16.2.8) we get (16.2.9) with p = 1 ), while EBτ = 0whenever E

√τ < ∞ . These observations clearly lead us to (16.3.1), at least for

some stopping times. To extend this to all stopping times, we shall use a simplemartingale argument.

Proof of (16.3.1): Set St = max0≤s≤t Bs for t ≥ 0 . Since (B2t − t)t≥0 is a

martingale, and (St −Bt)t≥0 is equally distributed as (|Bt|)t≥0 , we see that

Zt = c((St −Bt)2 − t

)+ 1/4c (16.3.3)

is a martingale (with respect to the natural filtration which is known to be thesame as the natural filtration of B ). Using EBτ = 0 , by the optional samplingtheorem (page 60) and the elementary inequality x− ct ≤ c(x2 − t)+1/4c , we find

E (Sτ − cτ) = E (Sτ −Bτ − cτ) ≤ EZτ = EZ0 = 1/4c (16.3.4)

for any bounded stopping time τ . Hence we get

ESτ ≤ infc>0

(cEτ+

14c

)=

√Eτ (16.3.5)

for any bounded stopping time τ . Passing to the limit, we obtain (16.3.1) for allstopping times with finite expectation. This completes the proof of (16.3.1).

2. Next we extend (16.3.1) to any continuous local martingale M = (Mt)t≥0

with M0 = 0 . For this, note that by the time change and (16.3.1) we obtain

E(

max0≤s≤t

Ms

)= E

(max0≤s≤t

B〈M〉s

)= E

(max

0≤s≤〈M〉t

Bs

)≤√

E 〈M〉t (16.3.6)

for all t > 0 .

3. In the next step we will apply (16.3.6) to the continuous martingale Mdefined by

Mt = E(|Bτ |−E |Bτ |

∣∣Ft∧τ

)(16.3.7)

for t ≥ 0 . In this way we get

E(

max0≤t<∞

E(|Bτ |−E |Bτ |

∣∣Ft∧τ

)) ≤√

E(|Bτ |−E |Bτ |

)2. (16.3.8)

We now pass to the proof of the “square-root-of-two” inequality.

Proof of (16.3.2): Since√

A − x2 + x ≤ √2A for 0 < x <

√A , by (16.3.8)

Section 17. Bessel inequalities 251

we find

E(

max0≤t≤τ

|Bt|)

= E(

max0≤t<∞

|Bt∧τ |)≤ E

(max

0≤t<∞E(|Bτ |

∣∣ Ft∧τ

))(16.3.9)

= E(

max0≤t<∞

E(|Bτ |−E |Bτ |

∣∣ Ft∧τ

))+ E |Bτ |

≤√

E(|Bτ |−E |Bτ |

)2 + E |Bτ |

=√

Eτ −(E |Bτ |)2 + E |Bτ | ≤

√2Eτ.

This establishes (16.3.2) and completes the first part of the proof.

4. To prove the sharpness of (16.3.1) one may take the stopping time

τ∗1 = inf

t>0 : |Bt| = a

(16.3.10)

for any a > 0 . Then the equality in (16.3.1) is attained. It follows by Wald’sidentity. Note that for any a > 0 the stopping time τ∗

1 could be equivalently (indistribution) defined by

τ∗1 = inf

t>0 : max

0≤s≤tBs − Bt ≥ a

. (16.3.11)

5. To prove the sharpness of (16.3.2) one may take the stopping time

τ∗2 = inf

t>0 : max

0≤s≤t|Bs| − |Bt| ≥ a

(16.3.12)

for any a > 0 . Then it is easily verified that E(max0≤t≤τ∗

2|Bt|

)= 2a and

Eτ∗2 = 2a2 (see [45]). Thus the equality in (16.3.2) is attained, and the proof of

the sharpness is complete.

17. Bessel inequalities

The aim of this section (following [45]) is to formulate and solve an optimal stop-ping problem for Bessel processes, and from this deduce a sharp maximal inequalitywhich gives bounds in terms of the expectation of the stopping time.


Recall that a Bessel process of dimension α ∈ R is a Markov process X = (Xt)t≥0

with the state space E = R+ and continuous sample paths being associated withthe infinitesimal generator

LX =α − 12x

d

dx+

12

d2

dx2(17.1.1)


and the boundary point 0 is a trap if α ≤ 0 , instantaneously reflecting if 0 <α < 2 , and entrance if α ≥ 2 . (For a detailed study of Bessel processes see [132],[100], [53], [94], [174].)

In the case α = n ∈ N the Bessel process X can be realised as the radialpart of an n -dimensional Brownian motion (B1, . . . , Bn) , i.e.

Xxt =

( n∑i=1

∣∣ai+Bit

∣∣2)1/2

(17.1.2)

for t ≥ 0 where Xx0 = x =

(∑ni=1 |ai|2

)1/2 .

Given a Bessel process X = (Xt)t≥0 of dimension α ∈ R , let S = (St)t≥0 bethe maximum process associated with X , and let Px,s be a probability measureunder which X0 = x and S0 = s where s ≥ x ≥ 0 . Recall that this is possibleto achieve by setting

St = s ∨ max0≤u≤t

Xu (17.1.3)

where X0 = x under Px .

The main purpose of the present section is to consider the following optimalstopping problem:

V (x, s) = supτ

Ex,s(Sτ − cτ) (17.1.4)

where 0 ≤ x ≤ s and c > 0 are given and fixed, and the supremum is taken overall stopping times τ of X .

The solution to this problem is presented in the next subsection (Theorem17.1). If we set V (0, 0) = V (c) to indicate the dependence on c in (17.1.4), thenit follows that

ESτ ≤ infc>0

(V (c) + c Eτ

)(17.1.5)

where X0 = S0 = 0 under P . This yields a sharp maximal inequality (Corol-lary 17.2) where the right-hand side defines a function of Eτ . In the case α = 1this inequality reduces to the inequality (16.3.2).


Recalling our discussion on the kinematics of the process (X, S) given in Sec-tion 13 and applying similar arguments in the present setting one obtains thefollowing result.

Theorem 17.1. Consider the optimal stopping problem (17.1.4) where S is themaximum process associated with the Bessel process X of dimension α ∈ R andc > 0 is given and fixed.

The following stopping time is optimal in (17.1.4):

τ∗ = inft ≥ 0 : St ≥ s∗ & Xt ≤ g∗(St)

(17.2.1)

Section 17. Bessel inequalities 253

where s → g∗(s) is the maximal solution of the nonlinear differential equation:

2c

α − 2g′(s)g(s)

(1 −

(g(s)s

)α−2)

= 1 (17.2.2)

satisfying g∗(s) < s for s > s∗ ≥ 0 where g∗(s∗) = 0 . If α = 2 then (17.2.2)reads :

2c g′(s)g(s) log( s

g(s)

)= 1 (17.2.3)

which is obtained from (17.2.2) by passing to the limit as α → 2 . The solutiong∗ to (17.2.2) or (17.2.3) may also be characterized by the boundary condition atinfinity:

lims→∞

g∗(s)s

= 1. (17.2.4)

The value function V in (17.1.4) is explicitly given as follows. Setting

C1∗ =

(x, s) ∈ R+×R+ : s > s∗ & g∗(s) ≤ x ≤ s

, (17.2.5)

C2∗ =

(x, s) ∈ R+×R+ : 0 ≤ x ≤ s ≤ s∗

(s∗ = 0), (17.2.6)

we denote by C∗ = C1∗ ∪C2

∗ the continuation set and by D∗ =(x, s) ∈ R+×R+ :

0 ≤ x ≤ s \ C∗ the stopping set.

If α > 0 then

V (x, s) =

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

s if (x, s) ∈ D∗,

s +c

α

(x2 − g2

∗(s))

+2cg2∗(s)α(α − 2)

[(g∗(s)

x

)α−2

− 1]

if (x, s) ∈ C1∗ and α = 2,

s +c

2(x2 − g2

∗(0))

+ cg2∗(s) log

(g∗(s)x

)if (x, s) ∈ C1

∗ and α = 2,

c

αx2 + s∗ if (x, s) ∈ C2

∗ .

(17.2.7)

If α = 0 then

V (x, s) =

⎧⎪⎪⎪⎨⎪⎪⎪⎩s if (x, s) ∈ D∗,

s +c

2(g2∗(s) − x2

)+ cx2 log

( x

g∗(s)

)if (x, s) ∈ C∗.

(17.2.8)


If α < 0 then

V (x, s) =

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩s if (x, s) ∈ D∗,

s +c

α

(x2 − g2

∗(s))

+2cg2

∗(s)α(α − 2)

[(g∗(s)

x

)α−2

− 1]

if (x, s) ∈ C∗.

(17.2.9)

(If α ≤ 0 then s∗ = 0 and (0, 0) ∈ D∗ so that C2∗ = ∅ .)

Proof. This can be derived using Corollary 13.3 (for remaining details see theoriginal article [45]).

Corollary 17.2. Let X = (Xt)t≥0 be a Bessel process of dimension α > 0 , and letS = (St)t≥0 be the maximum process associated with X such that X0 = S0 = 0under P .

Then we have:

E(

max0≤t≤τ

Xt

)≤√

4s∗(1, α)√

Eτ (17.2.10)

for all stopping times τ of X , where s∗(1, α) is the root of the equation g∗(s) = 0and s → g∗(s) is the maximal solution of (17.2.2) with c = 1 satisfying g∗(s) < sfor all s > s∗(1, α) . (This solution may also be characterized by the boundarycondition at infinity as in (17.2.4) above.)

Proof. From (17.2.7) we see that

supτ

E(

max0≤t≤τ

Xt − cτ)

= s∗(c, α). (17.2.11)

By self-similarity of X in the sense that(X

√cx

ct√c

)t≥0

law=(Xx

t

)t≥0

(17.2.12)

it is not difficult to verify that

s∗(c, α) =1c

s∗(1, α). (17.2.13)

From (17.2.12) and (17.2.13) we get

E(

max0≤t≤τ

Xt

)≤ inf

c>0

(s∗(1, α)c

+ c Eτ)

=√

4s∗(1, α)√

Eτ (17.2.14)

as claimed. Note that if α = 1 then g∗(s) = s − 1/2c so that s∗(1, 1) = 1/2 and

(17.2.10) becomes (16.3.2).

Section 18. Doob inequalities 255

18. Doob inequalities

The main purpose of the section (following [80]) is to derive and examine a sharpmaximal inequality of Doob type for one-dimensional Brownian motion which maystart at any point.


Let us assume that we are given a standard Brownian motion B = (Bt)t≥0 whichis defined on a probability space (Ω,F , P) and which starts at 0 under P . Thenthe well-known Doob maximal inequality states:

E(

max0≤t≤τ

|Bt|2)≤ 4 E |Bτ |2 (18.1.1)

where τ may be any stopping time for B with Eτ < ∞ (see [40, p. 353] and(C4) on page 62). The constant 4 is known to be best possible in (18.1.1). Forthis one can consider the stopping times

σλ,ε = inf

t > 0 : max0≤s≤t

|Bs| − λ|Bt| ≥ ε

(18.1.2)

where λ, ε > 0 . It is well known that E (σλ,ε)p/2 < ∞ if and only if λ < p/(p−1)whenever ε > 0 (see e.g. [221]). Applying Doob’s maximal inequality with ageneral constant K > 0 to the stopping time in (18.1.2) with some ε > 0 when0<λ<2 , we get

E(

max0≤t≤σλ,ε

|Bt|2)

= λ2E |Bσλ,ε|2 + 2λεE |Bσλ,ε

| + ε2 ≤ KE |Bσλ,ε|2. (18.1.3)

Dividing through in (18.3.1) by E |Bσλ,ε|2 and using that E |Bσλ,ε

|2 = E (σλ,ε) →∞ together with E |Bσλ,ε

|/E |Bσλ,ε|2 ≤ 1/

√Eσλ,ε → 0 as λ ↑ 4 , we see that

K ≥ 4 .

Motivated by these facts our main aim in this section is to find an analogueof the inequality (18.1.1) when the Brownian motion B does not necessarily startfrom 0 , but may start at any given point x ≥ 0 under Px . Thus Px(B0 = x) = 1for all x ≥ 0 , and we identify P0 with P . Our main result (Theorem 18.1) is theinequality

Ex

(max

0≤t≤τ|Bt|2

)≤ 4Ex|Bτ |2 − 2x2 (18.1.4)

which is valid for any stopping time τ for B with Exτ < ∞ , and which is shownto be sharp as such. This is obtained as a consequence of the following inequality:

Ex

(max

0≤t≤τ|Bt|2

)≤ c Exτ +

c

2

(1 −

√1− 4

c

)x2 (18.1.5)


which is valid for all c ≥ 4 . If c > 4 then

τc = inf

t > 0 : max0≤s≤t

|Bs| − 21 +

√1−4/c

|Bt| ≥ 0

(18.1.6)

is a stopping time at which equality in (18.1.5) is attained, and moreover we have

Exτc =

(1 −√1−4/c

)24√

1−4/cx2 (18.1.7)

for all x ≥ 0 and all c > 4 .

In particular, if we consider the stopping time

τλ,ε = inf

t > 0 : max0≤s≤t

Bs −λBt ≥ ε

(18.1.8)

then (18.1.7) can be rewritten to read as follows:

E0τλ,ε =ε2

λ(2−λ)(18.1.9)

for all ε > 0 and all 0 < λ < 2 . Quite independently from this formula and itsproof, below we present a simple argument for Eτ2,ε = ∞ which is based uponTanaka’s formula (page 67). Finally, since σλ,ε defined by (18.1.2) is shown to bea convolution of τλ,λε and Hε , where Hε = inf t > 0 : |Bt| = ε , from (18.1.9)we obtain the formula

E0σλ,ε =2ε2

2−λ(18.1.10)

for all ε > 0 and all 0 < λ < 2 (see Corollary 18.5 below).


In this subsection we will solve the problem formulated in the previous subsection.The main result is contained in the following theorem (see also Corollaries 18.2and 18.3 below).

Theorem 18.1. Let B = (Bt)t≥0 be a standard Brownian motion started at xunder Px for x ≥ 0 , and let τ be any stopping time for B such that Exτ < ∞ .Then the following inequality is valid :

Ex

(max

0≤t≤τ|Bt|2

)≤ 4 Ex|Bτ |2 − 2x2. (18.2.1)

The constants 4 and 2 are best possible.


Proof. We shall begin by considering the following optimal stopping problem:

V (x, s) = supτ

Ex,s(Sτ − cτ) (18.2.2)

where the supremum is taken over all stopping times τ for B satisfying Ex,sτ <∞ , while the maximum process S = (St)t≥0 is defined by

St =(

max0≤r≤t

|Br|2)∨ s (18.2.3)

where s ≥ x ≥ 0 are given and fixed. The expectation in (18.2.2) is taken withrespect to the probability measure Px,s under which S starts at s , and theprocess X = (Xt)t≥0 defined by

Xt = |Bt|2 (18.2.4)

starts at x . The Brownian motion B from (18.2.3) and (18.2.4) may be realizedas

Bt = Bt +√

x (18.2.5)

where B = (Bt)t≥0 is a standard Brownian motion started at 0 under P . Thusthe (strong) Markov process (X, S) starts at (x, s) under P , and Px,s may beidentified with P .

By Ito’s formula (page 67) we find

dXt = dt + 2√

Xt dBt. (18.2.6)

Hence we see that the infinitesimal operator of the (strong) Markov process X in(0,∞) acts like

LX =∂

∂x+ 2x

∂2

∂x2(18.2.7)

while the boundary point 0 is a point of the instantaneous reflection.

If we assume that the supremum in (18.2.2) is attained at the exit time froman open set by the (strong) Markov process (X, S) which is degenerated in thesecond component, then by the general Markov processes theory (cf. Chapter III)it is plausible to assume that the value function x → V (x, s) satisfies the followingequation:

LXV (x, s) = c (18.2.8)

for x ∈ (g∗(s), s) with s > 0 given and fixed, where s → g∗(s) is an optimalstopping boundary to be found (cf. Section 13). The boundary conditions whichmay be fulfilled are the following:

V (x, s)∣∣x=g∗(s)+


∂V

∂x(x, s)

∣∣∣x=g∗(s)+

= 0 (smooth fit), (18.2.10)

∂V

∂s(x, s)

∣∣∣x=s−

= 0 (normal reflection). (18.2.11)


The general solution to the equation (18.2.8) for fixed s is given by

V (x, s) = A(s)√

x + B(s) + cx (18.2.12)

where A(s) and B(s) are unspecified constants. From (18.2.9)–(18.2.10) we findthat

A(s) = −2 c√

g∗(s), (18.2.13)B(s) = s + c g∗(s). (18.2.14)

Inserting this into (18.2.12) gives

V (x, s) = −2 c√

g∗(s)√

x + s + cg∗(s) + cx. (18.2.15)

By (18.2.11) we find that s → g∗(s) is to satisfy the (nonlinear) differentialequation

cg′(s)(

1 −√

s

g(s)

)+ 1 = 0. (18.2.16)

The general solution of the equation (18.2.16) can be expressed in closed form.Instead of going into this direction we shall rather note that this equation admitsa linear solution of the form

g∗(s) = αs (18.2.17)

where the given α > 0 is to satisfy

α −√α + 1/c = 0. (18.2.18)

Motivated by the maximality principle (see Section 13) we shall choose thegreater α satisfying (18.2.18) as our candidate:

α =(

1 +√

1−4/c

2

)2

. (18.2.19)

Inserting this into (18.2.15) gives

V∗(x, s) =

−2c

√αxs + (1 + cα)s + cx if αs ≤ x ≤ s,

s if 0 ≤ x ≤ αs(18.2.20)

as a candidate for the value function V (x, s) defined in (18.2.2). The optimalstopping time is then to be

τ∗ = inf

t>0 : Xt≤g∗(St)

(18.2.21)

where s → g∗(s) is defined by (18.2.17)+(18.2.19).

To verify that the formulae (18.2.20) and (18.2.21) are indeed correct, weshall use the Ito–Tanaka–Meyer formula (page 68) being applied two-dimensionally


(see [81] for a formal justification of its use in this context — note that (x, s) →V∗(x, s) is C2 outside (g∗(s), s) : s > 0 while x → V∗(x, s) is convex and C2

on (0, s) but at g∗(s) where it is only C1 whenever s > 0 is given and fixed —for the standard one-dimensional case see (3.3.23)). In this way we obtain

V∗(Xt, St) = V∗(X0, S0) +∫ t

0

∂V∗∂x

(Xr, Sr) dXr (18.2.22)

+∫ t

0

∂V∗∂s

(Xr, Sr) dSr +12

∫ t

0

∂2V∗∂x2

(Xr, Sr) d〈X, X〉r

where we set (∂2V∗/∂x2)(g∗(s), s) = 0 . Since the increment dSr equals zerooutside the diagonal x = s , and V∗(x, s) at the diagonal satisfies (18.2.11), we seethat the second integral in (18.2.22) is identically zero. Thus by (18.2.6)–(18.2.7)and the fact that d〈X, X〉t = 4Xt dt , we see that (18.2.22) can be equivalentlywritten as follows:

V∗(Xt, St) = V∗(x, s) +∫ t

0

LXV∗(Xr, Sr) dr (18.2.23)

+ 2∫ t

0

√Xr

∂V∗∂x

(Xr, Sr) dBr.

Next note that LXV∗(y, s) = c for g∗(s) < y < s , and LXV∗(y, s) = 0 for0 ≤ y ≤ g∗(s) . Moreover, due to the normal reflection of X , the set of thoser > 0 for which Xr = Sr is of Lebesgue measure zero. This by (18.2.23) showsthat

V∗(Xτ , Sτ ) ≤ V∗(x, s) + cτ + Mτ (18.2.24)

for any stopping time τ for B , where M = (Mt)t≥0 is a continuous local mar-tingale defined by

Mt = 2∫ t

0

√Xr

∂V∗∂x

(Xr, Sr) dBr. (18.2.25)

Moreover, this also shows that

V∗(Xτ , Sτ ) = V∗(x, s) + c τ + Mτ (18.2.26)

for any stopping time τ for B satisfying τ ≤ τ∗ .

Next we show thatEx,sMτ = 0 (18.2.27)

whenever τ is a stopping time for B with Ex,sτ < ∞ . For (18.2.27), by theBurkholder–Davis–Gundy inequality for continuous local martingales (see (C5)on page 63), it is sufficient to show that

Ex,s

(∫ τ

0

(√Xr

∂V∗∂x

(Xr, Sr))2

1Xr≥g∗(Sr) dr

)1/2

:= I < ∞. (18.2.28)


From (18.2.20) we compute:

∂V∗∂x

(y, s) = −c√

αs√y

+ c (18.2.29)

for αs ≤ y ≤ s . Inserting this into (18.2.28) we get:

I = c Ex,s

(∫ τ

0

(√Xr −

√αSr

)21Xr≥αSr dr

)1/2

(18.2.30)

≤ c (1−√α) Ex,s

(∫ τ

0

Sr dr

)1/2

≤ c (1−√α) Ex,s

(√Sτ

√τ)

≤ c (1−√α)√

Ex,sSτ

√Ex,sτ

= c (1−√α)(

Ex,s

((max0≤t≤τ

∣∣Bt+√

x∣∣2) ∨ s

))1/2√Ex,sτ

≤ c (1−√α)(2 Ex,s

(max0≤t≤τ

|Bt|2)+ 2 x+s

)1/2√Ex,sτ

≤ c (1−√α)(8 Ex,sτ + 2x + s

)1/2√Ex,sτ < ∞

where we used Holder’s inequality, Doob’s inequality (18.1.1), and the fact thatEx,s|Bτ |2 = Ex,sτ whenever Ex,sτ < ∞ .

Since V∗(x, s) ≥ s , from (18.2.24)+(18.2.27) we find

V (x, s) = supτ

Ex,s

(Sτ − cτ

) ≤ supτ

Ex,s

(Sτ −V∗(Xτ , Sτ )

)(18.2.31)

+ supτ

Ex,s

(V∗(Xτ , Sτ )− cτ

) ≤ V∗(x, s).

Moreover, from (18.2.26)–(18.2.27) with τ = τ∗ we see that

Ex,s

(Sτ∗ − c τ∗

)= Ex,s

(V∗(Xτ∗ , Sτ∗)− c τ∗

)= V∗(x, s) (18.2.32)

provided that Ex,sτ∗ < ∞ , which is known to be true if and only if c > 4 (see[221]). (Below we present a different proof of this fact and moreover computethe value Ex,sτ∗ exactly.) Matching (18.2.31) and (18.2.32) we see that the valuefunction (18.2.2) is indeed given by the formula (18.2.20), and an optimal stoppingtime for (18.2.2) (at which the supremum is attained) is given by (18.2.21) withs → g∗(s) from (18.2.17) and α ∈ (0, 1) from (18.2.19).

In particular, note that from (18.2.20) with α from (18.2.19) we get

V∗(x, x) =c

2

(1

√1 − 4

c

)x. (18.2.33)

Applying the very definition of V (x, x) = V∗(x, x) and letting c ↓ 4 , this yields

Ex

(max0≤t≤τ

|Bt|2)≤ 4Exτ + 2x. (18.2.34)


Finally, standard arguments show that

Ex|Bτ |2 = Ex|Bτ +√

x|2 = Ex|Bτ |2 + 2√

x Ex(Bτ ) + x = Exτ + x. (18.2.35)

Inserting this into (18.2.34) we obtain (18.2.1). The sharpness clearly follows fromthe definition of the value function in (18.2.2) completing the proof of the theorem.

The previous result and method easily extend to the case p>1 . For reader’sconvenience we state this extension and sketch the proof.

Corollary 18.2. Let B = (Bt)t≥0 be a standard Brownian motion started at xunder Px for x ≥ 0 , let p > 1 be given and fixed, and let τ be any stoppingtime for B such that Exτp/2 < ∞ . Then the following inequality is sharp:

Ex

(max0≤t≤τ

|Bt|p)≤( p

p− 1

)pEx|Bτ |p −

( p

p−1

)xp. (18.2.36)

The constants (p/(p−1))p and p/(p−1) are best possible.

Proof. In parallel to (18.2.2) let us consider the following optimal stopping problem:

V (x, s) = supτ

Ex,s

(Sτ −cIτ

)(18.2.37)

where the supremum is taken over all stopping times τ for B satisfying Ex,sτp/2

< ∞ , and the underlying processes are given as follows:

St =(

max0≤r≤t

Xr

)∨ s, (18.2.38)

It =∫ t

0

(Xr

)(p−2)/pdr, (18.2.39)

Xt = |Bt|p, (18.2.40)

Bt = Bt + x1/p, (18.2.41)

where B = (Bt)t≥0 is a standard Brownian motion started at 0 under P = Px,s .This problem can be solved in exactly the same way as the problem (18.2.2) alongthe following lines.

The infinitesimal operator of X equals

LX =p(p−1)

2x1−2/p ∂

∂x+

p2

2x2−2/p ∂2

∂x2. (18.2.42)

The analogue of the equation (18.2.8) is

LXV (x, s) = c x(p−2)/p. (18.2.43)


The conditions (18.2.9)–(18.2.11) are to be satisfied again. The analogue of thesolution (18.2.15) is

V (x, s) = − 2c

p−1g1−1/p∗ (s)x1/p + s +

2c

pg∗(s) +

2c

p(p−1)x (18.2.44)

where s → g∗(s) is to satisfy the equation

2c

pg′(s)

(1−

( s

g(s)

)1/p)

+ 1 = 0. (18.2.45)

Again, as in (18.2.16), this equation admits a linear solution of the form

g∗(s) = αs (18.2.46)

where 0 < α < 1 is the maximal root (out of two possible ones) of the equation

α − α1−1/p + p/2c = 0. (18.2.47)

By standard arguments one can verify that (18.2.47) admits such a root ifand only if c ≥ pp+1/2(p−1)(p−1) . The optimal stopping time is then to be

τ∗ = inf t>0 : Xt ≤ g∗(St) (18.2.48)

where s → g∗(s) is from (18.2.46).

To verify that the guessed formulae (18.2.44) and (18.2.48) are indeed correctwe can use exactly the same procedure as in the proof of Theorem 18.1. For this,it should be recalled that Ex,sτ

p/2∗ < ∞ if and only if c > pp+1/2(p−1)(p−1) (see

[221]). Note also by Ito’s formula (page 67) and the optional sampling theorem(page 60) that the analogue of (18.2.35) is given by

Ex,s

(Xτ

)= x +

p(p− 1)2

Ex,s(Iτ ) (18.2.49)

whenever Ex,s(τp/2) < ∞ , which was the motivation for considering the problem(18.2.37) with (18.2.39). The remaining details are easily completed and will beleft to the reader.

Due to the universal role of Brownian motion in this context, the inequality(18.2.36) extends to all non-negative submartingales. This can be obtained byusing the maximal embedding result of Jacka [101].

Corollary 18.3. Let X = (Xt)t≥0 be a non-negative cadlag (right continuous withleft limits) uniformly integrable submartingale started at x ≥ 0 under P . Let X∞denote the P-a.s. limit of Xt for t → ∞ (which exists by (B1) on page 61). Thenthe following inequality is satisfied and sharp:

E(

supt>0

Xpt

)≤( p

p−1

)pEXp

∞ − p

p− 1xp (18.2.50)

for all p > 1 .


Proof. Given such a submartingale X = (Xt)t≥0 satisfying EX∞ < ∞ , and aBrownian motion B = (Bt)t≥0 started at X0 = x under Px , by the result of Jacka[101] we know that there exists a stopping time τ for B , such that |Bτ | ∼ X∞and P supt≥0 Xt ≥ λ ≤ Pxmax0≤t≤τ |Bτ | ≥ λ for all λ > 0 , with (Bt∧τ )t≥0

being uniformly integrable. The result then easily follows from Corollary 18.2 byusing the integration by parts formula. Note that by the submartingale propertyof (|Bt∧τ |)t≥0 we get supt≥0 Ex|Bt∧τ |p = Ex|Bτ |p for all p > 1 , so that Exτp/2

is finite if and only if Ex|Bτ |p is so.

Notes. There are other ways to derive the inequalites (18.2.36). Burkholderobtained these inequalities as a by-product from his new proof of Doob’s inequalityfor discrete non-negative submartingales (see [25, p. 14]). While the proof giventhere in essence relies on a submartingale property, the proof given above is basedon the (strong) Markov property. An advantage of the latter approach lies inits applicability to all diffusions (see [81]). Another advantage is that during theproof one explicitly writes down the optimal stopping times (those through whichequality is attained). Cox [32] also derived the analogue of these inequalities fordiscrete martingales by a method which is based on results from the theory ofmoments. In his paper Cox notes that “the method does have the drawback ofcomputational complexity, which sometimes makes it difficult or impossible topush the calculations through”. Cox [32] also observed that equality in Doob’smaximal inequality (18.2.36) cannot be attained by a non-zero (sub)martingale.It may be noted that this fact follows from the method and results above (equalityin (18.2.36) is attained only in the limit). For an extension of the results in thissubsection to Bessel processes see [150].

18.3. The expected waiting time

In this subsection we will derive an explicit formula for the expectation of theoptimal stopping time τ∗ constructed in the proof of Theorem 18.1 (or Corollary18.2).

Throughout we will work within the setting and notation of Theorem 18.1and its proof. By (18.2.21) with (18.2.17) we have

τ∗ = inf t>0 : Xt≤αSt (18.3.1)

where α = α(c) is the constant given in (18.2.19) for c > 4 . Note that 1/4 <α(c) ↑ 1 as c ↑ ∞ . Our main task in this subsection is to compute explicitly thefunction

m(x, s) = Ex,sτ∗ (18.3.2)

for 0 ≤ x ≤ s , where Ex,s denotes the expectation with respect to Px,s underwhich X starts at x and S starts at s . Since clearly m(x, s) = 0 for 0 ≤ x ≤αs , we shall assume throughout that αs < x ≤ s are given and fixed.


Because τ∗ may be viewed as the exit time from an open set by the (strong)Markov process (X, S) which is degenerated in the second component, by thegeneral Markov processes theory (see Subsection 7.2) it is plausible to assumethat x → m(x, s) satisfies the equation

LXm(x, s) = −1 (18.3.3)

for αs < x < s with LX given by (18.2.7). The following two boundary conditionsare apparent:

m(x, s)∣∣x=αs+

= 0 (instantaneous stopping), (18.3.4)

∂m

∂s(x, s)

∣∣∣x=s−


The general solution to (18.3.3) is given by

m(x, s) = A(s)√

x + B(s) − x (18.3.6)

where A(s) and B(s) are unspecified constants. By (18.3.4) and (18.3.5) we find

A(s) = Cs∆ +2α

2√

α − 1√

s, (18.3.7)

B(s) = −C√

α s∆+1/2 − α

2√

α − 1s (18.3.8)

where C = C(α) is a constant to be determined, and where

∆ =√

α

2(1−√α)

. (18.3.9)

In order to determine the constant C , we shall note by (18.3.6)–(18.3.9) that

m(x, x) = C(1−√α) x1/2(1−√

α) +(√

α − 1)2

2√

α − 1x. (18.3.10)

Observe now that the power 1/2(1−√α) > 1 , due to the fact that α = α(c) >

1/4 when c > 4 . However, the value function in (18.2.33) is linear and given by

V∗(x, x) := V∗(x; c) = K(c) · x (18.3.11)

where K(c) = (c/2)(1 −√1−4/c) . This indicates that the constant C must beidentically zero. Formally, this is verified as follows.

Since c > 4 there is λ ∈ (0, 1) such that λc > 4 . By definition of the valuefunction we have

0 < V∗(x; c) = Ex,x

(Sτ∗(c) − c τ∗(c)

)(18.3.12)

= Ex,x

(Sτ∗(c) − λc τ∗(c)

)− (1−λ)c Ex,xτ∗(c)≤ V∗(x; λc) − (1−λ)c Ex,xτ∗(c)≤ K(λc) · x − (1−λ)c Ex,xτ∗(c).


This shows that x → m(x, x) is at most linear:

m(x, x) = Ex,xτ∗(c) ≤ K(λc)(1−λ)c

x. (18.3.13)

Looking back at (18.3.10) we may conclude that C ≡ 0 .

Thus by (18.3.6)–(18.3.8) with C ≡ 0 we end up with the following candi-date:

m(x, s) =2α

2√

α − 1√

xs − α

2√

α − 1s − x (18.3.14)

for Ex,sτ∗ when αs < x ≤ s . In order to verify that this formula is indeed correctwe shall use the Ito–Tanaka–Meyer formula (page 68) in the proof below.

Theorem 18.4. Let B = (Bt)t≥0 be a standard Brownian motion, and let X =(Xt)t≥0 and S = (St)t≥0 be associated with B by formulae (18.2.3)–(18.2.4).Then for the stopping time τ∗ defined in (18.3.1) we have:

Ex,sτ∗ =

⎧⎨⎩2α

2√

α−1√

xs− α

2√

α − 1s − x if αs ≤ x ≤ s,

0 if 0 ≤ x ≤ αs(18.3.15)

where α > 1/4 .

Proof. Denote the function on the right-hand side of (18.3.15) by m(x, s) . Notethat x → m(x, s) is concave and non-negative on [αs, s] for each fixed s > 0 .

By the Ito–Tanaka–Meyer formula (see [81] for a justification of its use) weget:

m(Xt, St) = m(X0, S0) +∫ t

0

LXm(Xr, Sr) dr (18.3.16)

+ 2∫ t

0

√Xr

∂m

∂x(Xr, Sr) dBr +

∫ t

0

∂m

∂s(Xr, Sr) dSr.

Due to (18.3.5) the final integral in (18.3.16) is identically zero.

In addition, let us consider the region G = (x, s) : αs < x < s + 1 . Given(x, s) ∈ G choose bounded open sets G1 ⊂ G2 ⊂ · · · such that

⋃∞n=1 Gn = G

and (x, s) ∈ G1 . Denote the exit time of (X, S) from Gn by τn . Then clearlyτn ↑ τ∗ as n → ∞ . Denote further the second integral in (18.3.16) by Mt . ThenM = (Mt)t≥0 is a continuous local martingale, and we have

Ex,sMτn = 0 (18.3.17)

for all n ≥ 1 . For this (see page 60) note that

Ex,s

(∫ τn

0

(√Xr

∂m

∂x(Xr, Sr)

)2dr

)≤ KEx,sτn < ∞ (18.3.18)


with some K > 0 , since (x, s) → √x (∂m/∂x)(x, s) is bounded on the closure

of Gn .

By (18.3.3) from (18.3.16)–(18.3.17) we find

Ex,sm(Xτn , Sτn) = m(x, s) − Ex,sτn. (18.3.19)

Since (x, s) → m(x, s) is non-negative, hence first of all we may deduce

Ex,sτ∗ = limn→∞ Ex,sτn ≤ m(x, s) < ∞. (18.3.20)

This proves the finiteness of the expectation of τ∗ (see [221] for another proofbased on random walk).

Moreover, motivated by a uniform integrability argument we may note that

m(Xτn , Sτn) ≤ 2α

2√

α − 1

√XτnSτn ≤ 2α

2√

α − 1Sτ∗ (18.3.21)

uniformly over all n ≥ 1 . By Doob’s inequality (18.1.1) and (18.3.20) we find

Ex,sSτ∗ ≤ 2(4Ex,sτ∗ + x

)+ s < ∞. (18.3.22)

Thus the sequence (m(Xτn , Sτn))n≥1 is uniformly integrable, while it clearly con-verges pointwise to zero. Hence we may conclude

limn→∞Ex,sm(Xτn , Sτn) = 0. (18.3.23)

This shows that we have an equality in (18.3.20), and the proof is complete.

Corollary 18.5. Let B = (Bt)t≥0 be a standard Brownian motion started at 0under P . Consider the stopping times

τλ,ε = inf

t > 0 : max0≤s≤t

Bs − λBt ≥ ε, (18.3.24)

σλ,ε = inf

t > 0 : max0≤s≤t

|Bs| − λ|Bt| ≥ ε

(18.3.25)

for ε > 0 and 0 < λ < 2 . Then σλ,ε is a convolution of τλ,λε and Hε , whereHε = inf t > 0 : |Bt| = ε , and the following formulae are valid :

Eτλ,ε =ε2

λ(2−λ), (18.3.26)

Eσλ,ε =2ε2

2−λ(18.3.27)

for all ε > 0 and all 0 < λ < 2 .


Proof. Consider the definition rule for σλ,ε in (18.3.25). Clearly σλ,ε > Hε andafter hitting ε , the reflected Brownian motion |B| = (|Bt|)t≥0 does not hit zerobefore σλ,ε . Thus its absolute value sign may be dropped out during the timeinterval between Hε and σλ,ε , and the claim about the convolution identityfollows by the reflection property and the strong Markov property of Brownianmotion.

(18.3.26): Consider the stopping time τ∗ defined in (18.3.1) for s = x . Bythe very definition it can be rewritten to read as follows:

τ∗ = inf

t > 0 : |Bt|2 ≤ α max0≤s≤t

|Bs|2

(18.3.28)

= inf

t > 0 : max0≤s≤t

|Bs| − 1√α|Bt| ≥ 0

= inf

t > 0 : max

0≤s≤t|Bs+

√x| − 1√

α|Bt+

√x| ≥ 0

= inf

t > 0 : max

0≤s≤t

(Bs+

√x)− 1√

α

(Bt+

√x) ≥ 0

= inf

t > 0 : max

0≤s≤tBs − 1√

αBt ≥

(1√α− 1

)√x

.

Setting λ = 1/√

α and ε = (1/√

α − 1)√

x , by (18.3.15) hence we find

Eτλ,ε = Ex,xτ∗ =(√

α − 1)2

2√

α − 1x =

ε2

λ(2−λ). (18.3.29)

(18.3.27): Since EHε = ε2 , by (18.3.26) we get

Eσλ,ε = Eτλ,λε + EHε =2ε2

2−λ. (18.3.30)

The proof is complete.

Remark 18.6. Let B = (Bt)t≥0 be a standard Brownian motion started at 0under P . Consider the stopping time

τ2,ε = inf

t > 0 : max0≤s≤t

Bs −2Bt ≥ ε

(18.3.31)

for ε ≥ 0 . It follows from (18.3.26) in Corollary 18.5 that

Eτ2,ε = +∞ (18.3.32)

if ε > 0 . Here we present another argument based upon Tanaka’s formula (page67) which implies (18.3.32).


For this consider the process

βt =∫ t

0

sign (Bs) dBs (18.3.33)

where sign (x) = −1 for x ≤ 0 and sign (x) = 1 for x > 0 . Then β = (βt)t≥0

is a standard Brownian motion, and Tanaka’s formula (page 67) states:

|Bt| = βt + Lt (18.3.34)

where L = (Lt)t≥0 is the local time process of B at 0 given by

Lt = max0≤s≤t

(−βs). (18.3.35)

Thus τ2,ε is equally distributed as

σ = inf

t > 0 : max0≤s≤t

(−βs) − 2(−βt) ≥ ε

(18.3.36)

= inf

t > 0 : |Bt| ≥ ε−βt

.

Note that σ is an (Fβt ) -stopping time, and since Fβ

t = F |B|t ⊂ FB

t , we seethat σ is an (FB

t ) -stopping time too. Assuming now that Eτ2,ε which equalsEσ is finite, by the standard Wald identity for Brownian motion (see (3.2.6)) weobtain

Eσ = E |Bσ|2 = E (ε − βσ)2 = ε2 − 2εEβσ + E |βσ|2 = ε2 + Eσ. (18.3.37)

Hence we see that ε must be zero. This completes the proof of (18.3.32).

Notes. Theorem 18.4 extends a result of Wang [221] who showed that theexpectation of τ∗ is finite. Stopping times of the form τ∗ have been studied by anumber of people. Instead of going into a historical exposition on this subject wewill refer the interested reader to the paper by Azema and Yor [6] where furtherdetails in this direction can be found. One may note however that as long asone is concerned with the expectation of such a stopping time only, the Laplacetransform method (developed in some of these works) may have the drawback ofcomputational complexity in comparison with the method used above (see also[154] for a related result).

18.4. Further examples

The result of Theorem 18.1 and Corollary 18.2 can also be obtained directly fromthe maximality principle (see Section 13). We will present this line of argumentthrough several examples.


Example 18.7. (The Doob inequality) Consider the optimal stopping problem(18.2.37) being the same as the optimal stopping problem (13.1.4) with Xt =|Bt +x|p and c(x) = cx(p−2)/p for p > 1 . Then X is a non-negative diffu-sion having 0 as an instantaneously-reflecting regular boundary point, and theinfinitesimal generator of X in (0,∞) is given by the expression

LX =p(p−1)

2x1−2/p ∂

∂x+

p2

2x2−2/p ∂2

∂x2. (18.4.1)

The equation (13.2.22) takes the form

g′(s) =pg1/p(s)

2c(s1/p − g1/p(s)

) , (18.4.2)

and its maximal admissible solution of (18.4.2) is given by

g∗(s) = αs (18.4.3)

where 0 < α < 1 is the maximal root (out of two possible ones) of the equation

α − α1−1/p +p

2c= 0. (18.4.4)

It can be verified that equation (18.4.4) admits such a root if and only if c ≥pp+1/2(p− 1)(p−1) . Then by the result of Corollary 13.3, upon using (13.2.65)and letting c ↓ pp+1/2(p−1)(p−1) , we get

E(

max0≤t≤τ

|Bt + x|p)≤(

p

p−1

)p

E |Bτ + x|p − p

p− 1xp (18.4.5)

for all stopping times τ of B such that Eτp/2 < ∞ . The constants (p/(p−1))p

and p/(p− 1) are best possible, and equality in (18.4.5) is attained in the limitthrough the stopping times τ∗=inft>0 : Xt≤αSt when c ↓ pp+1/2(p−1)(p−1) .These stopping times are pointwise the smallest possible with this property, andthey satisfy Eτ

p/2∗ < ∞ if and only if c > pp+1/2(p−1)(p−1) . For more informa-

tion and remaining details we refer to [80].

Example 18.8. (Further Doob type bounds) The inequality (18.4.5) can be furtherextended using the same method as follows (for simplicity we state this extensiononly for x = 0 ):

E(

max0≤t≤τ

|Bt|p)≤ γ∗

p,q

(E

∫ τ

0

|Bt|q−1 dt

)p/(q+1)

(18.4.6)

for all stopping times τ of B , all 0 0 , with the bestpossible value for the constant γ∗

p,q being equal

γ∗p,q = (1+κ)

(s∗κκ

)1/(1+κ)

(18.4.7)


where we set κ = p/(q−p+1) , and s∗ is the zero point of the maximal admissiblesolution s → g∗(s) of

g′(s) =p g(1−q/p)(s)

2(s1/p − g1/p(s))(18.4.8)

satisfying 0 < g∗(s) < s for all s > s∗ . (This solution is also characterized byg∗(s)/s → 1 for s → ∞ .) The equality in (18.4.6) is attained at the stoppingtime τ∗ = inf t > 0 : Xt = g∗(St) which is pointwise the smallest possible withthis property. In the case p = 1 the closed form for s → g∗(s) is given by

s exp(− 2

pqgq∗(s)

)+

2p

∫ g∗(s)

0

tq exp(− 2

pqtq)

dt =(

pq

2

)1/q

Γ(

q+1q

)(18.4.9)

for s ≥ s∗ . This, in particular, yields

γ∗1,q =

(q(1+q)

2

)1/(1+q)(Γ(2 +

1q

))q/(1+q)

(18.4.10)

for all q > 0 . In the case p = 1 no closed form for s → g∗(s) seems to exist.For more information and remaining details in this direction, as well as for theextension of inequality (18.4.6) to x = 0 , we refer to [158] (see also [156]). To givea more familiar form to the inequality (18.4.6), note by Ito’s formula (page 67)and the optional sampling theorem (page 60) that

E

(∫ τ

0

|Bt|q−1 dt

)=

2q(q+1)

E |Bτ |q+1 (18.4.11)

whenever τ is a stopping time of B satisfying E (τ (q+1)/2) < ∞ for q > 0 .Hence we see that the right-hand side in (18.4.6) is the well-known Doob bound(see (C4) on page 62). The advantage of formulation (18.4.6) lies in its validity forall stopping times.

Notes. While the inequality (18.4.6) (with some constant γp,q > 0 ) can bederived quite easily, the question of its sharpness has gained interest. The case p =1 was treated independently by Jacka [103] (probabilistic methods) and Gilat [75](analytic methods) who both found the best possible value γ∗

1,q for q > 0 . Thisin particular yields γ∗

1,1 =√

2 which was independently obtained by Dubins andSchwarz [44], and later again by Dubins, Shepp and Shiryaev [45] who studied themore general case of Bessel processes. (A simple probabilistic proof of γ∗

1,1 =√

2 isgiven in [78] — see Subsection 16.3 above). The Bessel processes results are furtherextended in [150]. In the case p = 1+q with q > 0 , the inequality (18.4.6) reducesto the Doob maximal inequality (18.4.5). The best values γ∗

p,q in (18.4.6) and thecorresponding optimal stopping times τ∗ for all 0 0 aregiven in [158]. A novel fact about (18.4.5) and (18.4.6) disclosed is that the optimalτ∗ from (13.2.58) is pointwise the smallest possible stopping time at which the


equalities in (18.4.5) (in the limit) and in (18.4.6) can be attained. The resultsabout (18.4.5) and (18.4.6) extend to all non-negative submartingales. This canbe obtained by using the maximal embedding result of Jacka [101] (for details see[80] and [158]).

Example 18.9. (A maximal inequality for geometric Brownian motion) Considerthe optimal stopping problem (13.1.4) where X is geometric Brownian motionand c(x) ≡ c . Recall that X is a non-negative diffusion having 0 as an entranceboundary point, and the infinitesimal generator of X in (0,∞) is given by theexpression

LX = ρx∂

∂x+

σ2

2x2 ∂2

∂x2(18.4.12)

where ρ ∈ R and σ > 0 . The process X may be realized as

Xt = x exp(

σBt +(ρ− σ2

2

)t

)(18.4.13)

with x ≥ 0 . The equation (13.2.22) takes the form

g′(s) =∆ σ2 g∆+1(s)

2 c (s∆ − g∆(s))(18.4.14)

where ∆ = 1 − 2ρ/σ2 . By using Picard’s method of successive approximations itis possible to prove that for ∆ > 1 the equation (18.4.14) admits the maximaladmissible solution s → g∗(s) satisfying

g∗(s) ∼ s1−1/∆ (18.4.15)

for s → ∞ (see Figure IV.12 and [81] for further details). There seems to be noclosed form for this solution. In the case ∆ = 1 it is possible to find the generalsolution of (18.4.14) in a closed form, and this shows that the only non-negativesolution is the zero function (see [81]). By the result of Corollary 13.3 we mayconclude that the value function (13.1.4) is finite if and only if ∆ > 1 (note thatanother argument was used in [81] to obtain this equivalence), and in this case itis given by

V∗(x, s) (18.4.16)

=

⎧⎨⎩2c

∆2σ2

(( x

g∗(s)

)∆− log

( x

g∗(s)

)∆− 1

)+ s if g∗(s) < x ≤ s,

s if 0 < x ≤ g∗(s).

The optimal stopping time is given by (13.2.58) with s∗ = 0 . By using explicitestimates on s → g∗(s) from (18.4.15) in (18.4.16), and then minimizing over all


c > 0 , we obtain

E

(max

0≤t≤τexp

(σBt +

(ρ− σ2

2

)t))

(18.4.17)

≤ 1 − σ2

2ρ+

σ2

2ρexp

(− (σ2− 2ρ)2

2σ2Eτ − 1

)for all stopping times τ of B . This inequality extends the well-known estimatesof Doob in a sharp manner from deterministic times to stopping times. For moreinformation and remaining details we refer to [81]. Observe that the cost functionc(x) = cx in the optimal stopping problem (13.1.4) would imply that the maximaladmissible solution of (13.2.22) is linear. This shows that such a cost functionbetter suits the maximum process and therefore is more natural. Explicit formulaefor the value function, and the maximal inequality obtained by minimizing overc>0 , are also easily derived in this case from the result of Corollary 13.3.

19. Hardy–Littlewood inequalities

The main purpose of this section (following [83]) is to derive and examine sharpversions of the L logL -inequality of Hardy and Littlewood for one-dimensionalBrownian motion which may start at any point.


Let B = (Bt)t≥0 be a standard Brownian motion defined on a probability space(Ω,F , P) such that B0 = 0 under P . The L logL -inequality of Hardy andLittlewood [90] formulated in the optimal stopping setting of B states:

E(

max0≤t≤τ

|Bt|)≤ C1

(1+E

(|Bτ | log+|Bτ |))

(19.1.1)

for all stopping times τ of B with Eτr < ∞ for some r > 1/2 , where C1 isa universal numerical constant (see [40]). The analogue of the problem consideredby Gilat [74] may be stated as follows: Determine the best value for the constantC1 in (19.1.1), and find the corresponding optimal stopping time (the one at whichequality in (19.1.1) is attained).

It is well known that the inequality (19.1.1) remains valid if the plus sign isremoved from the logarithm sign, so that we have

E(

max0≤t≤τ

|Bt|)≤ C2

(1+E

(|Bτ | log |Bτ |))

(19.1.2)

for all stopping times τ of B with Eτr < ∞ for some r > 1/2 , where C2 is auniversal numerical constant. The problem about (19.1.1) stated above extends in

Section 19. Hardy–Littlewood inequalities 273

exactly the same form to (19.1.2). It turns out that this problem is somewhat eas-ier, however, both problems have some new features which make them interestingfrom the standpoint of optimal stopping theory.

To describe this in more detail, note that in both cases of (19.1.1) and (19.1.2)we are given an optimal stopping problem with the value function

V = supτ

E(Sτ − cF (Xτ )

)(19.1.3)

where c > 0 , and in the first case F (x) = x log+ x , while in the second caseF (x) = x log x , with Xt = |Bt| and St = max0≤r≤t |Br| . The interesting featureof the first problem is that the cost x → cF (x) is somewhat artificially set tozero for x ≤ 1 , while in the second problem the cost is not monotone all overas a function of time. Moreover, in the latter case the Ito formula (page 67) isnot directly applicable to F (Xt) , due to the fact that F ′′(x) = 1/x so that∫ τ

0 F ′′(Xt) dt = ∞ for all stopping times τ for which Xτ = 0 P-a.s. Thismakes it difficult to find a “useful” increasing functional t → It with the sameexpectation as the cost (the fact which enables one to write down a differentialequation for the value function).

Despite these difficulties one can solve both optimal stopping problems andin turn get solutions to (19.1.1) and (19.1.2) as consequences. The first problem issolved by guessing and then verifying that the guess is correct (cf. Theorem 19.1and Corollary 19.2). The second problem is solved by a truncation method (cf.Theorem 19.3 and Corollary 19.4). The obtained results extend to all non-negativesubmartingales (Corollary 19.6).


In this subsection we present the main results and proofs. Since the problem(19.1.2) is somewhat easier, we begin by stating the main results in this direction(Theorem 19.1). The facts obtained in the proof will be used later (Theorem 19.3)in the solution for the problem (19.1.1). It is instructive to compare these twoproofs and notice the essential argument needed to conclude in the latter (notethat dF /dx from the proof of Theorem 19.1 is continuous at 1/e , while dF+/dxfrom the proof of Theorem 19.3 is discontinuous at 1 , thus bringing the localtime of X at 1 into the game — this is the crucial difference between these twoproblems). The Gilat paper [74] finishes with a concluding remark where a gapbetween the L log L and L log+ L case is mentioned. The discovery of the exactsize of this gap is stressed to be the main point of his paper. The essential argumentmentioned above offers a probabilistic explanation for this gap and gives its exactsize in terms of optimal stopping strategies (compare (19.2.2) and (19.2.30) andnotice the middle term in (19.2.45) in comparison with (19.2.28)).


Theorem 19.1. Let B = (Bt)t≥0 be standard Brownian motion started at zerounder P . Then the following inequality is satisfied :

E(

max0≤t≤τ

|Bt|)≤ c2

e(c−1)+ c E

(|Bτ | log |Bτ |)

(19.2.1)

for all c > 1 and all stopping times τ of B satisfying Eτr < ∞ for somer > 1/2 . This inequality is sharp: equality is attained at the stopping time

σ∗ = inf

t>0 : St ≥ v∗, Xt = αSt

(19.2.2)

where v∗ = c/e(c− 1) and α = (c− 1)/c for c > 1 with Xt = |Bt| and St =max0≤r≤t |Br| .Proof. Given c > 1 consider the optimal stopping problem

V (x, s) = supτ

Ex,s

(Sτ − c F (Xτ )

)(19.2.3)

where F (x) = x log x for x ∈ R , Xt = |Bt+x| and St = s ∨ max 0≤r≤t |Br+x|for 0 ≤ x ≤ s . Note that the (strong) Markov process (X, S) starts at (x, s)under P := Px,s .

The main difficulty in this problem is that we cannot apply Ito’s formula(page 67) to F (Xt) . We thus truncate F (x) by setting F (x) = F (x) for x ≥ 1/e

and F (x) = −1/e for 0 ≤ x ≤ 1/e . Then F ∈ C1 and F ′′ exists everywherebut 1/e . Since the time spent by X at 1/e is of Lebesgue measure zero, settingF ′′(1/e) := e , by the Ito–Tanaka–Meyer formula (page 68) we get

F (Xt) = F (x) +∫ t

0

F ′(Xr) dXr +12

∫ t

0

F ′′(Xr) d〈X, X〉r (19.2.4)

= F (x) +∫ t

0

F ′(Xr) d(βr+r) +12

∫ t

0

F ′′(Xr) dr

= F (x) + Mt +12

∫ t

0

F ′′(Xr) dr

where β = (βt)t≥0 is a standard Brownian motion, = (t)t≥0 is the local timeof X at zero, and Mt =

∫ t

0F ′(Xr) dβr is a continuous local martingale, due

to F ′(0) = 0 and the fact that dr is concentrated at t : Xt = 0 . By theoptional sampling theorem (page 60) and the Burkholder–Davis–Gundy inequalityfor continuous local martingales (see (C5) on page 63), we easily find

Ex,sF (Xτ ) = F (x) +12

Ex,s

(∫ τ

0

F ′′(Xt) dt

)(19.2.5)

for all stopping times τ of B satisfying Ex,sτr < ∞ for some r > 1/2 . By

(19.2.5) we see that the value function V (x, s) from (19.2.3) should be identical


to V (x, s) := W (x, s)− c F (x) where

W (x, s) = supτ

Ex,s

(Sτ − c

2

∫ τ

0

F ′′(Xt) dt

)(19.2.6)

for 0 ≤ x ≤ s . For this, note that clearly V (x, s) ≤ V (x, s) , so if we provethat the optimal stopping time σ∗ in (19.2.6) satisfies Xσ∗ ≥ 1/e , then due toEx,s(Sσ∗ − cF (Xσ∗)) = Ex,s(Sσ∗ − cF (Xσ∗)) this will show that V (x, s) = V (x, s)with σ∗ being optimal in (19.2.3) too. In the rest of the proof we solve the optimalstopping problem (19.2.6) and show that the truncation procedure indicated aboveworks as desired.

Supposing that the supremum in (19.2.6) is attained at the exit time ofdiffusion (X, S) from an open set, we see (cf. Section 7) that the value functionW (x, s) should satisfy

LXW (x, s) =c

2F ′′(x) (g∗(s) < x < s) (19.2.7)

where LX = ∂2/2∂x2 is the infinitesimal operator of X in (0,∞) and s → g∗(s)is the optimal stopping boundary to be found. To solve (19.2.7) in an explicit form,we shall make use of the following boundary conditions:

W (x, s)∣∣x=g∗(s)+


∂W

∂x(x, s)

∣∣∣x=g∗(s)+

= 0 (smooth fit), (19.2.9)

∂W

∂s(x, s)

∣∣∣x=s−


Note that (19.2.7)–(19.2.10) forms a problem with free boundary s → g∗(s) . Thegeneral solution of (19.2.7) is given by

W (x, s) = C(s)x + D(s) + c F (x) (19.2.11)

where s → C(s) and s → D(s) are unknown functions. By (19.2.8) and (19.2.9)we find

C(s) = −c F ′(g∗(s)), (19.2.12)

D(s) = s + c g∗(s)F ′(g∗(s)) − c F (g∗(s)). (19.2.13)

Inserting (19.2.12) and (19.2.13) into (19.2.11) we obtain

W (x, s) = s − c(x− g∗(s)

)F ′(g∗(s)) − c F (g∗(s)) + cF (x) (19.2.14)

for g∗(s) ≤ x ≤ s . Clearly W (x, s) = s for 0 ≤ x ≤ g∗(s) . Finally, by (19.2.10)we find that s → g∗(s) should satisfy

g′∗(s) F ′′(g∗(s))(s− g∗(s)

)=

1c

(19.2.15)


for s > 0 . Note that this equation makes sense only for F ′′(g∗(s)) > 0 or equiv-alently g∗(s) ≥ 1/e when it reads as follows:

g′∗(s)(

s

g∗(s)− 1

)=

1c

(19.2.16)

for s ≥ v∗ where g∗(v∗) = 1/e . Next observe that (19.2.16) admits a linearsolution of the form

g∗(s) = αs (19.2.17)

for s ≥ v∗ where α = (c− 1)/c . (Note that this solution is the maximal admissiblesolution to either (19.2.15) or (19.2.16). This is in accordance with the maximalityprinciple (see Section 13) and is the main motivation for the candidate (19.2.17).)This in addition indicates that the formula (19.2.14) will be valid only if s ≥ v∗ ,where v∗ is determined from g∗(v∗) = 1/e , so that

v∗ = c/e(c−1). (19.2.18)

The corresponding candidate for the optimal stopping time is

σ∗ = inf

t > 0 : Xt ≤ g∗(St)

(19.2.19)

where s → g∗(s) is given by (19.2.17) for s ≥ v∗ . The candidate for the valuefunction (19.2.6) given by the formula (19.2.14) for g∗(s) ≤ x ≤ s with s ≥ v∗ willbe denoted by W∗(x, s) in the sequel. Clearly W∗(x, s) = s for 0 ≤ x ≤ g∗(s)with s ≥ v∗ . In the next step we verify that this candidate equals the valuefunction (19.2.6), and that σ∗ from (19.2.19) is the optimal stopping time.

To verify this, we shall apply the (natural extension of the) Ito–Tanaka–Meyer formula (page 68) to W∗(Xt, St) . Since F ′′ ≥ 0 , this gives

W∗(Xt, St) = W∗(x, s) +∫ t

0

∂W∗∂x

(Xr, Sr) dXr (19.2.20)

+∫ t

0

∂W∗∂s

(Xr, Sr) dSr +12

∫ t

0

∂2W∗∂x2


≤ W∗(x, s) +∫ t

0

∂W∗∂x

(Xr, Sr) d(βr+r) +c

2

∫ t

0

F ′′(Xr) dr

= W∗(x, s) + Mt +c

2

∫ t

0

F ′′(Xr) dr

with Mt =∫ t

0(∂W∗/∂x)(Xr, Sr) dβr being a continuous local martingale for t ≥

0 , where we used that dSr equals zero for Xr < Sr , so that by (19.2.10) theintegral over dSr is equal to zero, while due to (∂W∗/∂x)(0, s) = 0 the integralover dr is equal to zero too. Now since W∗(x, s) ≥ s for all x ≥ g∗(s) (withequality if x = g∗(s) ) it follows that

Sτ − c

2

∫ τ

0

F ′′(Xt) dt ≤ W∗(x, s) + Mτ (19.2.21)


for all (bounded) stopping times τ of B with equality (in (19.2.20) as well) ifτ = σ∗ . Taking the expectation on both sides we get

Ex,s

(Sτ − c

2

∫ τ

0

F ′′(Xt) dt

)≤ W∗(x, s) (19.2.22)

for all (bounded) stopping times τ of B satisfying

Ex,sMτ = 0 (19.2.23)

with equality in (19.2.22) under the validity of (19.2.23) if τ = σ∗ . We first showthat (19.2.23) holds for all stopping times τ of B satisfying Ex,s

√τ < ∞ . For

this, we compute

Ex,s

(∫ τ

0

(∂W∗∂x

(Xr, Sr))2

dr

)1/2

(19.2.24)

= c Ex,s

(∫ τ

0

log2

(Xr

g∗(Sr)

)1g∗(Sr)≤Xr dr

)1/2

= c log( 1

α

)Ex,s

(√τ)

so that (19.2.23) follows by the optional sampling theorem (page 60) and theBurkholder–Davis–Gundy inequality for continuous local martingales (see (C5)on page 63) whenever Ex,s

√τ < ∞ . Moreover, it is well known (see [221]) that

Ex,sσr∗ < ∞ for all r < c/2 . In particular Ex,s

√σ∗ < ∞ , so that (19.2.23) holds

for τ = σ∗ , and thus we have equality in (19.2.22) for τ = σ∗ . This completesthe proof that the value function (19.2.6) equals W∗(x, s) for 0 ≤ x ≤ s withs ≥ v∗ , and that σ∗ is the optimal stopping time.

Note that Xσ∗ ≥ 1/e so that by (19.2.14) and the remark following (19.2.6)we get

V (x, s) = V (x, s) = W (x, s) − c F (x) (19.2.25)

= s − c x − c x log g∗(s) + c g∗(s)

for all g∗(s) ≤ x ≤ s with s ≥ v∗ , where g∗(s) = αs with α = (c− 1)/c andv∗ = c/e(c−1) . To complete the proof it remains to compute the value functionV (x, s) for 0 ≤ x ≤ s with 0 ≤ s < v∗ . A simple observation which motivatesour formal move in this direction is as follows.

The best point to stop in the region 0 ≤ x ≤ s < v∗ would be (1/e , s)with s as close as possible to v∗ , since the cost function x → cx log x attainsits minimal value at 1/e . The value function V equals (tends) to v∗+c/e if theprocess (X, S) is started and stopped at (1/e , s) with s being equal (tending)to v∗ . However, it is easily seen that the value function V (x, s) computed abovefor s ≥ v∗ satisfies V (v∗, v∗) = v∗+c/e = c2/e(c−1) . This indicates that in the


region 0 ≤ x ≤ s < v∗ there should be no point of stopping. This can be formallyverified as follows. Given a (bounded) stopping time τ of B , define τ ′ to be τon τ ≥ τv∗ and σ∗ on τ < τv∗ . Then τ ′ is a stopping time of B , andclearly (Xτ ′ , Sτ ′) does not belong to the region 0 ≤ x ≤ s < v∗ . Moreover, bythe strong Markov property,

Ex,s

(Sτ ′ − cF (Xτ ′)

)(19.2.26)

= Ex,s

((Sτ − cF (Xτ )) 1τ≥τv∗

)+ Ex,s

((Sσ∗ − cF (Xσ∗)) 1τ<τv∗

)= Ex,s

((Sτ − cF (Xτ )) 1τ≥τv∗

)+ Ex,s

(Ev∗,v∗(Sσ∗ − cF (Xσ∗)

)1τ<τv∗

)= Ex,s

((Sτ − cF (Xτ )) 1τ≥τv∗

)+ V (v∗, v∗)Px,s

(τ < τv∗

)≥ Ex,s

(Sτ − cF (Xτ )

)for all 0 ≤ x ≤ s with 0 ≤ s < v∗ , where τv∗ = inf t > 0 : Xt = v∗ . ThusV (x, s) = V (v∗, v∗) = c2/e(c−1) for all 0 ≤ x ≤ v∗ , and noting that σ′∗ in thiscase equals σ∗ from (19.2.2), the proof is complete.

The result of Theorem 19.1 extends to the case when Brownian motion Bstarts at points different from zero.

Corollary 19.2. Let B = (Bt)t≥0 be standard Brownian motion started at zerounder P . Then the following inequality is satisfied :

E(

max0≤t≤τ

|Bt+x|)≤ V (x; c) + cE

(|Bτ +x| log |Bτ +x|) (19.2.27)

for all c > 1 and all stopping times τ of B satisfying Eτr < ∞ for somer>1/2 , where

V (x; c) =

c2

e(c − 1) if 0 ≤ x ≤ v∗,

cx log cx(c − 1) if x ≥ v∗

(19.2.28)

with v∗ = c/e(c−1) . This inequality is sharp: for each c > 1 and x ≥ 0 givenand fixed, equality in (19.2.27) is attained at the stopping time σ∗ defined in(19.2.2) with Xt = |Bt+x| and St = max0≤r≤t |Br + x| .

Proof. It follows from the proof of Theorem 19.1. Note that V (x; c) equals V (x, x)in the notation of this proof, so that the explicit expression for V (x; c) is givenin (19.2.25).

In the next theorem we present the solution in the L log+ L -case. The firstpart of the proof (i.e. the proof of (19.2.29)) is identical to the first part of theproof of Theorem 19.1, and therefore it is omitted.


Theorem 19.3. Let B = (Bt)t≥0 be standard Brownian motion started at zerounder P . Then the following inequality is satisfied :

E(

max0≤t≤τ

|Bt|)≤ 1 +

1ec(c− 1)

+ c E(|Bτ | log+ |Bτ |

)(19.2.29)

for all c > 1 and all stopping times τ of B satisfying Eτr < ∞ for somer > 1/2 . This inequality is sharp: equality is attained at the stopping time

τ∗ = inf

t>0 : St ≥ u∗, Xt = 1 ∨ αSt

(19.2.30)

where u∗ = 1 + 1/ec(c− 1) and α = (c − 1)/c for c > 1 with Xt = |Bt| andSt = max0≤r≤t |Br| .Proof. Given c > 1 consider the optimal stopping problem

V+(x, s) = supτ

Ex,s

(Sτ − cF+(Xτ )

)(19.2.31)

where F+(x) = x log+ x , Xt = |Bt +x| and St = s ∨ max 0≤r≤t |Br +x| for0 ≤ x ≤ s . Since F+(x) = F (x) for all x ≥ 1 , it is clear that V+(x, s) coincideswith the value function V (x, s) from (19.2.3) (with the same optimal stoppingtime given by either (19.2.2) or (19.2.30)) for 0 ≤ x ≤ s with s ≥ s∗ , wheres∗ is determined from g∗(s∗) = 1 with g∗(s) = αs and α = (c− 1)/c , so thats∗ = 1/α = c/(c− 1) . It is also clear that the process (X, S) cannot be optimallystopped at some τ with Xτ < 1 since F+(x) = 0 for x < 1 . This shows thatV+(0, 0) = V+(x, s) = V+(1, 1) for all 0 ≤ x ≤ s ≤ 1 . So it remains to computethe value function V+(x, s) for 0 ≤ x ≤ s with 1 ≤ s < s∗ . This evaluation isthe main content of the proof. We begin by giving some intuitive arguments whichare followed by a rigorous justification.

The best place to stop in the region 0 ≤ x ≤ s with 1 ≤ s ≤ s∗ is clearlyat (1, s) , so that there should exist a point 1 ≤ u∗ ≤ s∗ such that the process(X, S) should be stopped at the vertical line (1, s) : u∗ ≤ s ≤ s∗ , as well as tothe left from it (if started there ). We also expect that V+(u∗, u∗) = u∗ (since wedo not stop at (1, u∗− ε) where the value function V+ would be equal u∗− ε forε > 0 as small as desired). Clearly, the value function V+ should be constant inthe region 0 ≤ x ≤ s ≤ u∗ (note that there is no running cost), and then (whenrestricted to the diagonal x = s for u∗ ≤ s ≤ s∗ ) it should decrease. Note from(19.2.25) that V+(s∗, s∗) = V (s∗, s∗) = 0 . So let us try to determine such a pointu∗ .

Thus we shall try to compute

sup1≤s≤s∗

Es,s

(Sτ ′∗ − cF+(Xτ ′∗)

)(19.2.32)

where τ ′∗ = τ ′∗(s) = inf t > 0 : Xt = 1 ∨ αSt . For this, note by the strongMarkov property (and V+(s∗, s∗) = 0 ) that if τ ′

∗ is to be an optimal stopping


time (for some s = u∗ ), we should have

V+,∗(x, s) := Ex,s

(Sτ ′∗ − cF+(Xτ ′∗)

)(19.2.33)

= Ex,s

(Sτ ′∗ 1τ ′∗<τs∗

)+ Ex,s

((Sτ ′∗ − c F+(Xτ ′∗)) 1τ ′∗≥τs∗

)= Ex,s

(Sτ ′∗ 1τ ′∗<τs∗

)+ Ex,s

(Es∗,s∗(Sτ ′∗ − c F+(Xτ ′∗)) 1τ ′∗≥τs∗

)= Ex,s

(Sτ ′∗ 1τ ′∗<τs∗

)for all 1 ≤ x ≤ s ≤ s∗ where τs∗ = inf t> 0 : Xt = s∗ . Note further that τ ′

∗(on τ ′

∗ < τs∗ ) may be viewed as the exit time of (X, S) from an open set, sothat V+,∗(x, s) should satisfy

LXV+,∗(x, s) = 0 (1 < x < s), (19.2.34)

V+,∗(x, s)∣∣x=1


∂V+,∗∂s

(x, s)∣∣∣x=s−

= 0 (normal reflection), (19.2.36)

V+,∗(x, s)∣∣∣x=s=s∗

= 0 (strong Markov property) (19.2.37)

for 1 ≤ x ≤ s ≤ s∗ . System (19.2.34)–(19.2.37) has a unique solution given by

V+,∗(x, s) = s + (1−x) log(s−1) + K(x−1) (19.2.38)

for 1 ≤ x ≤ s ≤ s∗ where

K = −c − log(c− 1). (19.2.39)

Solving (∂V+,∗/∂s)(s, s) = 0 we find the point at which the supremum in (19.2.32)is to be attained:

u∗ = 1 + eK = 1 + 1/ec(c− 1). (19.2.40)

Thus the candidate V+,∗(x, s) for the value function (19.2.31) is given by (19.2.38)for all 1 ≤ x ≤ s with u∗ ≤ s ≤ s∗ . Clearly we have to put V+,∗(x, s) = s for0 ≤ x ≤ 1 with u∗ ≤ s ≤ s∗ . Note moreover that V+,∗(x, s) = V+,∗(u∗, u∗) =u∗ = 1+1/ec(c− 1) for all 0 ≤ x ≤ s ≤ u∗ as suggested above. So if we showin the sequel that this candidate is indeed the value function with the optimalstopping time τ∗ = τ ′

∗(u∗) , the proof of the theorem will be complete.

That there should be no point of stopping in the region 0 ≤ x ≤ s ≤ u∗ isverified in exactly the same way as in (19.2.26). So let us concentrate on the casewhen u∗ ≤ s ≤ s∗ . To apply Ito’s formula (page 67) we shall redefine V+,∗(x, s)for x < 1 by (19.2.38). This extension will be denoted by V+,∗(x, s) . ObviouslyV+,∗ ∈ C2 and V+,∗(x, s) = V+,∗(x, s) for 1 ≤ x ≤ s with u∗ ≤ s ≤ s∗ . ApplyingIto’s formula (page 67) to V+,∗(Xt, St) and noting that (∂V+,∗/∂x)(0, s) ≤ 0 for


u∗ ≤ s ≤ s∗ (any such C2 -extension would do) we get

V+,∗(Xt, St) = V+,∗(x, s) +∫ t

0

∂V+,∗∂x

(Xr, Sr) dXr (19.2.41)

+∫ t

0

∂V+,∗∂s

(Xr, Sr) dSr +12

∫ t

0

∂2V+,∗∂x2


= V+,∗(x, s) +∫ t

0

∂V+,∗∂x

(Xr, Sr) d(βr+r)

= V+,∗(x, s) + Mt +∫ t

0

∂V+,∗∂x

(0, Sr) dr ≤ V+,∗(x, s) + Mt

for all 0 ≤ x ≤ s with u∗ ≤ s ≤ s∗ where Mt =∫ t

0 (∂V+,∗/∂x)(Xr, Sr) dβr is acontinuous local martingale. Moreover, hence we find

V+,∗(Xτ , Sτ ) ≤ V+,∗(x, s) + Mτ (19.2.42)

for all stopping times τ of B with equality if τ ≤ τ∗ . Due to ey ≤ ey it is easilyverified that V+,∗(x, s) = V+,∗(x, s) ≥ s− cF+(x) for 1 ≤ x ≤ s with u∗ ≤ s ≤ s∗(with equality if x = 1 ). Now given a stopping time τ of B , define τ ′ to be τon Xτ ≥ 1 and τ1 on Xτ < 1 , where τ1 = inf t > 0 : Xt = 1 . Thenτ ′ is a new stopping time of B , and by (19.2.42) and the remark following it, weclearly have

Ex,s

(Sτ − c F+(Xτ )

)= Ex,s

(Sτ ′ − c F+(Xτ ′)

)(19.2.43)

≤ Ex,s

(V+,∗(Xτ ′ , Sτ ′)

) ≤ V+,∗(x, s) + Ex,sMτ ′ = V+,∗(x, s)

for all 1 ≤ x ≤ s with u∗ ≤ s ≤ s∗ whenever Ex,s(τ ′)r < ∞ for some r > 1/2with equalities if τ = τ∗ (recall that Ex,sτ

r∗ <∞ for all r < c/2 ). The proof ofoptimality of the stopping time τ∗ defined in (19.2.30) above is complete.

The result of Theorem 19.3 also extends to the case when Brownian motionB starts at points different from zero.

Corollary 19.4. Let B = (Bt)t≥0 be standard Brownian motion started at zerounder P . Then the following inequality is satisfied :

E(

max0≤t≤τ

|Bt+x|)≤ V+(x; c) + cE

(|Bτ +x| log+|Bτ +x|) (19.2.44)

for all c > 1 and all stopping times τ of B satisfying Eτr < ∞ for somer > 1/2 , where

V+(x; c) =

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩1 + 1/ec(c−1) if 0 ≤ x ≤ u∗,

x + (1−x) log(x− 1)−(c + log(c−1))(x− 1) if u∗ ≤ x ≤ s∗,

cx log(c/x(c−1)

)if x ≥ s∗

(19.2.45)


with u∗ = 1 + 1/ec(c−1) and s∗ = c/(c− 1) . This inequality is sharp: for eachc > 1 and x ≥ 0 given and fixed, equality in (19.2.44) is attained at the stoppingtime τ∗ defined in (19.2.30) with Xt = |Bt+x| and St = max0≤r≤t |Br+x| .Proof. It follows from the proof of Theorem 19.3. Note that V+(x; c) equalsV+(x, x) in the notation of this proof, so that the explicit expression for V+(x; c)is given in (19.2.38).

Remark 19.5. The distribution law of Xτ∗ and Sτ∗ from Theorem 19.1 (Corollary19.2) and Theorem 19.3 (Corollary 19.4) can be computed explicitly (see [6]). Forthis one can use the fact that H(St) − (St −Xt)H ′(St) is a (local) martingalebefore X hits zero for sufficiently many functions H . We will omit further details.

Due to the universal role of Brownian motion in this context, the inequalities(19.2.27) and (19.2.44) extend to all non-negative submartingales. This can beobtained by using the maximal embedding result of Jacka [101].

Corollary 19.6. Let X = (Xt)t≥0 be a non-negative cadlag (right continuous withleft limits) uniformly integrable submartingale started at x ≥ 0 under P . Let X∞denote the P-a.s. limit of X for t → ∞ (which exists by (B1) on page 61). Thenthe following inequality is satisfied :

E supt>0

Xt ≤ WG(x; c) + c EG(X∞) (19.2.46)

for all c > 1 , where G(y) is either y log y and in this case WG(x; c) is given by(19.2.28), or G(y) is y log+ y and in this case WG(x; c) is given by (19.2.45).This inequality is sharp.

Proof. Given such a submartingale X = (Xt)t≥0 satisfying EG(X∞) < ∞ , anda Brownian motion B = (Bt)t≥0 started at X0 = x under Px , by the resultof Jacka [101] we know that there exists a stopping time τ of B , such that|Bτ | ∼ X∞ and P supt≥0 Xt ≥ λ ≤ Pxmax0≤t≤τ |Bt| ≥ λ for all λ > 0 ,with (Bt∧τ )t≥0 being uniformly integrable. The inequality (19.2.46) then eas-ily follows from Corollary 19.2 and Corollary 19.4 by using the integration byparts formula. Note that by the submartingale property of (|Bt∧τ |)t≥0 we havesupt≥0 ExG(|Bt∧τ |) = ExG(|Bτ |) . This completes the proof.

Notes. This section is motivated by the paper of Gilat [74] where he settlesa question raised by Dubins and Gilat [43], and later by Cox [32], on the L logL -inequality of Hardy and Littlewood. Instead of recalling his results in the analyticframework of the Hardy–Littlewood theory, we shall rather refer the reader toGilat’s paper [74] where a splendid historical exposition on the link between theHardy–Littlewood theory and probability (martingale theory) can be found too.Despite the fact that Gilat’s paper finishes with a comment on the use of his resultin the martingale theory, his proof is entirely analytic. The main aim of this section


is to present a new probabilistic solution to this problem. While Gilat’s result givesthe best value for C1 , it does not detect the optimal stopping strategy τ∗ in(19.1.1), but rather gives the distribution law of Bτ∗ and Sτ∗ (see Remark 19.5).In contrast to this, the proof above does both, and together with the extensionto the case when B starts at any point, this detection (of the optimal stoppingstrategy) forms the principal result of the section.

19.3. Further examples

The result of Theorem 19.1 and Theorem 19.3 can also be obtained directly fromthe maximality principle (see Section 13). We will illustrate this line of argumentby one more example.

Example 19.7. (A sharp integral inequality of the L log L-type) Consider the opti-mal stopping problem (13.1.4) with Xt = |Bt +x| and c(x) = c/(1+x) for x ≥ 0and c > 0 . Then X is a non-negative diffusion having 0 as an instantaneously-reflecting regular boundary point, and the infinitesimal generator of X in (0,∞)is given by (18.4.1) with p = 1 . The equation (13.2.22) takes the form

g′(s) =1 + g(s)

2c(s− g(s)), (19.3.1)

and its maximal admissible solution is given by

g∗(s) = αs − β (19.3.2)

where α = (2c−1)/2c and β = 1/2c . By applying the result of Corollary 13.3we get

E(

max0≤t≤τ

|Bt+x|)≤ W (x; c) + cE

(∫ τ

0

dt

1 + |Bt+x|)

(19.3.3)

for all stopping times τ of B , all c > 1/2 and all x ≥ 0 , where

W (x; c) =

⎧⎪⎨⎪⎩1

2c−1+ 2c

((1+x) log(1+x)− x

)if x ≤ 1

(2c− 1),

2c(1+x) log(1+

12c−1

)− 1 if x >

1(2c− 1)

.(19.3.4)

This inequality is sharp, and for each c > 1/2 and x ≥ 0 given and fixed, equalityin (19.3.4) is attained at the stopping time

τ∗ = inf

t > 0 : St − αXt ≥ β

(19.3.5)

which is pointwise the smallest possible with this property. By minimizing over allc > 1/2 on the right-hand side in (19.3.3) we get a sharp inequality (equality is


attained at each stopping time τ∗ from (19.3.5) whenever c > 1/2 and x ≥ 0 ).In particular, this for x = 0 yields

E(

max0≤t≤τ

|Bt|)≤ 1

2E

(∫ τ

0

dt

1 + |Bt|)

+√

2(

E

∫ τ

0

dt

1 + |Bt|)1/2

(19.3.6)

for all stopping times τ of B . This inequality is sharp, and equality in (19.3.6)is attained at each stopping time τ∗ from (19.3.5). Note by Ito’s formula (page67) and the optional sampling theorem (page 60) that

E

(∫ τ

0

dt

1 + |Bt|)

= 2 E((

1+|Bτ |)log(1+|Bτ |

)− |Bτ |)

(19.3.7)

for all stopping times τ of B satisfying Eτr < ∞ for some r > 1/2 . This showsthat the inequality (19.3.6) in essence is of the L logL -type. The advantage of(19.3.6) upon the classical Hardy–Littlewood L log L -inequality is its sharpnessfor small stopping times as well (note that equality in (19.3.6) is attained forτ ≡ 0 ). For more information on this inequality and remaining details we referto [157].

20. Burkholder–Davis–Gundy inequalities

All optimal stopping problems considered so far in this chapter were linear in thesense that the gain function is a linear function of time (recall our discussion inthe end of Section 15 above). In this section we will briefly consider a nonlinearproblem in order to address difficulties which such problems carry along.

Let B = (Bt)t≥0 be a standard Brownian motion defined on a probabilityspace (Ω,F , P) , and let p > 0 be given and fixed. Then the Burkholder–Davis–Gundy inequalities (see (C5) on page 63) state that

cp Eτp/2 ≤ E max0≤t≤τ

|Bt|p ≤ Cp Eτp/2 (20.0.8)

for all stopping times τ of B where cp > 0 and Cp > 0 are universal constants.

The question of finding the best possible values for cp and Cp in (20.0.8)appears to be of interest. Its emphasis is not so much on having these valuesbut more on finding a method of proof which can deliver them. Clearly, the casep = 2 reduces trivially to Doob’s maximal inequality treated in Section 18 aboveand C2 = 4 is the best possible constant in (20.0.8) when p = 2 . Likewise, itis easily seen that c2 = 1 is the best constant in (20.0.8) when p = 2 (considerτ = inf

t ≥ 0 : |Bt| = 1

for instance). In the case of other p however the

situation is much more complicated. For example, if p = 1 then (20.0.8) reads asfollows:

c1 E√

τ ≤ E max0≤t≤τ

|Bt| ≤ C2 E√

τ (20.0.9)

Section 20. Burkholder–Davis–Gundy inequalities 285

and the best constants c1 and C2 in (20.0.9), being valid for all stopping timesτ of B , seem to be unknown to date.

To illustrate difficulties which are inherently present in tackling these ques-tions let us concentrate on the problem of finding the best value for C2 . For this,consider the optimal stopping problem

V = supτ

E(

max0≤t≤τ

|Bt| − c√

τ)

(20.0.10)

where the supremum is taken over all stopping times τ of B satisfying E√

τ <∞ , and c > 0 is a given and fixed constant.

In order to solve this problem we need to determine its dimension (see Sub-section 6.2) and (if possible) try to reduce it by using some of the available trans-formations (see Sections 10–12). Leaving the latter aside for the moment notethat the underlying Markov process is Zt = (t, Xt, St) where Xt = |Bt| andSt = max 0≤s≤t |Bs| for t ≥ 0 . Due to the existence of the square root in (20.0.10)it is not possible to remove the time component t from Zt and therefore the non-linear problem (20.0.10) is inherently three-dimensional.

Recalling our discussion in Subsection 13.2 it is plausible to assume that thefollowing optimal stopping time should be optimal in (20.0.10):

τ∗ = inf

t ≥ 0 : Xt ≤ g∗(t, St)

(20.0.11)

for c > C1 , where (t, s) → g∗(t, s) is the optimal stopping time which now de-pends on both time t and maximum s so that its explicit determination becomesmuch more delicate. (To be more precise one should consider (20.0.11) under Pu,x,s

where Pu,x,s(Xu = x, Su = s) = 1 for u ≥ 0 and s > x > 0 .)

Note that when max0≤t≤τ |Bt| is replaced by |Bτ | in (20.0.11) then theproblem can be successfully tackled by the method of time change (see Section10). We will omit further details in this direction.

Chapter VI.

Optimal stopping in mathematical statistics

21. Sequential testing of a Wiener process

In the Bayesian formulation of the problem it is assumed that we observe a trajec-tory of the Wiener process (Brownian motion) X = (Xt)t≥0 with drift θµ wherethe random variable θ may be 1 or 0 with probability π or 1−π , respectively.

1. For a precise probabilistic formulation of the Bayesian problem it is con-venient to assume that all our considerations take place on a probability-statisticalspace (Ω;F ; Pπ , π ∈ [0, 1]) where the probability measure Pπ has the followingstructure:

Pπ = πP1 + (1−π)P0 (21.0.1)

for π ∈ [0, 1] . (Sometimes (Ω;F ; Pπ, π ∈ [0, 1]) is called a statistical experiment.)

Let θ be a random variable taking two values 1 and 0 with probabilitiesPπ(θ = 1) = π and Pπ(θ = 0) = 1 − π , and let W = (Wt)t≥0 be a standardWiener process started at zero under Pπ . It is assumed that θ and W areindependent.

It is further assumed that we observe a process X = (Xt)t≥0 of the form

Xt = θµt + σWt (21.0.2)

where µ = 0 and σ2 > 0 are given and fixed. Thus Pπ(X ∈ · | θ = i ) = Pi(X ∈ · )is the distribution law of a Wiener process with drift iµ and diffusion coefficientσ2 > 0 for i = 0, 1 , so that π and 1−π play the role of a priori probabilities ofthe statistical hypotheses

H1 : θ = 1 and H0 : θ = 0 (21.0.3)

respectively.

288 Chapter VI. Optimal stopping in mathematical statistics

Being based upon the continuous observation of X our task is to test sequen-tially the hypotheses H1 and H0 with a minimal loss. For this, we consider a se-quential decision rule (τ, d) , where τ is a stopping time of the observed process X(i.e. a stopping time with respect to the natural filtration FX

t = σ(Xs : 0 ≤ s ≤ t)generated by X for t ≥ 0 ), and d is an FX

τ -measurable random variable takingvalues 0 and 1 . After stopping the observation at time τ , the terminal decisionfunction d indicates which hypothesis should be accepted according to the fol-lowing rule: if d = 1 we accept H1 , and if d = 0 we accept H0 . The problemthen consists of computing the risk function

V (π) = inf(τ,d)

Eπ

(τ + aI(d = 0, θ = 1) + bI(d = 1, θ = 0)

)(21.0.4)

and finding the optimal decision rule (τ∗, d∗) at which the infimum in (21.0.4)is attained. Here Eπτ is the average loss due to a cost of the observations, andaPπ(d = 0, θ = 1) + bPπ(d = 1, θ = 0) is the average loss due to a wrong terminaldecision, where a > 0 and b > 0 are given constants.

2. By means of standard arguments (see [196, pp. 166–167]) one can reducethe Bayesian problem (21.0.4) to the optimal stopping problem

V (π) = infτ

Eπ

(τ + aπτ ∧ b(1−πτ )

)(21.0.5)

for the a posteriori probability process πt = Pπ(θ = 1 | FXt ) with t ≥ 0 and

Pπ(π0 = π) = 1 (where x ∧ y = minx, y ). Setting c = b/(a + b) the optimaldecision function is then given by d∗ = 1 if πτ∗ ≥ c and d∗ = 0 if πτ∗ < c .

3. It can be shown (see [196, pp. 180–181]) that the likelihood ratio process(ϕt)t≥0 , defined as the Radon–Nikodym derivative

ϕt =d(P1|FX

t )d(P0|FX

t ), (21.0.6)

admits the following representation:

ϕt = exp( µ

σ2

(Xt − µ

2t))

(21.0.7)

while the a posteriori probability process (πt)t≥0 can be expressed as

πt =(

π

1−πϕt

)/(1 +

π

1−πϕt

)(21.0.8)

and hence solves the stochastic differential equation

dπt =µ

σπt(1−πt) dWt (π0 = π) (21.0.9)

Section 21. Sequential testing of a Wiener process 289

where the innovation process (Wt)t≥0 defined by

Wt =1σ

(Xt − µ

∫ t

0

πs ds)

(21.0.10)

is a standard Wiener process (see also [127, Chap. IX]). Using (21.0.7) and (21.0.8)it can be verified that (πt)t≥0 is a time-homogeneous (strong) Markov processunder Pπ with respect to the natural filtration. As the latter clearly coincideswith (FX

t )t≥0 it is also clear that the infimum in (21.0.5) can equivalently betaken over all stopping times of (πt)t≥0 .


1. In order to solve the problem (21.0.5) above when the horizon is infinite let usconsider the optimal stopping problem for the Markov process (πt)t≥0 given by

V (π) = infτ

Eπ

(M(πτ ) + τ

)(21.1.1)

where Pπ(π0 = π) = 1 , i.e. Pπ is a probability measure under which the diffusionprocess (πt)t≥0 solving (21.0.9) starts at π , the infimum in (21.1.1) is taken overall stopping times τ of (πt)t≥0 , and we set M(π) = aπ ∧ b(1−π) for π ∈ [0, 1] .For further reference recall that the infinitesimal generator of (πt)t≥0 is given by

L =µ2

2σ2π2(1−π)2

∂2

∂π2. (21.1.2)

2. The optimal stopping problem (21.1.1) will be solved in two steps. In thefirst step we will make a guess for the solution. In the second step we will verifythat the guessed solution is correct (Theorem 21.1).

From (21.1.1) and (21.0.9) above we see that the closer (πt)t≥0 gets to either0 or 1 the less likely that the loss will decrease upon continuation. This suggeststhat there exist points A ∈ [0, c] and B ∈ [c, 1] such that the stopping time

τA,B = inft ≥ 0 : πt /∈ (A, B)

(21.1.3)

is optimal in (21.1.1).

Standard arguments based on the strong Markov property (cf. Chap. III)lead to the following free-boundary problem for the unknown function V and the


unknown points A and B :

LV = −1 for π ∈ (A, B), (21.1.4)V (A) = aA, (21.1.5)V (B) = b(1−B), (21.1.6)V ′(A) = a (smooth fit), (21.1.7)V ′(B) = −b (smooth fit), (21.1.8)V < M for π ∈ (A, B), (21.1.9)V = M for π ∈ [0, A) ∪ (B, 1]. (21.1.10)

3. To solve the free-boundary problem (21.1.4)–(21.1.10) denote

ψ(π) = (1 − 2π) log( π

1−π

)(21.1.11)

and with fixed A ∈ (0, c) note by a direct verification that the function

V (π; A) =2σ2

µ2

(ψ(π) − ψ(A)

)+(a − 2σ2

µ2ψ′(A)

)(π − A) + aA (21.1.12)

is the unique solution of the equation (21.1.4) for π ≥ A satisfying (21.1.5) and(21.1.7) at A .

When A ∈ (0, c) is close to c , then π → V (π; A) intersects π → b(1−π)at some B ∈ (c, 1) . The function π → V (π; A) is concave on (0, 1) , it satis-fies V (0+; A) = V (1−; A) = −∞ , and π → V (π; A′) and π → V (π; A′′) donot intersect at any π > A′ ∨ A′′ when A′ = A′′ . For the latter note that(∂/∂A)V (π; A) = (2σ2/µ2)ψ′′(A)(A − π) > 0 for all π > A since ψ′′(A) < 0 .Let π0

A denote the zero point of π → V (π; A) on (A, 1) . Then π0A ↓ 0 as

A ↓ 0 since clearly π0A ↓ l while assuming l > 0 and passing to the limit for

A ↓ 0 in the equation V (π0A; A) = 0 leads to a contradiction. Finally, reducing

A from c down to 0 and using the properties established above we get the ex-istence of a unique point A∗ ∈ (0, c) for which there is B∗ ∈ (c, 1) such thatV (B∗; A∗) = b(1−B∗) and V ′(B∗; A∗) = −b as required by (21.1.6) and (21.1.8)above. This establishes the existence of a unique solution (V ( · ; A∗), A∗, B∗) tothe free-boundary problem (21.1.4)–(21.1.10). Note that π → V (π; A∗) is C2 on(0, 1) \ A, B but only C1 at A∗ and B∗ when extended to be equal to Mon [0, A∗) and (B∗, 1] . Note also that the extended function π → V (π; A∗) isconcave on [0, 1] .

4. In this way we have arrived at the conclusions of the following theorem.


Theorem 21.1. The value function V from (21.1.1) is explicitly given by

V (π) =

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩2σ2

µ2

(ψ(π)−ψ(A)

)+(a− 2σ2

µ2ψ′(A∗)

)×(π−A∗) + aA∗ if π ∈ (A∗, B∗),

aπ ∧ b(1−π) if π ∈ [0, A∗) ∪ (B∗, 1]

(21.1.13)

where ψ is given by (21.1.11) above while A∗ ∈ (0, c) and B∗ ∈ (c, 1) are theunique solution of the system of transcendental equations

V (B∗; A∗) = b(1−B∗), (21.1.14)V ′(B∗; A∗) = −b (21.1.15)

where π → V (π; A) is given by (21.1.12) above. The stopping time τA∗,B∗ givenby (21.1.3) above is optimal in (21.1.1).

Proof. Denote the function on the right-hand side of (21.1.13) by V∗ . The prop-erties of V∗ stated in the end of paragraph 3 above show that Ito’s formula (page67) can be applied to V (πt) in its standard form (cf. Subsection 3.5). This gives

V∗(πt) = V∗(π) +∫ t

0

LV∗(πs)I(πs /∈ A, B) ds (21.1.16)

+µ

σ

∫ t

0

πs(1−πs)V ′∗(πs) dWs.

Recalling that V∗(π) = M(π) = aπ ∧ b(1−π) for π ∈ [0, A∗) ∪ (B∗, 1] and usingthat V∗ satisfies (21.1.4) for π ∈ (A∗, B∗) , we see that

LV∗ ≥ −1 (21.1.17)

everywhere on [0, 1] but A∗ and B∗ . By (21.1.9), (21.1.10), (21.1.16) and (21.1.17)it follows that

M(πt) ≥ V∗(πt) ≥ V∗(π) − t + Mt (21.1.18)


Mt =µ

σ

∫ t

0

πs(1−πs)V ′∗(πs) dWs. (21.1.19)

Using that |V ′∗(π)| ≤ a ∨ b < ∞ for all π ∈ [0, 1] it is easily verified by standard

means that M is a martingale. Moreover, by the optional sampling theorem (page60) this bound also shows that EπMτ = 0 whenever Eπ

√τ < ∞ for a stopping

time τ . In particular, the latter condition is satisfied if Eπτ < ∞ . As clearly in


(21.1.1) it is enough to take the infimum only over stopping times τ satisfyingEπτ < ∞ , we may insert τ in (21.1.18) instead of t , take Eπ on both sides, andconclude that

Eπ

(M(πτ ) + τ

) ≥ V∗(π) (21.1.20)

for all π ∈ [0, 1] . This shows that V ≥ V∗ . On the other hand, using (21.1.4) andthe definition of τA∗,B∗ in (21.1.3), we see from (21.1.16) that

M(πτA∗,B∗

)= V∗

(πτA∗,B∗

)= V∗(π) − τA∗,B∗ + MτA∗,B∗ . (21.1.21)

Since EπτA∗,B∗ < ∞ (being true for any A and B ) we see by taking Eπ on bothsides of (21.1.21) that equality in (21.1.20) is attained at τ = τA∗,B∗ , and thusV = V∗ . Combining this with the conclusions on the existence and uniqueness ofA∗ and B∗ derived in paragraph 3 above, we see that the proof is complete.

For more details on the Wiener sequential testing problem with infinite hori-zon (including the fixed probability error formulation) we refer to [196, Chap. 4,Sect. 1–2].

21.2. Finite horizon

1. In order to solve the problem (21.0.5) when the horizon T is finite let us considerthe extended optimal stopping problem for the Markov process (t, πt)0≤t≤T givenby

V (t, π) = inf0≤τ≤T−t

Et,πG(t + τ, πt+τ ) (21.2.1)

where Pt,π(πt = π) = 1 , i.e. Pt,π is a probability measure under which the dif-fusion process (πt+s)0≤s≤T−t solving (21.0.9) starts at π at time t , the infi-mum in (21.2.1) is taken over all stopping times τ of (πt+s)0≤s≤T−t , and we setG(t, π) = t + aπ ∧ b(1−π) for (t, π) ∈ [0, T ] × [0, 1] . Since G is bounded andcontinuous on [0, T ] × [0, 1] it is possible to apply Corollary 2.9 (Finite horizon)with Remark 2.10 and conclude that an optimal stopping time exists in (21.2.1).

2. Let us now determine the structure of the optimal stopping time in theproblem (21.2.1).

(i) It follows from (21.0.9) that the scale function of (πt)t≥0 is given byS(x) = x for x ∈ [0, 1] and the speed measure of (πt)t≥0 is given by the equationm(dx) = (2σ)/(µ x (1−x)) dx for x ∈ (0, 1) . Hence the Green function of (πt)t≥0

on [π0, π1] ⊂ (0, 1) is given by Gπ0,π1(x, y) = (π1 − x)(y − π0)/(π1 − π0) forπ0 ≤ y ≤ x and Gπ0,π1(x, y) = (π1 − y)(x−π0)/(π1 −π0) for x ≤ y ≤ π1 .

Set H(π) = aπ ∧ b(1−π) for π ∈ [0, 1] and let d = H(c) . Take ε ∈ (0, d)and denote by π0 = π0(ε) and π1 = π1(ε) the unique points 0 < π0 < c < π1 < 1satisfying H(π0) = H(π1) = d − ε . Let σε = inf t > 0 : πt /∈ (π0, π1) and set


σTε = σε ∧ T . Then σε and σT

ε are stopping times and it is easily verified that

EcσTε ≤ Ecσε =

∫ π1

π0

Gπ0,π1(x, y)m(dy) ≤ ε2 (21.2.2)

for some K > 0 large enough (not depending on ε ). Similarly, we find that

EcH(πσTε) = Ec[H(πσε)I(σε < T )] + Ec[H(πT )I(σε ≥ T )] (21.2.3)

≤ d − ε + d Pc(σε > T ) ≤ d − ε + (d/T )Ecσε

≤ d − ε + L ε2

where L = dK/T .

Combining (21.2.2) and (21.2.3) we see that

EcG(σTε , πσT

ε) = Ec[σT

ε + H(πσTε)] ≤ d − ε + (K+L) ε2 (21.2.4)

for all ε ∈ (0, d) . If we choose ε > 0 in (21.2.4) small enough, we observe thatEcG(σT

ε , πσTε) < d . Using the fact that G(t, π) = t + H(π) is linear in t , and

T > 0 above is arbitrary, this shows that it is never optimal to stop in (21.2.1)when πt+s = c for 0 ≤ s < T − t . In other words, this shows that all points (t, c)for 0 ≤ t < T belong to the continuation set

C = (t, π) ∈ [0, T )×[0, 1] : V (t, π) < G(t, π). (21.2.5)

(ii) Recalling the solution to the problem (21.0.5) in the case of infinitehorizon, where the stopping time τ∗ = inf t > 0 : πt /∈ (A∗, B∗) is optimaland 0 < A∗ < c < B∗ < 1 are uniquely determined from the system (21.1.14)–(21.1.15) (see also (4.85) in [196, p. 185]), we see that all points (t, π) for 0 ≤ t ≤ Twith either 0 ≤ π ≤ A∗ or B∗ ≤ π ≤ 1 belong to the stopping set. Moreover,since π → V (t, π) with 0 ≤ t ≤ T given and fixed is concave on [0, 1] (thisis easily deduced using the same arguments as in [123, p. 105] or [196, p. 168]),it follows directly from the previous two conclusions about the continuation andstopping set that there exist functions g0 and g1 satisfying 0 < A∗ ≤ g0(t) <c < g1(t) ≤ B∗ < 1 for all 0 ≤ t < T such that the continuation set is an openset of the form

C = (t, π) ∈ [0, T )×[0, 1] : π ∈ (g0(t), g1(t)) (21.2.6)

and the stopping set is the closure of the set

D = (t, π) ∈ [0, T )×[0, 1] : π ∈ [0, g0(t)) ∪ (g1(t), 1]. (21.2.7)

(Below we will show that V is continuous so that C is open indeed. We will alsosee that g0(T ) = g1(T ) = c .)


(iii) Since the problem (21.2.1) is time-homogeneous, in the sense that G(t, π)= t + H(π) is linear in t and H depends on π only, it follows that the mapt → V (t, π) − t is increasing on [0, T ] . Hence if (t, π) belongs to C for someπ ∈ (0, 1) and we take any other 0 ≤ t′ < t ≤ T , then V (t′, π) − G(t′, π) =V (t′, π) − t′ − H(π) ≤ V (t, π) − t − H(π) = V (t, π) − G(t, π) < 0 , showing that(t′, π) belongs to C as well. From this we may conclude in (21.2.6)–(21.2.7) thatthe boundary t → g0(t) is increasing and the boundary t → g1(t) is decreasingon [0, T ] .

(iv) Let us finally observe that the value function V from (21.2.1) and theboundaries g0 and g1 from (21.2.6)–(21.2.7) also depend on T and let them bedenoted here by V T , gT

0 and gT1 , respectively. Using the fact that T → V T (t, π)

is a decreasing function on [t,∞) and V T (t, π) = G(t, π) for all π ∈ [0, gT0 (t)] ∪

[gT1 (t), 1] , we conclude that if T < T ′ , then 0 ≤ gT ′

0 (t) ≤ gT0 (t) < c < gT

1 (t) ≤gT ′1 (t) ≤ 1 for all t ∈ [0, T ) . Letting T ′ in the previous expression go to ∞ , we

get that 0 < A∗ ≤ gT0 (t) < c < gT

1 (t) ≤ B∗ < 1 with A∗ ≡ limT→∞ gT0 (t) and

B∗ ≡ limT→∞ gT1 (t) for all t ≥ 0 , where A∗ and B∗ are the optimal stopping

points in the infinite horizon problem referred to above.

3. Let us now show that the value function (t, π) → V (t, π) is continuous on[0, T ]× [0, 1] . For this it is enough to prove that

π → V (t0, π) is continuous at π0, (21.2.8)t → V (t, π) is continuous at t0 uniformly over π ∈ [π0 − δ, π0 + δ] (21.2.9)

for each (t0, π0) ∈ [0, T ] × [0, 1] with some δ > 0 small enough (it may dependon π0 ). Since (21.2.8) follows by the fact that π → V (t, π) is concave on [0, 1] ,it remains to establish (21.2.9).

For this, let us fix arbitrary 0 ≤ t1 < t2 ≤ T and 0 < π < 1 , and letτ1 = τ∗(t1, π) denote the optimal stopping time for V (t1, π) . Set τ2 = τ1∧(T−t2)and note since t → V (t, π) is increasing on [0, T ] and τ2 ≤ τ1 that we have

0 ≤ V (t2, π) − V (t1, π) (21.2.10)≤ Eπ[(t2 + τ2) + H(πt2+τ2)] − Eπ[(t1 + τ1) + H(πt1+τ1)]≤ (t2 − t1) + Eπ[H(πt2+τ2) − H(πt1+τ1)]

where we recall that H(π) = aπ ∧ b(1−π) for π ∈ [0, 1] . Observe further that

Eπ[H(πt2+τ2) − H(πt1+τ1)] (21.2.11)

=1∑

i=0

1 + (−1)i(1− 2π)2

Ei

[h(ϕτ2) − h(ϕτ1)

]where for each π ∈ (0, 1) given and fixed the function h is defined by

h(x) =∣∣∣∣H( π

1 − π x

1 + π1− π x

)∣∣∣∣ (21.2.12)


for all x > 0 . Then for any 0 < x1 < x2 given and fixed it follows by the meanvalue theorem (note that h is C1 on (0,∞) except one point) that there existsξ ∈ [x1, x2] such that

|h(x2) − h(x1)| ≤ |h′(ξ)| (x2 −x1) (21.2.13)

where the derivative h′ at ξ satisfies

|h′(ξ)| =∣∣∣∣H ′( π

1 − π ξ

1 + π1− π ξ

)∣∣∣∣ π(1−π)(1−π+πξ)2

≤ Kπ(1−π)(1−π)2

= Kπ

1−π(21.2.14)

with some K > 0 large enough.

On the other hand, the explicit expression (21.0.7) yields

ϕτ2 − ϕτ1 = ϕτ2

(1 − ϕτ1

ϕτ2

)(21.2.15)

= ϕτ2

(1 − exp

( µ

σ2(Xτ1 − Xτ2) −

µ2

2σ2(τ1 − τ2)

))and thus the strong Markov property (stationary independent increments) to-gether with the representation (21.0.2) and the fact that τ1 − τ2 ≤ t2 − t1 imply

Ei|ϕτ2 − ϕτ1 | (21.2.16)

= Ei

∣∣∣∣ϕτ2

(1 − exp

(µ

σ(Wτ1 −Wτ2) − (−1)i µ2

2σ2(τ1 − τ2)

))∣∣∣∣= Ei

[ϕτ2 Ei

[∣∣∣∣1 − exp(

µ

σ(Wτ1 −Wτ2) − (−1)i µ2

2σ2(τ1 − τ2)

)∣∣∣∣ ∣∣∣FXτ2

]]≤ Eiϕτ2 Ei

[sup

0≤t≤t2−t1

exp(

µ

σWt +

µ2

2σ2t

)− 1

]for i = 0, 1 . Since it easily follows that

Eiϕτ2 = Ei

[exp

(µ

σWτ2 − (−1)i µ2

2σ2τ2

)](21.2.17)

≤ exp(

µ2

σ2(T − t2)

)≤ exp

(µ2

σ2T

)from (21.2.12)–(21.2.17) we get

Ei|h(ϕτ2) − h(ϕτ1)| ≤ Kπ

1−πEi|ϕτ2 − ϕτ1 | ≤ K

π

1−πL(t2 − t1) (21.2.18)

where the function L is defined by

L(t2 − t1) = exp(

µ2

σ2T

)Ei

[sup

0≤t≤t2−t1

exp(

µ

σWt +

µ2

2σ2t

)− 1

]. (21.2.19)


Therefore, combining (21.2.18) with (21.2.10)–(21.2.11) above, we obtain

V (t2, π) − V (t1, π) ≤ (t2 − t1) + Kπ

1−πL(t2 − t1) (21.2.20)

from where, by virtue of the fact that L(t2 − t1) → 0 in (21.2.19) as t2 −t1 ↓ 0 , we easily conclude that (21.2.9) holds. In particular, this shows that theinstantaneous-stopping conditions (21.2.40) below are satisfied.

4. In order to prove that the smooth-fit conditions (21.2.41) hold, i.e. thatπ → V (t, π) is C1 at g0(t) and g1(t) , let us fix a point (t, π) ∈ [0, T ) × (0, 1)lying on the boundary g0 so that π = g0(t) . Then for all ε > 0 such thatπ < π + ε < c we have

V (t, π + ε) − V (t, π)ε

≤ G(t, π + ε) − G(t, π)ε

(21.2.21)

and hence, taking the limit in (21.2.21) as ε ↓ 0 , we get

∂+V

∂π(t, π) ≤ ∂G

∂π(t, π) (21.2.22)

where the right-hand derivative in (21.2.22) exists (and is finite) by virtue ofthe concavity of π → V (t, π) on [0, 1] . Note that the latter will also be provedindependently below.

Let us now fix some ε > 0 such that π < π + ε < c and consider thestopping time τε = τ∗(t, π + ε) being optimal for V (t, π + ε) . Note that τε isthe first exit time of the process (πt+s)0≤s≤T−t from the set C in (21.2.6). Thenby (21.0.1) and (21.0.8) it follows using the mean value theorem that there existsξ ∈ [π, π + ε] such that

V (t, π + ε) − V (t, π) ≥ Eπ+εG(t + τε, πt+τε) − EπG(t + τε, πt+τε) (21.2.23)

=1∑

i=0

Ei[Si(π + ε) − Si(π)] = ε

1∑i=0

EiS′i(ξi)

where the function Si is defined by

Si(π) =1 + (−1)i(1 − 2π)

2G

(t + τε,

(ξi/(1− ξi))ϕτε

1 + (ξi/(1− ξi))ϕτε

)(21.2.24)

and its derivative S′i at ξi is given by

S′i(ξi) = (−1)i+1 G

(t + τε,


1 + (ξi/(1− ξi))ϕτε

)(21.2.25)

+1 + (−1)i(1 − 2ξi)

2∂G

∂π

(t + τε,


1 + (ξi/(1− ξi))ϕτε

)ϕτε

(1 − ξi + ξiϕτε)2


for i = 0, 1 . Since g0 is increasing it is easily verified using (21.0.7)–(21.0.8) andthe fact that t → ± µ

2σ t is a lower function for the standard Wiener process Wthat τε → 0 Pi -a.s. and thus ϕτε → 1 Pi -a.s. as ε ↓ 0 for i = 0, 1 . Hence weeasily find

S′i(ξi) → (−1)i+1G(t, π) +

1 + (−1)i(1−2π)2

∂G

∂π(t, π) Pi -a.s. (21.2.26)

as ε ↓ 0 , and clearly |S′i(ξi)| ≤ Ki with some Ki > 0 large enough for i = 0, 1 .

It thus follows from (21.2.23) using (21.2.26) that

V (t, π + ε) − V (t, π)ε

≥1∑

i=0

EiS′i(ξi) → ∂G

∂π(t, π) (21.2.27)

as ε ↓ 0 by the dominated convergence theorem. This combined with (21.2.21)above proves that V +

π (t, π) exists and equals Gπ(t, π) . The smooth fit at theboundary g1 is proved analogously.

5. We proceed by proving that the boundaries g0 and g1 are continuous on[0, T ] and that g0(T ) = g1(T ) = c .

(i) Let us first show that the boundaries g0 and g1 are right-continuouson [0, T ] . For this, fix t ∈ [0, T ) and consider a sequence tn ↓ t as n → ∞ .Since gi is monotone, the right-hand limit gi(t+) exists for i = 0, 1 . Because(tn, gi(tn)) ∈ D for all n ≥ 1 , and D is closed, we see that (t, gi(t+)) ∈ D fori = 0, 1 . Hence by (21.2.7) we see that g0(t+) ≤ g0(t) and g1(t+) ≥ g1(t) . Thereverse inequalities follow obviously from the fact that g0 is increasing and g1 isdecreasing on [0, T ] , thus proving the claim.

(ii) Suppose that at some point t∗ ∈ (0, T ) the function g1 makes a jump,i.e. let g1(t∗−) > g1(t∗) ≥ c . Let us fix a point t′ < t∗ close to t∗ and consider thehalf-open set R ⊂ C being a curved trapezoid formed by the vertices (t′, g1(t′)) ,(t∗, g1(t∗−)) , (t∗, π′) and (t′, π′) with any π′ fixed arbitrarily in the interval(g1(t∗), g1(t∗−)) . Observe that the strong Markov property implies that the valuefunction V from (21.2.1) is C1,2 on C . Note also that the gain function G isC1,2 in R so that by the Newton–Leibniz formula using (21.2.40) and (21.2.41)it follows that

V (t, π) − G(t, π) =∫ g1(t)

π

∫ g1(t)

u

(∂2V

∂π2− ∂2G

∂π2

)(t, v) dv du (21.2.28)

for all (t, π) ∈ R .

Let us fix some (t, π) ∈ C and take an arbitrary ε > 0 such that (t+ε, π) ∈C . Then denoting by τε = τ∗(t + ε, π) the optimal stopping time for V (t + ε, π) ,


we have

V (t+ε, π) − V (t, π)ε

(21.2.29)

≥ Et+ε,πG(t+ε+τε, πt+ε+τε) − Et,πG(t+τε, πt+τε)ε

=Eπ[G(t + ε + τε, πτε) − G(t + τε, πτε)]

ε= 1

and thus, taking the limit in (21.2.29) as ε ↓ 0 , we get

∂V

∂t(t, π) ≥ ∂G

∂t(t, π) = 1 (21.2.30)

at each (t, π) ∈ C .

Since the strong Markov property implies that the value function V from(21.2.1) solves the equation (21.2.39), using (21.2.30) we obtain

∂2V

∂π2(t, π) = −2σ2

µ2

1π2(1−π)2

∂V

∂t(t, π) ≤ −ε

σ2

µ2(21.2.31)

for all t′ ≤ t < t∗ and all π′ ≤ π < g1(t′) with ε > 0 small enough.

Hence by (21.2.28) using that Gππ = 0 we get

V (t′, π′) − G(t′, π′) (21.2.32)

≤ −εσ2

µ2

(g1(t′) − π′)2

2→ −ε

σ2

µ2

(g1(t∗−) − π′)2

2< 0

as t′ ↑ t∗ . This implies that V (t∗, π′) < G(t∗, π′) which contradicts the fact that(t∗, π′) belongs to the stopping set D . Thus g1(t∗−) = g1(t∗) showing that g1

is continuous at t∗ and thus on [0, T ] as well. A similar argument shows that thefunction g0 is continuous on [0, T ] .

(iii) We finally note that the method of proof from the previous part (ii) alsoimplies that g0(T ) = g1(T ) = c . To see this, we may let t∗ = T and likewisesuppose that g1(T−) > c . Then repeating the arguments presented above wordby word we arrive to a contradiction with the fact that V (T, π) = G(T, π) for allπ ∈ [c, g1(T−)] thus proving the claim.

6. Summarizing the facts proved in paragraphs 5–8 above we may concludethat the following exit time is optimal in the extended problem (21.2.1):

τ∗ = inf0 ≤ s ≤ T − t : πt+s /∈ (g0(t + s), g1(t + s)) (21.2.33)

(the infimum of an empty set being equal to T − t ) where the two boundaries


(g0, g1) satisfy the following properties (see Figure VI.1):

g0 : [0, T ] → [0, 1] is continuous and increasing , (21.2.34)g1 : [0, T ] → [0, 1] is continuous and decreasing , (21.2.35)

A∗ ≤ g0(t) < c < g1(t) ≤ B∗ for all 0 ≤ t < T , (21.2.36)gi(T ) = c for i = 0, 1, (21.2.37)

where A∗ and B∗ satisfying 0 < A∗ < c < B∗ < 1 are the optimal stoppingpoints for the infinite horizon problem uniquely determined from the system oftranscendental equations (21.1.14)–(21.1.15) or [196, p. 185, (4.85)].

Standard arguments imply that the infinitesimal operator L of the process(t, πt)0≤t≤T acts on a function f ∈ C1,2([0, T )× [0, 1]) according to the rule

(Lf)(t, π) =(

∂f

∂t+

µ2

2σ2π2(1−π)2

∂2f

∂π2

)(t, π) (21.2.38)

for all (t, π) ∈ [0, T ) × [0, 1] . In view of the facts proved above we are thus natu-rally led to formulate the following free-boundary problem for the unknown valuefunction V from (21.2.1) and the unknown boundaries (g0, g1) from (21.2.6)–(21.2.7):

(LV )(t, π) = 0 for (t, π) ∈ C, (21.2.39)

V (t, π)∣∣π=g0(t)+

= t + ag0(t), V (t, π)∣∣π=g1(t)− = t + b(1 − g1(t)), (21.2.40)

∂V

∂π(t, π)

∣∣∣π=g0(t)+

= a,∂V

∂π(t, π)

∣∣∣π=g1(t)−

= −b, (21.2.41)

V (t, π) < G(t, π) for (t, π) ∈ C, (21.2.42)V (t, π) = G(t, π) for (t, π) ∈ D, (21.2.43)

where C and D are given by (21.2.6) and (21.2.7), and the instantaneous-stoppingconditions (21.2.40) are satisfied for all 0 ≤ t ≤ T and the smooth-fit conditions(21.2.41) are satisfied for all 0 ≤ t < T .

Note that the superharmonic characterization of the value function (cf. Chap-ter I) implies that V from (21.2.1) is a largest function satisfying (21.2.39)–(21.2.40) and (21.2.42)–(21.2.43).

7. Making use of the facts proved above we are now ready to formulate themain result of the section. Below we set ϕ(x) = (1/

√2π)e−x2/2 and Φ(x) =∫ x

−∞ ϕ(y) dy for x ∈ R .

Theorem 21.2. In the Bayesian problem (21.0.4)–(21.0.5) of testing two simplehypotheses (21.0.3) the optimal decision rule (τ∗, d∗) is explicitly given by

τ∗ = inf0 ≤ t ≤ T : πt /∈ (g0(t), g1(t)), (21.2.44)

d∗ =

1 (accept H1) if πτ∗ = g1(τ∗),0 (accept H0) if πτ∗ = g0(τ∗),

(21.2.45)


0

1

π

T

t π t→

→t g (t)1

t g (t)→0

τ∗

c

Figure VI.1: A computer drawing of the optimal stopping boundaries g0

and g1 from Theorem 21.2. In the case above it is optimal to accept thehypothesis H1 .

where the two boundaries (g0, g1) can be characterized as a unique solution of thecoupled system of nonlinear integral equations

Et,gi(t)[aπT ∧ b(1−πT )] = agi(t) ∧ b(1 − gi(t)) (21.2.46)

+1∑

j=0

∫ T−t

0

(−1)jPt,gi(t)[πt+u ≤ gj(t + u)] du (i = 0, 1)

for 0 ≤ t ≤ T satisfying (21.2.34)–(21.2.37) [see Figure VI.1] .

More explicitly, the six terms in the system (21.2.46) read as follows :

Et,gi(t)[aπT ∧ b(1−πT )] (21.2.47)

= gi(t)∫ ∞

−∞

1

1 − gi(t) + gi(t) exp

µzσ

√T − t + µ2

2σ2 (T − t)

× minagi(t) exp

µzσ

√T − t + µ2

2σ2 (T − t), b(1 − gi(t))

ϕ(z) dz

+ (1 − gi(t))∫ ∞

−∞

1

1 − gi(t) + gi(t) exp

µzσ

√T − t − µ2

2σ2 (T − t)

× minagi(t) exp

µzσ

√T − t − µ2

2σ2 (T − t), b(1−gi(t))

ϕ(z) dz,


Pt,gi(t)

(πt+u ≤ gj(t + u)

)(21.2.48)

= gi(t)Φ(

σ

µ√

ulog(

gj(t + u)1 − gj(t + u)

1 − gi(t)gi(t)

)− µ

√u

2σ

)+ (1 − gi(t))Φ

(σ

µ√

ulog(

gj(t + u)1 − gj(t + u)

1 − gi(t)gi(t)

)+

µ√

u

2σ

)for 0 ≤ u ≤ T − t with 0 ≤ t ≤ T and i, j = 0, 1 .

[Note that in the case when a = b we have c = 1/2 and the system (21.2.46)reduces to one equation only since g1 = 1 − g0 by symmetry. ]

Proof. 1. The existence of boundaries (g0, g1) satisfying (21.2.34)–21.2.37 suchthat τ∗ from (21.2.44) is optimal in (21.0.4)–(21.0.5) was proved in paragraphs2-6 above. By the local time-space formula (cf. Subsection 3.5) it follows thatthe boundaries (g0, g1) solve the system (21.2.46) (cf. (21.2.52)–(21.2.56) below).Thus it remains to show that the system (21.2.46) has no other solution in theclass of functions (h0, h1) satisfying (21.2.34)–(21.2.37).

Let us thus assume that two functions (h0, h1) satisfying (21.2.34)–(21.2.37)solve the system (21.2.46), and let us show that these two functions (h0, h1) mustthen coincide with the optimal boundaries (g0, g1) . For this, let us introduce thefunction

V h(t, π) =

Uh(t, π) if (t, π) ∈ Ch,

G(t, π) if (t, π) ∈ Dh

(21.2.49)

where the function Uh is defined by

Uh(t, π) = Et,πG(T, πT ) −∫ T−t

0

Pt,π

((t + u, πt+u) ∈ Dh

)du (21.2.50)

for all (t, π) ∈ [0, T ) × [0, 1] and the sets Ch and Dh are defined as in (21.2.6)and (21.2.7) with hi instead of gi for i = 0, 1 . Note that (21.2.50) with G(t, π)instead of Uh(t, π) on the left-hand side coincides with (21.2.46) when π = gi(t)and hj = gj for i, j = 0, 1 . Since (h0, h1) solve (21.2.46) this shows that V h iscontinuous on [0, T )× [0, 1] . We need to verify that V h coincides with the valuefunction V from (21.2.1) and that hi equals gi for i = 0, 1 .

2. Using standard arguments based on the strong Markov property (orverifying directly) it follows that V h i.e. Uh is C1,2 on Ch and that

(LV h)(t, π) = 0 for (t, π) ∈ Ch . (21.2.51)

Moreover, since Uhπ := ∂Uh/∂π is continuous on [0, T ) × (0, 1) (which is readily

verified using the explicit expressions (21.2.47) and (21.2.48) above with π insteadof gi(t) and hj instead of gj for i, j = 0, 1 ), we see that V h

π := ∂V h/∂π iscontinuous on Ch . Finally, since h0(t) ∈ (0, c) and h1(t) ∈ (c, 1) we see that V h


i.e. G is C1,2 on Dh . Therefore, with (t, π) ∈ [0, T )× (0, 1) given and fixed, thelocal time-space formula (cf. Subsection 3.5) can be applied, and in this way weget

V h(t + s, πt+s) = V h(t, π) (21.2.52)

+∫ s

0

(LV h)(t + u, πt+u) I(πt+u = h0(t + u), πt+u = h1(t + u)

)du

+ Mhs +

12

1∑i=0

∫ s

0

∆πV hπ (t + u, πt+u) I

(πt+u = hi(t + u)

)dhi

u

for 0 ≤ s ≤ T − t where

∆πV hπ (t + u, hi(t + u)) (21.2.53)

= V hπ (t + u, hi(t + u)+) − V h

π (t + u, hi(t + u)−),

the process (his )0≤s≤T−t is the local time of (πt+s)0≤s≤T−t at the boundary hi

given by

his = P- lim

ε↓012ε

∫ s

0

I(hi(t + u) − ε < πt+u < hi(t+u) + ε) (21.2.54)

× µ2

σ2π2

t+u(1−πt+u)2 du

for i = 0, 1 , and (Mhs )0≤s≤T−t defined by

Mhs =

∫ s

0

V hπ (t + u, πt+u) I(πt+u = h0(t + u), πt+u = h1(t + u)) (21.2.55)

× µ

σπt+u (1−πt+u) dWu

is a martingale under Pt,π .

Setting s = T − t in (21.2.52) and taking the Pt,π -expectation, using thatV h satisfies (21.2.51) in Ch and equals G in Dh , we get

Et,πG(T, πT ) = V h(t, π) (21.2.56)

+∫ T−t

0

Pt,π

((t + u, πt+u) ∈ Dh

)du +

12F (t, π)

where (by the continuity of the integrand) the function F is given by

F (t, π) =1∑

i=0

∫ T−t

0

∆πV hπ (t + u, hi(t + u)) duEt,πhi

u (21.2.57)


for all (t, π) ∈ [0, T ) × [0, 1] and i = 0, 1 . Thus from (21.2.56) and (21.2.49) wesee that

F (t, π) =

0 if (t, π) ∈ Ch,

2 (Uh(t, π) − G(t, π)) if (t, π) ∈ Dh

(21.2.58)

where the function Uh is given by (21.2.50).

3. From (21.2.58) we see that if we are to prove that

π → V h(t, π) is C1 at hi(t) (21.2.59)

for each 0 ≤ t < T given and fixed and i = 0, 1 , then it will follow that

Uh(t, π) = G(t, π) for all (t, π) ∈ Dh. (21.2.60)

On the other hand, if we know that (21.2.60) holds, then using the followinggeneral facts (obtained directly from the definition (21.2.49) above):

∂

∂π(Uh(t, π)−G(t, π))

∣∣∣π=h0(t)

(21.2.61)

= V hπ (t, h0(t)+) − V h

π (t, h0(t)−) = ∆πV hπ (t, h0(t)),

∂

∂π(Uh(t, π)−G(t, π))

∣∣∣π=h1(t)

(21.2.62)

= V hπ (t, h1(t)−) − V h

π (t, h1(t)+) = −∆πV hπ (t, h1(t))

for all 0 ≤ t < T , we see that (21.2.59) holds too. The equivalence of (21.2.59) and(21.2.60) suggests that instead of dealing with the equation (21.2.58) in order toderive (21.2.59) above we may rather concentrate on establishing (21.2.60) directly.

To derive (21.2.60) first note that using standard arguments based on thestrong Markov property (or verifying directly) it follows that Uh is C1,2 in Dh

and that(LUh)(t, π) = 1 for (t, π) ∈ Dh. (21.2.63)

It follows that (21.2.52) can be applied with Uh instead of V h , and this yields

Uh(t + s, πt+s) = Uh(t, π) +∫ s

0

I((t + u, πt+u) ∈ Dh) du + Nhs (21.2.64)

using (21.2.51) and (21.2.63) as well as that ∆πUhπ (t + u, hi(t + u)) = 0 for all

0 ≤ u ≤ s and i = 0, 1 since Uhπ is continuous. In (21.2.64) we have

Nhs =

∫ s

0

Uhπ (t + u, πt+u) I(πt+u = h0(t + u), πt+u = h1(t + u)) (21.2.65)

× µ

σπt+u(1−πt+u) dWu


from where we see that (Nhs )0≤s≤T−t is a martingale under Pt,π .

Next note that (21.2.52) applied to G instead of V h yields

G(t + s, πt+s) = G(t, π) +∫ s

0

I(πt+u = c) du − a + b

2cs + Ms (21.2.66)

using that LG = 1 off [0, T ]×c as well as that ∆πGπ(t + u, c) = −b − a for0 ≤ u ≤ s . In (21.2.66) we have

Ms =∫ s

0

Gπ(t + u, πt+u) I(πt+u = c)µ

σπt+u(1−πt+u) dWu (21.2.67)

=∫ s

0

[a I(πt+u <c) − b I(πt+u >c)

] µ

σπt+u(1−πt+u) dWu

from where we see that (Ms)0≤s≤T−t is a martingale under Pt,π .

For 0 < π ≤ h0(t) or h1(t) ≤ π < 1 consider the stopping time

σh = inf 0 ≤ s ≤ T − t : πt+s ∈ [h0(t + s), h1(t + s)]. (21.2.68)

Then using that Uh(t, hi(t)) = G(t, hi(t)) for all 0 ≤ t < T and i = 0, 1 since(h0, h1) solve (21.2.46)), and that Uh(T, π) = G(T, π) for all 0 ≤ π ≤ 1 , we seethat Uh(t + σh, πt+σh

) = G(t + σh, πt+σh) . Hence from (21.2.64) and (21.2.66)

using the optional sampling theorem (page 60) we find

Uh(t, π) = Et,πUh(t+σh, πt+σh) − Et,π

[ ∫ σh

0

I((t+u, πt+u) ∈ Dh) du

](21.2.69)

= Et,πG(t+σh, πt+σh) − Et,π

[∫ σh

0

I((t + u, πt+u) ∈ Dh) du

]= G(t, π)+Et,π

[ ∫ σh

0

I(πt+u = c) du

]− Et,π

[ ∫ σh

0


]= G(t, π)

since πt+u = c and (t + u, πt+u) ∈ Dh for all 0 ≤ u < σh . This establishes(21.2.60) and thus (21.2.59) holds as well.

It may be noted that a shorter but somewhat less revealing proof of (21.2.60)[and (21.2.59)] can be obtained by verifying directly (using the Markov propertyonly) that the process

Uh(t + s, πt+s) −∫ s

0

I((t + u, πt+u) ∈ Dh) du (21.2.70)

is a martingale under Pt,π for 0 ≤ s ≤ T − t . This verification moreover showsthat the martingale property of (21.2.70) does not require that h0 and h1 are


continuous and monotone (but only measurable). Taken together with the rest ofthe proof below this shows that the claim of uniqueness for the equations (21.2.46)holds in the class of continuous functions h0 and h1 from [0, T ] to R such that0 < h0(t) < c and c < h1(t) < 1 for all 0 < t < T .

4. Let us consider the stopping time

τh = inf 0 ≤ s ≤ T − t : πt+s /∈ (h0(t + s), h1(t + s)) . (21.2.71)

Observe that, by virtue of (21.2.59), the identity (21.2.52) can be written as

V h(t + s, πt+s) = V h(t, π) +∫ s

0

I((t + u, πt+u) ∈ Dh) du + Mhs (21.2.72)

with (Mhs )0≤s≤T−t being a martingale under Pt,π . Thus, inserting τh into

(21.2.72) in place of s and taking the Pt,π -expectation, by means of the optionalsampling theorem (page 60) we get

V h(t, π) = Et,πG(t + τh, πt+τh) (21.2.73)

for all (t, π) ∈ [0, T )× [0, 1] . Then comparing (21.2.73) with (21.2.1) we see that

V (t, π) ≤ V h(t, π) (21.2.74)

for all (t, π) ∈ [0, T )× [0, 1] .

5. Let us now show that g0 ≤ h0 and h1 ≤ g1 on [0, T ] . For this, recallthat by the same arguments as for V h we also have

V (t + s, πt+s) = V (t, π) +∫ s

0

I((t + u, πt+u) ∈ D) du + Mgs (21.2.75)

where (Mgs )0≤s≤T−t is a martingale under Pt,π . Fix some (t, π) belonging to

both D and Dh (first below g0 and h0 and then above g1 and h1 ) andconsider the stopping time

σg = inf 0 ≤ s ≤ T − t : πt+s ∈ [g0(t + s), g1(t + s)] . (21.2.76)

Inserting σg into (21.2.72) and (21.2.75) in place of s and taking the Pt,π -expectation, by means of the optional sampling theorem (page 60) we get

Et,πV h(t + σg, πt+σg ) (21.2.77)

= G(t, π) + Et,π

[∫ σg

0


],

Et,πV (t + σg, πt+σg ) = G(t, π) + Et,πσg. (21.2.78)

Hence by means of (21.2.74) we see that

Et,π

[∫ σg

0


]≥ Et,πσg (21.2.79)


from where, by virtue of the continuity of hi and gi on (0, T ) for i = 0, 1 ,it readily follows that D ⊆ Dh , i.e. g0(t) ≤ h0(t) and h1(t) ≤ g1(t) for all0 ≤ t ≤ T .

6. Finally, we show that hi coincides with gi for i = 0, 1 . For this, let usassume that there exists some t ∈ (0, T ) such that g0(t) < h0(t) or h1(t) < g1(t)and take an arbitrary π from (g0(t), h0(t)) or (h1(t), g1(t)) , respectively. Theninserting τ∗ = τ∗(t, π) from (21.2.33) into (21.2.72) and (21.2.75) in place of sand taking the Pt,π -expectation, by means of the optional sampling theorem (page60) we get

Et,πG(t+τ∗, πt+τ∗) = V h(t, π) + Et,π

[∫ τ∗

0


], (21.2.80)

Et,πG(t+τ∗, πt+τ∗) = V (t, π). (21.2.81)


Et,π

[∫ τ∗

0


]≤ 0 (21.2.82)

which is clearly impossible by the continuity of hi and gi for i = 0, 1 . We maytherefore conclude that V h defined in (21.2.49) coincides with V from (21.2.1)and hi is equal to gi for i = 0, 1 . This completes the proof of the theorem.

Remark 21.3. Note that without loss of generality it can be assumed that µ >0 in (21.0.2). In this case the optimal decision rule (21.2.44)–(21.2.45) can beequivalently written as follows:

τ∗ = inf 0 ≤ t ≤ T : Xt /∈ (bπ0 (t), bπ

1 (t)), (21.2.83)

d∗ =

1 (accept H1) if Xτ∗ = bπ

1 (τ∗),0 (accept H0) if Xτ∗ = bπ

0 (τ∗),(21.2.84)

where we set

bπi (t) =

σ2

µlog(

1−π

π

gi(t)1− gi(t)

)+

µ

2t (21.2.85)

for t ∈ [0, T ] , π ∈ [0, 1] and i = 0, 1 . The result proved above shows that thefollowing sequential procedure is optimal: Observe Xt for t ∈ [0, T ] and stopthe observation as soon as Xt becomes either greater than bπ

1 (t) or smaller thanbπ0 (t) for some t ∈ [0, T ] . In the first case conclude that the drift equals µ , and

in the second case conclude that the drift equals 0 .

Remark 21.4. In the preceding procedure we need to know the boundaries (bπ0 , bπ

1 )i.e. the boundaries (g0, g1) . We proved above that (g0, g1) is a unique solution ofthe system (21.2.46). This system cannot be solved analytically but can be dealtwith numerically. The following simple method can be used to illustrate the latter


(better methods are needed to achieve higher precision around the singularitypoint t = T and to increase the speed of calculation).

Set tk = kh for k = 0, 1, . . . , n where h = T/n and denote

J(t, gi(t)) = Et,gi(t)[aπT ∧ b(1−πT )] − agi(t) ∧ b(1− gi(t)), (21.2.86)

K(t, gi(t); t+u, g0(t+u), g1(t+u)) (21.2.87)

=1∑

j=0

(−1)jPt,gi(t)(πt+u ≤ gj(t+u))

for i = 0, 1 upon recalling the explicit expressions (21.2.47) and (21.2.48) above.Note that K always depends on both g0 and g1 . Then the following discreteapproximation of the integral equations (21.2.46) is valid:

J(tk, gi(tk)) =n−1∑l=k

K(tk, gi(tk); tl+1, g0(tl+1), g1(tl+1))h (21.2.88)

for k = 0, 1, . . . , n − 1 and i = 0, 1 . Setting k = n − 1 and g0(tn) = g1(tn) =c we can solve the system of two equations (21.2.88) numerically and get num-bers g0(tn−1) and g1(tn−1) . Setting k = n − 2 and using the values g0(tn−1),g0(tn), g1(tn−1), g1(tn) we can solve (21.2.88) numerically and get numbersg0(tn−2) and g1(tn−2) . Continuing the recursion we obtain gi(tn), gi(tn−1), . . . ,gi(t1), gi(t0) as an approximation of the optimal boundary gi at the pointsT, T − h, . . . , h, 0 for i = 0, 1 (cf. Figure VI.1).

Notes. The problem of sequential testing of two simple hypotheses about themean value of an observed Wiener process seeks to determine (as soon as possibleand with minimal probability error) which of the given two values is a true mean.The problem admits two different formulations (cf. Wald [216]). In the Bayesianformulation it is assumed that the unknown mean has a given distribution, and inthe variational formulation no probabilistic assumption about the unknown meanis made a priori. In this section we only study the Bayesian formulation.

The history of the problem is long and we only mention a few points startingwith Wald and Wolfowitz [218]–[219] who used the Bayesian approach to provethe optimality of the sequential probability ratio test (SPRT) in the variationalproblem for i.i.d. sequences of observations. Dvoretzky, Kiefer and Wolfowitz [51]stated without proof that if the continuous-time log-likelihood ratio process hasstationary independent increments, then the SPRT remains optimal in the vari-ational problem. Mikhalevich [136] and Shiryaev [193] (see also [196, Chap. IV])derived an explicit solution of the Bayesian and variational problem for a Wienerprocess with infinite horizon by reducing the initial optimal stopping problem toa free-boundary problem for a differential operator. A complete proof of the state-ment from [51] (under some mild assumptions) was given by Irle and Schmitz [96].


An explicit solution of the Bayesian and variational problem for a Poisson processwith infinite horizon was derived in [168] by reducing the initial optimal stoppingproblem to a free-boundary problem for a differential-difference operator (see Sec-tion 23 below). The main aim of Subsection 21.2 above (following [71]) is to derivea solution of the Bayesian problem for a Wiener process with finite horizon.

22. Quickest detection of a Wiener process

In the Bayesian formulation of problem (proposed in [188] and [190]) it is as-sumed that we observe a trajectory of the Wiener process (Brownian motion)X = (Xt)t≥0 with a drift changing from 0 to µ = 0 at some random time θtaking the value 0 with probability π and being exponentially distributed withparameter λ > 0 given that θ > 0 .

1. For a precise probabilistic formulation of the Bayesian problem it is conve-nient to assume that all our considerations take place on a probability-statisticalspace (Ω;F ; Pπ , π ∈ [0, 1]) where the probability measure Pπ has the followingstructure:

Pπ = πP0 + (1−π)∫ ∞

0

λe−λsPs ds (22.0.1)

for π ∈ [0, 1] and Ps is a probability measure specified below for s ≥ 0 . Let θbe a non-negative random variable satisfying Pπ(θ = 0) = π and Pπ(θ > t | θ >0) = e−λt for all t ≥ 0 and some λ > 0 , and let W = (Wt)t≥0 be a standardWiener process started at zero under Pπ for π ∈ [0, 1] . It is assumed that θ andW are independent.

It is further assumed that we observe a process X = (Xt)t≥0 satisfying thestochastic differential equation

dXt = µI(t ≥ θ) dt + σ dWt (X0 = 0) (22.0.2)

and thus being of the form

Xt =

σWt if t < θ,

µ(t − θ) + σWt if t ≥ θ(22.0.3)

where µ = 0 and σ2 >0 are given and fixed. Thus Pπ(X∈ · | θ = s ) = Ps(X∈ · )is the distribution law of a Wiener process with the diffusion coefficient σ > 0and a drift changing from 0 to µ at time s ≥ 0 . It is assumed that the time θof “disorder” is unknown (i.e. it cannot be observed directly).

Being based upon the continuous observation of X , our task is to find astopping time τ∗ of X (i.e. a stopping time with respect to the natural filtrationFX

t = σXs : 0 ≤ s ≤ t generated by X for t ≥ 0 ) that is “as close aspossible” to the unknown time θ . More precisely, the Wiener disorder problem

Section 22. Quickest detection of a Wiener process 309

(or the quickest detection problem for the Wiener process) consists of computingthe risk function

V (π) = infτ

(Pπ(τ < θ) + c Eπ[τ − θ]+

)(22.0.4)

and finding the optimal stopping time τ∗ at which the infimum in (22.0.4) isattained. Here Pπ(τ < θ) is the probability of a “false alarm”, Eπ[τ − θ]+ isthe “average delay” in detecting the “disorder” correctly, and c > 0 is a givenconstant. Note that τ∗ = T corresponds to the conclusion that θ ≥ T .

2. By means of standard arguments (see [196, pp. 195–197]) one can reducethe Bayesian problem (22.0.4) to the optimal stopping problem

V (π) = infτ

Eπ

[1−πτ + c

∫ τ

0

πt dt

](22.0.5)

for the a posteriori probability process πt = Pπ(θ ≤ t | FXt ) for t ≥ 0 with

Pπ(π0 = π) = 1 .

3. By the Bayes formula,

πt = πdP0

dPπ(t, X) + (1−π)

∫ t

0

dPs

dPπ(t, X)λe−λs ds (22.0.6)

where (dPs/dPπ)(t, X) is a Radon–Nikodym density of the measure Ps|FXt with

respect to the measure Pπ |FXt .

Similarly

1 − πt = (1−π)e−λt dPt

dPπ(t, X) = (1−π)e−λt dP∞

dPπ(t, X) (22.0.7)

where P∞ is the probability law (measure) of the process (σWt)t≥0 . Hence forthe likelihood ratio process

ϕt =πt

1−πt(22.0.8)

we get

ϕt = eλtZt

(ϕ0 + λ

∫ t

0

e−λs

Zsds

)= eYt

(π

1 − π+ λ

∫ t

0

e−Ys ds

)(22.0.9)

where (see Subsection 5.3)

Zt =dP0

dP∞ (t, X) ≡ d(P0|FXt )

d(P∞|FXt )

= exp(

µ

σ2

(Xt − µ

2t))

(22.0.10)

and (for further reference) we set

Yt = λt +µ

σ2

(Xt − µ

2t)

. (22.0.11)


By Ito’s formula (page 67) one gets

dZt =µ

σ2Zt dXt (22.0.12)

so that from (22.0.9) we find that

dϕt = λ(1 + ϕt) dt +µ

σ2ϕt dXt. (22.0.13)

Due to the identityπt =

ϕt

1 + ϕt(22.0.14)

it follows that

dπt =(λ − µ2

σ2π2

t

)(1−πt) dt +

µ

σ2πt(1−πt) dXt (22.0.15)

or equivalentlydπt = λ(1−πt) dt +

µ

σπt(1−πt) dWt (22.0.16)

where the innovation process (Wt)t≥0 given by

Wt =1σ

(Xt − µ

∫ t

0

πs ds

)(22.0.17)

is a standard Wiener process (see [127, Chap. IX]).

Using (22.0.9)+(22.0.11)+(22.0.14) it can be verified that (πt)t≥0 is a time-homogeneous (strong) Markov process under Pπ for π ∈ [0, 1] with respect tothe natural filtration. As the latter clearly coincides with (FX

t )t≥0 it is also clearthat the infimum in (22.0.5) can equivalently be taken over all stopping times of(πt)t≥0 .


1. In order to solve the problem (22.0.5) when the horizon is infinite let us considerthe optimal stopping problem for the Markov process (πt)t≥0 given by

V (π) = infτ

Eπ

(M(πτ ) +

∫ τ

0

L(πt) dt

)(22.1.1)

where Pπ(π0 = π) = 1 , i.e. Pπ is a probability measure under which the diffusionprocess (πt)t≥0 solving (22.0.16) above starts at π , the infimum in (22.1.1) istaken over all stopping times τ of (πt)t≥0 , and we set M(π) = 1 − π andL(π) = cπ for π ∈ [0, 1] . From (22.0.16) above we see that the infinitesimalgenerator of (πt)t≥0 is given by

L = λ(1−π)∂

∂π+

µ2

2σ2π2(1−π)2

∂2

∂π2. (22.1.2)



From (22.1.1) and (22.0.16) above we see that the closer (πt)t≥0 gets to 1the less likely that the loss will decrease by continuation. This suggests that thereexists a point A ∈ (0, 1) such that the stopping time

τA = inft ≥ 0 : πt ≥ A

(22.1.3)


Standard arguments based on the strong Markov property (cf. Chapter III)lead to the following free-boundary problem for the unknown function V and theunknown point A :

LV = −cπ for π ∈ (0, 1), (22.1.4)V (A) = 1 − A, (22.1.5)V ′(A) = −1 (smooth fit), (22.1.6)V < M for π ∈ [0, A), (22.1.7)V = M for π ∈ (A, 1]. (22.1.8)

3. To solve the free-boundary problem (22.1.4)–(22.1.8) note that the equa-tion (22.1.4) using (22.1.2) can be written as

V ′′ +λ

γ

1π2(1−π)

V ′ = − c

γ

1π(1−π)2

(22.1.9)

where we set γ = µ2/(2σ2) . This is a first order linear differential equation in V ′

and noting that ∫dπ

π2(1−π)= log

( π

1−π

)− 1

π=: α(π) (22.1.10)

the general solution of this equation is given by

V ′(π) = e−λγ α(π)

(C − c

γ

∫ π

0

eα(ρ)

ρ(1−ρ)2dρ

)(22.1.11)

where C is an undetermined constant. Since e−(λ/γ)α(π) → +∞ as π ↓ 0 , andeα(ρ) → 0 exponentially fast as ρ ↓ 0 , we see from (22.1.11) that V ′(π) → ±∞as π ↓ 0 depending on if C > 0 or C < 0 respectively. We thus choose C = 0in (22.1.11). Note that this is equivalent to the fact that V ′(0+) = 0 .

With this choice of C denote the right-hand side of (22.1.11) by ψ(π) , i.e.let

ψ(π) = − c

γe−

λγ α(π)

∫ π

0

eα(ρ)

ρ(1− ρ)2dρ (22.1.12)


for π ∈ (0, 1) . It is then easy verified that there exists a unique root A∗ of theequation

ψ(A∗) = −1 (22.1.13)

corresponding to (22.1.6) above. To meet (22.1.5) and (22.1.8) as well let us set

V∗(π) =

(1−A∗) +

∫ π

A∗ψ(ρ) dρ if π ∈ [0, A∗),

1 − π if π ∈ [A∗, 1](22.1.14)

for π ∈ [0, 1] .

The preceding analysis shows that the function V∗ defined by (22.1.14)is the unique solution of the free-boundary problem (22.1.4)–(22.1.8) satisfying|V ′∗(0+)| < ∞ (or equivalently being bounded at zero). Note that V∗ is C2 on[0, A∗) ∪ (A∗, 1] but only C1 at A∗ . Note also that V∗ is concave on [0, 1] .

4. In this way we have arrived at the conclusions of the following theorem.

Theorem 22.1. The value function V from (22.1.1) is given explicitly by (22.1.14)above. The stopping time τA∗ given by (22.1.3) above is optimal in (22.1.1).

Proof. The properties of V∗ stated in the end of paragraph 3 above show that Ito’sformula (page 67) can be applied to V∗(πt) in its standard form (cf. Subsection3.5). This gives

V∗(πt) = V∗(π) +∫ t

0

LV∗(πs) I(πs = A∗) ds (22.1.15)

+µ

σ

∫ t

0

πs(1−πs)V ′∗(πs) dWs.

Recalling that V (π) = 1 − π for π ∈ (A∗, 1] and using that V∗ satisfies (22.1.4)for π ∈ (0, A∗) , we see that

LV∗(π) ≥ −c π (22.1.16)

for all π ∈ [ λ/(λ + c), 1] and thus for all π ∈ (0, 1] since A∗ ≥ λ/(λ + c) as iseasily seen. By (22.1.7), (22.1.8), (22.1.15) and (22.1.16) it follows that

M(πt) ≥ V∗(πt) ≥ V∗(π) −∫ t

0

L(πs) ds + Mt (22.1.17)


Mt =µ

σ

∫ t

0

πs(1−πs)V ′(πs) dWs. (22.1.18)

Using that |V ′∗(π)| ≤ 1 < ∞ for all π ∈ [0, 1] it is easily verified by stan-dard means that M is a martingale. Moreover, by the optional sampling theorem


(page 60) this bound also shows that EπMτ = 0 whenever Eπ√

τ < ∞ for astopping time τ . In particular, the latter condition is satisfied if Eπτ < ∞ . Asclearly in (22.1.1) it is enough to take the infimum only over stopping times τsatisfying Eπτ < ∞ , we may insert τ in (22.1.17) instead of t , take Eπ on bothsides, and conclude that

Eπ

(M(πτ ) +

∫ τ

0

L(Xt) dt

)≥ V∗(π) (22.1.19)

for all π ∈ [0, 1] . This shows that V ≥ V∗ . On the other hand, using (22.1.4) andthe definition of τA∗ in (22.1.3), we see from (22.1.15) that

M(πτA∗

)= V∗

(πτA∗

)= V∗(π) +

∫ τA∗

0

L(Xt) dt + MτA∗ . (22.1.20)

Since EπτA∗ < ∞ (being true for any A ) we see by taking Eπ on both sides of(22.1.20) that equality in (22.1.19) is attained at τ = τA∗ , and thus V = V∗ .This completes the proof.

For more details on the Wiener disorder problem with infinite horizon (in-cluding a fixed probability error formulation) we refer to [196, Chap. 4, Sect. 3–4].


1. Solution of the Bayesian problem. In order to solve the problem (22.0.5) whenthe horizon T is finite, let us consider the extended optimal stopping problem forthe Markov process (t, πt)0≤t≤T given by

V (t, π) = inf0≤τ≤T−t

Et,π

[G(πt+τ ) +

∫ τ

0

H(πt+s) ds

](22.2.1)

where Pt,π(πt = π) = 1 , i.e. Pt,π is a probability measure under which the dif-fusion process (πt+s)0≤s≤T−t solving (22.0.16) starts at π at time t , the infi-mum in (22.2.1) is taken over all stopping times τ of (πt+s)0≤s≤T−t , and we setG(π) = 1− π and H(π) = c π for all π ∈ [0, 1] . Note that (πt+s)0≤s≤T−t underPt,π is equally distributed as (πs)0≤s≤T−t under Pπ . This fact will be frequentlyused in the sequel without further mention. Since G and H are bounded andcontinuous on [0, 1] it is possible to apply Corollary 2.9 (Finite horizon) withRemark 2.10 and conclude that an optimal stopping time exists in (22.2.1).

2. Let us now determine the structure of the optimal stopping time in theproblem (22.2.1).

(i) Note that by (22.0.16) we get

G(πt+s) = G(π) − λ

∫ s

0

(1−πt+u) du + Ms (22.2.2)


where the process (Ms)0≤s≤T−t defined by Ms = − ∫ s

0 (µ/σ)πt+u(1−πt+u)dWu

is a continuous martingale under Pt,π . It follows from (22.2.2) using the optionalsampling theorem (page 60) that

Et,π

[G(πt+σ) +

∫ σ

0

H(πt+u) du

](22.2.3)

= G(π) + Et,π

[∫ σ

0

((λ + c)πt+u − λ) du

]for each stopping time σ of (πt+s)0≤s≤T−t . Choosing σ to be the exit timefrom a small ball, we see from (22.2.3) that it is never optimal to stop whenπt+s < λ/(λ + c) for 0 ≤ s < T − t . In other words, this shows that all points(t, π) for 0 ≤ t < T with 0 ≤ π < λ/(λ + c) belong to the continuation set

C = (t, π) ∈ [0, T )×[0, 1] : V (t, π) < G(π). (22.2.4)

(ii) Recalling the solution to the problem (2.5) in the case of infinite horizon,where the stopping time τ∗ = inf t > 0 : πt ≥ A∗ is optimal and 0 < A∗ <1 is uniquely determined from the equation (22.1.13) (see also (4.147) in [196,p. 201]), we see that all points (t, π) for 0 ≤ t ≤ T with A∗ ≤ π ≤ 1 belongto the stopping set. Moreover, since π → V (t, π) with 0 ≤ t ≤ T given andfixed is concave on [0, 1] (this is easily deduced using the same arguments as in[196, pp. 197–198]), it follows directly from the previous two conclusions aboutthe continuation and stopping set that there exists a function g satisfying 0 <λ/(λ + c) ≤ g(t) ≤ A∗ < 1 for all 0 ≤ t ≤ T such that the continuation set is anopen set of the form

C = (t, π) ∈ [0, T )×[0, 1] : π < g(t) (22.2.5)

and the stopping set is the closure of the set

D = (t, π) ∈ [0, T )×[0, 1] : π > g(t). (22.2.6)

(Below we will show that V is continuous so that C is open indeed. We will alsosee that g(T ) = λ/(λ + c) .)

(iii) Since the problem (22.2.1) is time-homogeneous, in the sense that Gand H are functions of space only (i.e. do not depend on time), it follows that themap t → V (t, π) is increasing on [0, T ] . Hence if (t, π) belongs to C for someπ ∈ [0, 1] and we take any other 0 ≤ t′ < t ≤ T , then V (t′, π) ≤ V (t, π) < G(π) ,showing that (t′, π) belongs to C as well. From this we may conclude in (22.2.5)–(22.2.6) that the boundary t → g(t) is decreasing on [0, T ] .

(iv) Let us finally observe that the value function V from (22.2.1) and theboundary g from (22.2.5)–(22.2.6) also depend on T and let them be denoted hereby V T and gT , respectively. Using the fact that T → V T (t, π) is a decreasingfunction on [t,∞) and V T (t, π) = G(π) for all π ∈ [gT (t), 1] , we conclude that


if T < T ′ , then 0 ≤ gT (t) ≤ gT ′(t) ≤ 1 for all t ∈ [0, T ] . Letting T ′ in the

previous expression go to ∞ , we get that 0 < λ/(λ + c) ≤ gT (t) ≤ A∗ < 1 andA∗ ≡ limT→∞ gT (t) for all t ≥ 0 , where A∗ is the optimal stopping point in theinfinite horizon problem referred to above (cf. Subsection 22.1).

3. Let us now show that the value function (t, π) → V (t, π) is continuous on[0, T ]× [0, 1] . For this it is enough to prove that

π → V (t0, π) is continuous at π0, (22.2.7)t → V (t, π) is continuous at t0 uniformly over π ∈ [π0 − δ, π0 + δ] (22.2.8)

for each (t0, π0) ∈ [0, T ] × [0, 1] with some δ > 0 small enough (it may dependon π0 ). Since (22.2.7) follows by the fact that π → V (t, π) is concave on [0, 1] ,it remains to establish (22.2.8).

For this, let us fix arbitrary 0 ≤ t1 < t2 ≤ T and 0 ≤ π ≤ 1 , and letτ1 = τ∗(t1, π) denote the optimal stopping time for V (t1, π) . Set τ2 = τ1∧(T−t2)and note since t → V (t, π) is increasing on [0, T ] and τ2 ≤ τ1 that we have

0 ≤ V (t2, π) − V (t1, π) (22.2.9)

≤ Eπ

[1 − πτ2 + c

∫ τ2

0

πu du]− Eπ

[1 − πτ1 + c

∫ τ1

0

πu du]

≤ Eπ[πτ1 − πτ2 ].

From (22.0.16) using the optional sampling theorem (page 60) we find that

Eππσ = π + λEπ

[∫ σ

0

(1−πt) dt

](22.2.10)

for each stopping time σ of (πt)0≤t≤T . Hence by the fact that τ1 − τ2 ≤ t2 − t1we get

Eπ[πτ1 − πτ2 ] = λEπ

[∫ τ1

0

(1−πt) dt −∫ τ2

0

(1−πt) dt

](22.2.11)

= λEπ

[∫ τ1

τ2

(1−πt) dt

]≤ λEπ[τ1 − τ2] ≤ λ (t2 − t1)

for all 0 ≤ π ≤ 1 . Combining (22.2.9) with (22.2.11) we see that (22.2.8) follows.In particular, this shows that the instantaneous-stopping condition (22.2.33) belowis satisfied.

4. In order to prove that the smooth-fit condition (22.2.34) below holds, i.e.that π → V (t, π) is C1 at g(t) , let us fix a point (t, π) ∈ [0, T )× (0, 1) lying onthe boundary g so that π = g(t) . Then for all ε > 0 such that 0 < π − ε < πwe have

V (t, π) − V (t, π− ε)ε

≥ G(π) − G(π− ε)ε

= −1 (22.2.12)



∂−V

∂π(t, π) ≥ G′(π) = −1 (22.2.13)

where the left-hand derivative in (22.2.13) exists (and is finite) by virtue of theconcavity of π → V (t, π) on [0, 1] . Note that the latter will also be provedindependently below.

Let us now fix some ε > 0 such that 0 < π − ε < π and consider thestopping time τε = τ∗(t, π− ε) being optimal for V (t, π− ε) . Note that τε is thefirst exit time of the process (πt+s)0≤s≤T−t from the set C in (22.2.5). Then from(22.2.1) using the equation (22.0.16) and the optional sampling theorem (page 60)we obtain

V (t, π) − V (t, π − ε) (22.2.14)

≤ Eπ

[1 − πτε + c

∫ τε

0

πu du

]− Eπ−ε

[1 − πτε + c

∫ τε

0

πu du

]= Eπ

[1 − πτε + c

(τε +

π − πτε

λ

)]− Eπ−ε

[1 − πτε + c

(τε +

π − ε − πτε

λ

)]=( c

λ+ 1

)(Eπ−επτε − Eππτε

)+ c(Eπτε − Eπ−ετε

)+ ε

c

λ.

By (22.0.1) and (22.0.9)+(22.0.11)+(22.0.14) it follows that

Eπ−επτε − Eππτε (22.2.15)

= (π− ε)E0S(π− ε) + (1−π+ε)∫ ∞

0

λe−λsEsS(π− ε) ds

− πE0S(π) − (1−π)∫ ∞

0

λe−λsEsS(π) ds

= πE0[S(π− ε) − S(π)] + (1−π)∫ ∞

0

λe−λsEs[S(π− ε) − S(π)] ds

− εE0S(π − ε) + ε

∫ ∞

0

λe−λsEsS(π − ε) ds

where the function S is defined by

S(π) = eYτε

(π

1−π+ λ

∫ τε

0

e−Yu du

)(22.2.16)

×(

1 + eYτε

(π

1−π+ λ

∫ τε

0

e−Yu du

))−1

.


By virtue of the mean value theorem there exists ξ ∈ [π − ε, π] such that

πE0[S(π− ε) − S(π)] + (1−π)∫ ∞

0

λe−λsEs[S(π− ε) − S(π)] ds (22.2.17)

= −ε

(πE0S′(ξ) + (1−π)

∫ ∞

0

λe−λsEsS′(ξ) ds

)where S′ is given by

S′(ξ) = eYτε

/((1 − ξ)2

[1 + eYτε

(ξ

1 − ξ+ λ

∫ τε

0

e−Yu du

)]2). (22.2.18)

Considering the second term on the right-hand side of (22.2.14) we find using(22.0.1) that

c(Eπτε − Eπ−ετε

)= cε

(E0τε +

∫ ∞

0

λe−λsEsτε ds)

(22.2.19)

=cε

1 − π

((1−2π)E0τε + Eπτε

).

Recalling that τε is equally distributed as τε = inf 0 ≤ s ≤ T − t : ππ−εs ≥

g(t+ s) , where we write ππ−εs to indicate dependance on the initial point π− ε

through (22.0.9) in (22.0.14) above, and considering the hitting time σε to theconstant level π = g(t) given by σε = inf s ≥ 0 : ππ−ε

s ≥ π , it followsthat τε ≤ σε for every ε > 0 since g is decreasing, and σε ↓ σ0 as ε ↓ 0 whereσ0 = inf s > 0 : ππ

s ≥ π . On the other hand, since the diffusion process (ππs )s≥0

solving (22.0.16) is regular (see e.g. [174, Chap. 7, Sect. 3]), it follows that σ0 = 0Pπ -a.s. This in particular shows that τε → 0 Pπ -a.s. Hence we easily find that

S(π − ε) → π, S(ξ) → π and S′(ξ) → 1 Pπ-a.s. (22.2.20)

as ε ↓ 0 for s ≥ 0 , and clearly |S′(ξ)| ≤ K with some K > 0 large enough.From (22.2.14) using (22.2.15)–(22.2.20) it follows that:

V (t, π) − V (t, π − ε)ε

≤( c

λ+ 1

)(− 1 + o(1)

)+ o(1) +

c

λ(22.2.21)

= −1 + o(1)

as ε ↓ 0 by the dominated convergence theorem and the fact that P0 Pπ . Thiscombined with (22.2.12) above proves that V −

π (t, π) exists and equals G′(π) =−1 .

5. We proceed by proving that the boundary g is continuous on [0, T ] andthat g(T ) = λ/(λ + c) .


(i) Let us first show that the boundary g is right-continuous on [0, T ] .For this, fix t ∈ [0, T ) and consider a sequence tn ↓ t as n → ∞ . Since gis decreasing, the right-hand limit g(t+) exists. Because (tn, g(tn)) ∈ D for alln ≥ 1 , and D is closed, we see that (t, g(t+)) ∈ D . Hence by (22.2.6) we seethat g(t+) ≥ g(t) . The reverse inequality follows obviously from the fact that gis decreasing on [0, T ] , thus proving the claim.

(ii) Suppose that at some point t∗ ∈ (0, T ) the function g makes a jump,i.e. let g(t∗−) > g(t∗) ≥ λ/(λ + c) . Let us fix a point t′ < t∗ close to t∗ andconsider the half-open set R ⊂ C being a curved trapezoid formed by the vertices(t′, g(t′)) , (t∗, g(t∗−)) , (t∗, π′) and (t′, π′) with any π′ fixed arbitrarily in theinterval (g(t∗), g(t∗−)) . Observe that the strong Markov property implies that thevalue function V from (22.2.1) is C1,2 on C . Note also that the gain function Gis C2 in R so that by the Newton–Leibniz formula using (22.2.33) and (22.2.34)it follows that

V (t, π) − G(π) =∫ g(t)

π

∫ g(t)

u

(∂2V

∂π2(t, v) − ∂2G

∂π2(v))

dv du (22.2.22)

for all (t, π) ∈ R .

Since t → V (t, π) is increasing, we have

∂V

∂t(t, π) ≥ 0 (22.2.23)

for each (t, π) ∈ C . Moreover, since π → V (t, π) is concave and (22.2.34) holds,we see that

∂V

∂π(t, π) ≥ −1 (22.2.24)

for each (t, π) ∈ C . Finally, since the strong Markov property implies that thevalue function V from (22.2.1) solves the equation (22.2.32), using (22.2.23) and(22.2.24) we obtain

∂2V

∂π2(t, π) =

2σ2

µ2

1π2(1−π)2

(−cπ − λ(1−π)

∂V

∂π(t, π) − ∂V

∂t(t, π)

)(22.2.25)

≤ 2σ2

µ2

1π2(1−π)2

(−cπ + λ(1−π)) ≤ −εσ2

µ2

for all t′ ≤ t < t∗ and all π′ ≤ π < g(t′) with ε > 0 small enough. Note in(22.2.25) that −cπ + λ(1− π) < 0 since all points (t, π) for 0 ≤ t < T with0 ≤ π < λ/(λ + c) belong to C and consequently g(t∗) ≥ λ/(λ + c) .

Hence by (22.2.22) using that Gππ = 0 we get

V (t′, π′) − G(π′) ≤ −εσ2

µ2

(g(t′) − π′)2

2(22.2.26)

→ −εσ2

µ2

(g(t∗−) − π′)2

2< 0


as t′ ↑ t∗ . This implies that V (t∗, π′) < G(π′) which contradicts the fact that(t∗, π′) belongs to the stopping set D . Thus g(t∗−) = g(t∗) showing that g iscontinuous at t∗ and thus on [0, T ] as well.

(iii) We finally note that the method of proof from the previous part (ii)also implies that g(T ) = λ/(λ + c) . To see this, we may let t∗ = T and likewisesuppose that g(T−) > λ/(λ + c) . Then repeating the arguments presented aboveword by word we arrive at a contradiction with the fact that V (T, π) = G(π) forall π ∈ [λ/(λ + c), g(T−)] thus proving the claim.

6. Summarizing the facts proved in paragraphs 2–5 above we may concludethat the following exit time is optimal in the extended problem (22.2.1):

τ∗ = inf 0 ≤ s ≤ T − t : πt+s ≥ g(t + s) (22.2.27)

(the infimum of an empty set being equal T − t ) where the boundary g satisfiesthe following properties (see Figure VI.2):

g : [0, T ] → [0, 1] is continuous and decreasing, (22.2.28)λ/(λ + c) ≤ g(t) ≤ A∗ for all 0 ≤ t ≤ T , (22.2.29)g(T ) = λ/(λ + c) (22.2.30)

where A∗ satisfying 0 < λ/(λ + c) < A∗ < 1 is the optimal stopping point forthe infinite horizon problem uniquely determined from the transcendental equation(22.1.13) (or (4.147) in [196, p. 201]).

Standard arguments imply that the infinitesimal operator L of the process(t, πt)0≤t≤T acts on a function f ∈ C1,2([0, T )× [0, 1]) according to the rule

(Lf)(t, π) =(

∂f

∂t+ λ(1−π)

∂f

∂π+

µ2

2σ2π2(1−π)2

∂2f

∂π2

)(t, π) (22.2.31)

for all (t, π) ∈ [0, T )×[0, 1] . In view of the facts proved above we are thus natu-rally led to formulate the following free-boundary problem for the unknown valuefunction V from (22.2.1) and the unknown boundary g from (22.2.5)–(22.2.6):

(LV )(t, π) = −cπ for (t, π) ∈ C, (22.2.32)

V (t, π)∣∣π=g(t)− = 1 − g(t) (instantaneous stopping), (22.2.33)

∂V

∂π(t, π)

∣∣∣π=g(t)−

= −1 (smooth fit), (22.2.34)

V (t, π) < G(π) for (t, π) ∈ C, (22.2.35)V (t, π) = G(π) for (t, π) ∈ D, (22.2.36)

where C and D are given by (22.2.5) and (22.2.6), and the condition (22.2.33) issatisfied for all 0 ≤ t ≤ T and the condition (22.2.34) is satisfied for all 0 ≤ t < T .


1

πt π t→

→t g(t)

0Tτ∗

λλ+c

Figure VI.2: A computer drawing of the optimal stopping boundary gfrom Theorem 22.2. At time τ∗ it is optimal to stop and conclude thatthe drift has been changed (from 0 to µ ).

Note that the superharmonic characterization of the value function (cf. Chap-ter I) implies that V from (22.2.1) is a largest function satisfying (22.2.32)–(22.2.33) and (22.2.35)–(22.2.36).

7. Making use of the facts proved above we are now ready to formulate themain result of this subsection.

Theorem 22.2. In the Bayesian formulation of the Wiener disorder problem(22.0.4)–(22.0.5) the optimal stopping time τ∗ is explicitly given by

τ∗ = inf 0 ≤ t ≤ T : πt ≥ g(t) (22.2.37)

where g can be characterized as a unique solution of the nonlinear integral equa-tion

Et,g(t)πT = g(t) + c

∫ T−t

0

Et,g(t)

[πt+u I(πt+u < g(t + u))

]du (22.2.38)

+ λ

∫ T−t

0

Et,g(t)

[(1−πt+u) I(πt+u > g(t+u))

]du

for 0 ≤ t ≤ T satisfying (22.2.28)–(22.2.30) [see Figure VI.2].


More explicitly, the three terms in the equation (22.2.38) are given as follows:

Et,g(t)πT = g(t) + (1− g(t))(1 − e−λ(T−t)

), (22.2.39)

Et,g(t)

[πt+u I(πt+u < g(t+u))

]=∫ g(t+u)

0

x p(g(t); u, x) dx, (22.2.40)

Et,g(t)

[(1−πt+u) I(πt+u > g(t+u))

]=∫ 1

g(t+u)

(1−x) p(g(t); u, x) dx (22.2.41)

for 0 ≤ u ≤ T − t with 0 ≤ t ≤ T , where p is the transition density function ofthe process (πt)0≤t≤T given in (22.2.103) below.

Proof. 1. The existence of a boundary g satisfying (22.2.28)–(22.2.30) such thatτ∗ from (22.2.37) is optimal in (22.0.4)–(22.0.5) was proved in paragraphs 2–6above. By the local time-space formula (cf. Subsection 3.5) it follows that theboundary g solves the equation (22.2.38) (cf. (22.2.45)–(22.2.48) below). Thus itremains to show that the equation (22.2.38) has no other solution in the class offunctions h satisfying (22.2.28)–(22.2.30).

Let us thus assume that a function h satisfying (22.2.28)–(22.2.30) solvesthe equation (22.2.38), and let us show that this function h must then coincidewith the optimal boundary g . For this, let us introduce the function

V h(t, π) =

Uh(t, π) if π < h(t),G(π) if π ≥ h(t),

(22.2.42)

where the function Uh is defined by

Uh(t, π) = Et,πG(πT ) + c

∫ T−t

0

Et,π

[πt+u I(πt+u < h(t+u))

]du (22.2.43)

+ λ

∫ T−t

0

Et,π

[(1−πt+u) I(πt+u > h(t+u))

]du

for all (t, π) ∈ [0, T ) × [0, 1] . Note that (22.2.43) with G(π) instead of Uh(t, π)on the left-hand side coincides with (22.2.38) when π = g(t) and h = g . Sinceh solves (22.2.38) this shows that V h is continuous on [0, T ) × [0, 1] . We needto verify that V h coincides with the value function V from (22.2.1) and that hequals g .

2. Using standard arguments based on the strong Markov property (orverifying directly) it follows that V h i.e. Uh is C1,2 on Ch and that

(LV h)(t, π) = −cπ for (t, π) ∈ Ch (22.2.44)


where Ch is defined as in (22.2.5) with h instead of g . Moreover, since Uhπ :=

∂Uh/∂π is continuous on [0, T )×(0, 1) (which is readily verified using the explicitexpressions (22.2.39)–(22.2.41) above with π instead of g(t) and h instead ofg ), we see that V h

π := ∂V h/∂π is continuous on Ch . Finally, it is clear that V h

i.e. G , is C1,2 on Dh , where Dh is defined as in (22.2.6) with h instead ofg . Therefore, with (t, π) ∈ [0, T ) × (0, 1) given and fixed, the local time-spaceformula (cf. Subsection 3.5) can be applied, and in this way we get

V h(t+s, πt+s) = V h(t, π) (22.2.45)

+∫ s

0

(LV h)(t+u, πt+u) I(πt+u = h(t+u)) du

+ Mhs +

12

∫ s

0

∆πV hπ (t+u, πt+u) I(πt+u = h(t+u)) dh

u

for 0 ≤ s ≤ T − t where ∆πV hπ (t + u, h(t + u)) = V h

π (t + u, h(t + u)+) −V h

π (t+u, h(t+u)−) , the process (hs )0≤s≤T−t is the local time of (πt+s)0≤s≤T−t

at the boundary h given by

hs = Pt,π- lim

ε↓012ε

∫ s

0

I(h(t + u) − ε < πt+u < h(t + u) + ε) (22.2.46)

× µ2

σ2π2

t+u(1−πt+u)2 du

and (Mhs )0≤s≤T−t defined by

Mhs =

∫ s

0

V hπ (t + u, πt+u) I(πt+u = h(t + u))

µ

σπt+u(1−πt+u) dWu (22.2.47)

is a martingale under Pt,π .

Setting s = T − t in (22.2.45) and taking the Pt,π -expectation, using thatV h satisfies (22.2.44) in Ch and equals G in Dh , we get

Et,πG(πT ) = V h(t, π) − c

∫ T−t

0

Et,π

[πt+u I(πt+u < h(t + u))

]du (22.2.48)

− λ

∫ T−t

0

Et,π

[(1−πt+u) I(πt+u > h(t+u))

]du +

12F (t, π)

where (by the continuity of the integrand) the function F is given by

F (t, π) =∫ T−t

0

∆πV hπ (t + u, h(t + u)) duEt,πh

u (22.2.49)


for all (t, π) ∈ [0, T )× [0, 1] . Thus from (22.2.48) and (22.2.42) we see that

F (t, π) =

0 if π < h(t),

2 (Uh(t, π) − G(π)) if π ≥ h(t)(22.2.50)

where the function Uh is given by (22.2.43).


π → V h(t, π) is C1 at h(t) (22.2.51)

for each 0 ≤ t < T given and fixed, then it will follow that

Uh(t, π) = G(π) for all h(t) ≤ π ≤ 1. (22.2.52)

On the other hand, if we know that (22.2.52) holds, then using the general factobtained directly from the definition (22.2.42) above,

∂

∂π(Uh(t, π)−G(π))

∣∣∣π=h(t)

= V hπ (t, h(t)−) − V h

π (t, h(t)+) (22.2.53)

= −∆πV hπ (t, h(t))

for all 0 ≤ t < T , we see that (22.2.51) holds too. The equivalence of (22.2.51) and(22.2.52) suggests that instead of dealing with the equation (22.2.50) in order toderive (22.2.51) above we may rather concentrate on establishing (22.2.52) directly.

To derive (22.2.52) first note that using standard arguments based on thestrong Markov property (or verifying directly) it follows that Uh is C1,2 in Dh

and that(LUh)(t, π) = −λ(1−π) for (t, π) ∈ Dh. (22.2.54)

It follows that (22.2.45) can be applied with Uh instead of V h , and this yields

Uh(t + s, πt+s) = Uh(t, π) − c

∫ s

0

πt+u I(πt+u < h(t + u)) du (22.2.55)

− λ

∫ s

0

(1−πt+u) I(πt+u > h(t + u)) du + Nhs

using (22.2.44) and (22.2.54) as well as that ∆πUhπ (t + u, h(t + u)) = 0 for

all 0 ≤ u ≤ s since Uhπ is continuous. In (22.2.55) we have Nh

s =∫ s

0Uh

π (t +u, πt+u) I(πt+u = h(t+u)) (µ/σ)πt+u(1−πt+u) dWu and (Nh

s )0≤s≤T−t is a mar-tingale under Pt,π .

For h(t) ≤ π < 1 consider the stopping time

σh = inf 0 ≤ s ≤ T − t : πt+s ≤ h(t + s) . (22.2.56)


Then using that Uh(t, h(t)) = G(h(t)) for all 0 ≤ t < T since h solves (22.2.38),and that Uh(T, π) = G(π) for all 0 ≤ π ≤ 1 , we see that Uh(t + σh, πt+σh

) =G(πt+σh

) . Hence from (22.2.55) and (22.2.2) using the optional sampling theorem(page 60) we find

Uh(t, π) = Et,πUh(t+σh, πt+σh) (22.2.57)

+ cEt,π

[ ∫ σh

0

πt+u I(πt+u <h(t+u)) du]

+ λEt,π

[ ∫ σh

0

(1−πt+u) I(πt+u > h(t + u)) du]

= Et,πG(πt+σh) + c Et,π

[ ∫ σh

0

πt+u I(πt+u < h(t + u)) du]

+ λEt,π

[ ∫ σh

0

(1−πt+u) I(πt+u > h(t + u)) du]

= G(π) − λEt,π

[ ∫ σh

0

(1−πt+u) du]

+ cEt,π

[ ∫ σh

0

πt+u I(πt+u < h(t + u)) du]

+ λEt,π

[ ∫ σh

0

(1−πt+u) I(πt+u > h(t + u)) du]

= G(π)

since πt+u > h(t + u) for all 0 ≤ u < σh . This establishes (22.2.52) and thus(22.2.51) holds as well.


Uh(t + s, πt+s) + c

∫ s

0

πt+u I(πt+u < h(t + u)) du (22.2.58)

+ λ

∫ s

0

(1−πt+u) I(πt+u > h(t + u)) du

is a martingale under Pt,π for 0 ≤ s ≤ T − t . This verification moreover showsthat the martingale property of (22.2.58) does not require that h is continuousand increasing (but only measurable). Taken together with the rest of the proofbelow this shows that the claim of uniqueness for the equation (22.2.38) holds inthe class of continuous functions h : [0, T ] → R such that 0 ≤ h(t) ≤ 1 for all0 ≤ t ≤ T .

4. Let us consider the stopping time

τh = inf 0 ≤ s ≤ T − t : πt+s ≥ h(t + s) . (22.2.59)


Observe that, by virtue of (22.2.51), the identity (22.2.45) can be written as

V h(t + s, πt+s) = V h(t, π) − c

∫ s

0

πt+u I(πt+u < h(t + u)) du (22.2.60)

− λ

∫ s

0

(1−πt+u) I(πt+u > h(t + u)) du + Mhs

with (Mhs )0≤s≤T−t being a martingale under Pt,π . Thus, inserting τh into

(22.2.60) in place of s and taking the Pt,π -expectation, by means of the optionalsampling theorem (page 60) we get

V h(t, π) = Et,π

[G(πt+τh

) + c

∫ τh

0

πt+u du

](22.2.61)

for all (t, π) ∈ [0, T )× [0, 1] . Then comparing (22.2.61) with (22.2.1) we see that

V (t, π) ≤ V h(t, π) (22.2.62)

for all (t, π) ∈ [0, T )× [0, 1] .

5. Let us now show that h ≤ g on [0, T ] . For this, recall that by the samearguments as for V h we also have

V (t + s, πt+s) = V (t, π) − c

∫ s

0

πt+u I(πt+u < g(t + u)) du (22.2.63)

− λ

∫ s

0

(1−πt+u) I(πt+u > g(t + u)) du + Mgs

where (Mgs )0≤s≤T−t is a martingale under Pt,π . Fix some (t, π) such that π >

g(t) ∨ h(t) and consider the stopping time

σg = inf 0 ≤ s ≤ T − t : πt+s ≤ g(t + s) . (22.2.64)

Inserting σg into (22.2.60) and (22.2.63) in place of s and taking the Pt,π -expectation, by means of the optional sampling theorem (page 60) we get

Et,π

[V h(t+σg, πt+σg ) + c

∫ σg

0

πt+u du

](22.2.65)

= G(π) + Et,π

[∫ σg

0

(cπt+u − λ(1−πt+u)) I(πt+u > h(t+u)) du

],

Et,π

[V (t+σg, πt+σg ) + c

∫ σg

0

πt+u du

](22.2.66)

= G(π) + Et,π

[∫ σg

0

(cπt+u − λ(1−πt+u)) du

].



Et,π

[∫ σg

0


](22.2.67)

≥ Et,π

[∫ σg

0

(cπt+u − λ(1−πt+u)) du

]from where, by virtue of the continuity of h and g on (0, T ) and the firstinequality in (22.2.29), it readily follows that h(t) ≤ g(t) for all 0 ≤ t ≤ T .

6. Finally, we show that h coincides with g . For this, let us assume thatthere exists some t ∈ (0, T ) such that h(t) < g(t) and take an arbitrary πfrom (h(t), g(t)) . Then inserting τ∗ = τ∗(t, π) from (22.2.27) into (22.2.60) and(22.2.63) in place of s and taking the Pt,π -expectation, by means of the optionalsampling theorem (page 60) we get

Et,π

[G(πt+τ∗) + c

∫ τ∗

0

πt+u du

]= V h(t, π) (22.2.68)

+ Et,π

[∫ τ∗

0

(cπt+u − λ(1−πt+u)) I(πt+u > h(t + u)) du

],

Et,π

[G(πt+τ∗) + c

∫ τ∗

0

πt+u du

]= V (t, π). (22.2.69)


Et,π

[∫ τ∗

0


]≤ 0 (22.2.70)

which is clearly impossible by the continuity of h and g and the fact that h ≥λ/(λ + c) on [0, T ] . We may therefore conclude that V h defined in (22.2.42)coincides with V from (22.2.1) and h is equal to g . This completes the proof ofthe theorem.

Remark 22.3. Note that without loss of generality it can be assumed that µ >0 in (22.0.2)–(22.0.3). In this case the optimal stopping time (22.2.37) can beequivalently written as follows:

τ∗ = inf 0 ≤ t ≤ T : Xt ≥ bπ(t, Xt0) (22.2.71)

where we set

bπ(t, Xt0) =

σ2

µlog

g(t)/(1− g(t))

π/(1−π) + λ∫ t

0e−λse−

µσ2 (Xs−µs

2 ) ds(22.2.72)

+(

µ

2− λσ2

µ

)t


for (t, π) ∈ [0, T ]×[0, 1] and Xt0 denotes the sample path s → Xs for s ∈ [0, t] .

The result proved above shows that the following sequential procedure is optimal:Observe Xt for t ∈ [0, T ] and stop the observation as soon as Xt becomes greaterthan bπ(t, Xt

0) for some t ∈ [0, T ] . Then conclude that the drift has been changedfrom 0 to µ .

Remark 22.4. In the preceding procedure we need to know the boundary bπ i.e.the boundary g . We proved above that g is a unique solution of the equation(22.2.38). This equation cannot be solved analytically but can be dealt with nu-merically. The following simple method can be used to illustrate the latter (bettermethods are needed to achieve higher precision around the singularity point t = Tand to increase the speed of calculation). See also paragraph 3 of Section 27 belowfor further remarks on numerics.

Set tk = kh for k = 0, 1, . . . , n where h = T/n and denote

J(t, g(t)) = (1 − g(t))(1 − e−λ(T−t)

), (22.2.73)

K(t, g(t); t + u, g(t + u)) (22.2.74)

= Et,g(t)

[cπt+uI(πt+u < g(t + u)) + λ(1−πt+u)I(πt+u > g(t+u))

]upon recalling the explicit expressions (22.2.40) and (22.2.41) above. Then thefollowing discrete approximation of the integral equation (22.2.38) is valid:

J(tk, g(tk)) =n−1∑l=k

K(tk, g(tk); tl+1, g(tl+1))h (22.2.75)

for k = 0, 1, . . . , n−1 . Setting k = n−1 and g(tn) = λ/(λ+ c) we can solve theequation (22.2.75) numerically and get a number g(tn−1) . Setting k = n− 2 andusing the values g(tn−1) , g(tn) we can solve (22.2.75) numerically and get a num-ber g(tn−2) . Continuing the recursion we obtain g(tn), g(tn−1), . . . , g(t1), g(t0) asan approximation of the optimal boundary g at the points T, T − h, . . . , h, 0 (cf.Figure VI.2).

8. Solution of the variational problem. In the variational problem with fi-nite horizon (see [196, Chap. IV, Sect. 3–4] for the infinite horizon case) it isassumed that we observe a trajectory of the Wiener process (Brownian motion)X = (Xt)0≤t≤T with a drift changing from 0 to µ = 0 at some random time θtaking the value 0 with probability π and being exponentially distributed withparameter λ > 0 given that θ > 0 . (A more natural hypothesis may be that θis uniformly distributed on [0, T ] .)

1. Adopting the setting and notation of paragraph 1 above, let M(α, π, T )denote the class of stopping times τ of X satisfying 0 ≤ τ ≤ T and

Pπ(τ < θ) ≤ α (22.2.76)


where 0 ≤ α ≤ 1 and 0 ≤ π ≤ 1 are given and fixed. The variational problemseeks to determine a stopping time τ in the class M(α, π, T ) such that

Eπ[τ − θ]+ ≤ Eπ[τ − θ]+ (22.2.77)

for any other stopping time τ from M(α, π, T ) . The stopping time τ is thensaid to be optimal in the variational problem (22.2.76)–(22.2.77).

2. To solve the variational problem (22.2.76)–(22.2.77) we will follow thetrain of thought from [196, Chap. IV, Sect. 3] which is based on exploiting thesolution of the Bayesian problem found in Theorem 22.2 above. For this, let usfirst note that if α ≥ 1 − π , then letting τ ≡ 0 we see that Pπ(τ < θ) = Pπ(0 <θ) = 1 − π ≤ α and clearly Eπ[τ − θ]+ = Eπ[−θ]+ = 0 ≤ E [τ − θ]+ for everyτ ∈ M(α, π, T ) showing that τ ≡ 0 is optimal in (22.2.76)–(22.2.77). Similarly,if α = e−λT (1 − π) , then letting τ ≡ T we see that Pπ(τ < θ) = Pπ(T < θ) =e−λT (1 − π) = α and clearly Eπ[τ − θ]+ = Eπ[T − θ]+ ≤ E [τ − θ]+ for everyτ ∈ M(α, π, T ) showing that τ ≡ T is optimal in (22.2.76)–(22.2.77). The sameargument also shows that M(α, π) is empty if α < e−λT (1 − π) . We may thusconclude that the set of admissible α which lead to a nontrivial optimal stoppingtime τ in (22.2.76)–(22.2.77) equals (e−λT (1−π), 1−π) where π ∈ [0, 1) .

3. To describe the key technical points in the argument below leading tothe solution of (22.2.76)–(22.2.77), let us consider the optimal stopping problem(22.2.1) with c > 0 given and fixed. In this context set V (t, π) = V (t, π; c) andg(t) = g(t; c) to indicate the dependence on c and recall that τ∗ = τ∗(c) givenin (22.2.37) is an optimal stopping time in (22.2.1). We then have:

g(t; c) ≤ g(t; c′) for all t ∈ [0, T ] if c > c′, (22.2.78)

g(t; c) ↑ 1 if c ↓ 0 for each t ∈ [0, T ], (22.2.79)

g(t; c) ↓ 0 if c ↑ ∞ for each t ∈ [0, T ]. (22.2.80)

To verify (22.2.78) let us assume that g(t; c) > g(t; c′) for some t ∈ [0, T )and c > c′ . Then for any π ∈ (g(t; c′), g(t; c)) given and fixed we have V (t, π; c) <1 − π = V (t, π; c′) contradicting the obvious fact that V (t, π; c) ≥ V (t, π; c′) asit is clearly seen from (22.2.1). The relations (22.2.79) and (22.2.80) are verifiedin a similar manner.

4. Finally, to exhibit the optimal stopping time τ in (22.2.76)–(22.2.77)when α ∈ (e−λT (1−π), 1−π) and π ∈ [0, 1) are given and fixed, let us introducethe function

u(c; π) = Pπ(τ∗ < θ) (22.2.81)

for c > 0 where τ∗ = τ∗(c) from (22.2.37) is an optimal stopping time in (22.0.5).Using that Pπ(τ∗ < θ) = Eπ[1−πτ∗ ] and (22.2.78) above it is readily verified thatc → u(c; π) is continuous and strictly increasing on (0,∞) . [Note that a strictincrease follows from the fact that g(T ; c) = λ/(λ + c) . ] From (22.2.79) and


(22.2.80) we moreover see that u(0+; π) = e−λT (1−π) due to τ∗(0+) ≡ T andu(+∞; π) = 1 − π due to τ∗(+∞) ≡ 0 . This implies that the equation

u(c; π) = α (22.2.82)

has a unique root c = c(α) in (0,∞) .

5 . The preceding conclusions can now be used to formulate the main resultof this paragraph.

Theorem 22.5. In the variational formulation of the Wiener disorder problem(22.2.76)–(22.2.77) there exists a nontrivial optimal stopping time τ if and onlyif

α ∈ (e−λT (1−π), 1−π) (22.2.83)where π ∈ [0, 1) . In this case τ may be explicitly identified with τ∗ = τ∗(c)in (22.2.37) where g(t) = g(t; c) is the unique solution of the integral equation(22.2.38) and c = c(α) is a unique root of the equation (22.2.82) on (0,∞) .

Proof. It remains to show that τ = τ∗(c) with c = c(α) and α ∈ (e−λT (1−π),1−π) for π ∈ [0, 1) satisfies (22.2.77). For this note that since Pπ(τ < θ) = αby construction, it follows by the optimality of τ∗(c) in (22.0.4) that

α + cEπ[τ − θ]+ ≤ Pπ(τ < θ) + cEπ[τ − θ]+ (22.2.84)

for any other stopping time τ with values in [0, T ] . Moreover, if τ belongs toM(α, π) , then Pπ(τ < θ) ≤ α and from (22.2.84) we see that Eπ[τ − θ]+ ≤Eπ[τ − θ]+ establishing (22.2.77). The proof is complete. Remark 22.6. Recall from part (iv) of paragraph 2 above that g(t; c) ≤ A∗(c) forall 0 ≤ t ≤ T where 0 < A∗(c) < 1 is uniquely determined from the equation(22.1.13) (or (4.147) in [196, p. 201]). Since A∗(c(α)) = 1 − α by Theorem 10in [196, p. 205] it follows that the optimal stopping boundary t → g(t; c(α)) in(22.2.76)–(22.2.77) satisfies g(t; c(α)) ≤ 1 − α for all 0 ≤ t ≤ T .

9. Appendix. In this appendix we exhibit an explicit expression for the tran-sition density function of the a posteriori probability process (πt)0≤t≤T given in(22.0.14)–(22.0.16) above.

1. Let B = (Bt)t≥0 be a standard Wiener process defined on a probabilityspace (Ω,F , P) . With t > 0 and ν ∈ R given and fixed recall from [224, p. 527]that the random variable A

(ν)t =

∫ t

0 e2(Bs+νs) ds has the conditional distribution:

P(A

(ν)t ∈ dz |Bt + νt = y

)= a(t, y, z) dz (22.2.85)

where the density function a for z > 0 is given by

a(t, y, z) =1

πz2exp

(y2 + π2

2t+ y − 1

2z

(1 + e2y

))(22.2.86)

×∫ ∞

0

exp(−w2

2t− ey

zcosh(w)

)sinh(w) sin

(πw

t

)dw.


This implies that the random vector(2(Bt + νt), A(ν)

t

)has the distribution

P(2(Bt + νt) ∈ dy, A

(ν)t ∈ dz

)= b(t, y, z) dy dz (22.2.87)

where the density function b for z > 0 is given by

b(t, y, z) = a(t,

y

2, z) 1

2√

tϕ

(y − 2νt

2√

t

)(22.2.88)

=1

(2π)3/2z2√

texp

(π2

2t+(ν + 1

2

)y − ν2

2t − 1

2z

(1 + ey

))×∫ ∞

0

exp(−w2

2t− ey/2

zcosh(w)

)sinh(w) sin

(πw

t

)dw

and we set ϕ(x) = (1/√

2π)e−x2/2 for x ∈ R (for related expressions in terms ofHermite functions see [46] and [181]).

Denoting It = αBt + βt and Jt =∫ t

0eαBs+βs ds with α = 0 and β ∈ R

given and fixed, and using that the scaling property of B implies

P

(αBt + βt ≤ y,

∫ t

0

eαBs+βs ds ≤ z

)(22.2.89)

= P

(2(Bt′ + νt′) ≤ y,

∫ t′

0

e2(Bs+νs) ds ≤ α2

4z

)with t′ = α2t/4 and ν = 2β/α2 , it follows by applying (22.2.87) and (22.2.88)that the random vector (It, Jt) has the distribution

P(It ∈ dy, Jt ∈ dz

)= f(t, y, z) dy dz (22.2.90)

where the density function f for z > 0 is given by

f(t, y, z) =α2

4b

(α2

4t, y,

α2

4z

)(22.2.91)

=2√

2π3/2α3

1z2√

texp

2π2

α2t+( β

α2+

12

)y − β2

2α2t − 2

α2z

(1+ey

)×∫ ∞

0

exp(−2w2

α2t− 4ey/2

α2zcosh(w)

)sinh(w) sin

(4πw

α2t

)dw.

2. Letting α = −µ/σ and β = −λ− µ2/(2σ2) it follows from the explicitexpressions (22.0.9)+(22.0.11) and (22.0.3) that

P0(ϕt ∈ dx) = P(e−It

( π

1−π+ λJt

)∈ dx

)= g(π; t, x) dx (22.2.92)


where the density function g for x > 0 is given by

g(π; t, x) =d

dx

∫ ∞

−∞

∫ ∞

0

I(e−y

( π

1−π+ λz

)≤ x

)f(t, y, z) dy dz (22.2.93)

=∫ ∞

−∞f(t, y,

1λ

(xey − π

1−π

))ey

λdy.

Moreover, setting

It−s = α(Bt −Bs) + β(t− s) and Jt−s =∫ t

s

eα(Bu−Bs)+β(u−s) du (22.2.94)

as well as Is = αBs + βs and Js =∫ s

0 eαBu+bβudu with β = −λ + µ2/(2σ2) , itfollows from the explicit expressions (22.0.9)+(22.0.11) and (22.0.3) that

Ps(ϕt ∈ dx) (22.2.95)

= P(e−γse−eIt−s

(e(bβ−β)se−bIs

(π

1 − π +λJs

)+ λeγsJt−s

) ∈ dx)

= h(s; π; t, x) dx

for 0 < s < t where γ = µ2/σ2 . Since stationary independent increments of B

imply that the random vector (It−s, Jt−s) is independent of (Is, Js) and equallydistributed as (It−s, Jt−s) , we see upon recalling (22.2.92)–(22.2.93) that the den-sity function h for x > 0 is given by

h(s; π; t, x) =d

dx

∫ ∞

−∞

∫ ∞

0

∫ ∞

0

I(e−γse−y

(e(bβ−β)sw + λeγsz

) ≤ x)

(22.2.96)

× f(t − s, y, z) g(π; s, w) dy dz dw

=∫ ∞

−∞

∫ ∞

0

f

(t− s, y,

xey − e(bβ−β−γ)sw

λ

)g(π; s, w)

ey

λdy dw

where the density function g for w > 0 equals

g(π; s, w) =d

dx

∫ ∞

−∞

∫ ∞

0

I(e−y

( π

1−π+ λz

)≤ w

)f(s, y, z) dy dz (22.2.97)

=∫ ∞

−∞f(s, y,

1λ

(wey − π

1−π

))ey

λdy

and the density function f for z > 0 is defined as in (22.2.90)–(22.2.91) with βinstead of β .

Finally, by means of the same arguments as in (22.2.92)–(22.2.93) it followsfrom the explicit expressions (22.0.9)+(22.0.11) and (22.0.3) that

Pt(ϕt ∈ dx) = P(e−bIt

( π

1−π+ λJt

)∈ dx

)= g(π; t, x) dx (22.2.98)


where the density function g for x > 0 is given by (22.2.97).

3. Noting by (22.0.1) that

Pπ(ϕt ∈ dx) = πP0(ϕt ∈ dx) + (1−π)∫ t

0

λe−λs Ps(ϕt ∈ dx) ds (22.2.99)

+ (1−π) e−λt Pt(ϕt ∈ dx)

we see by (22.2.92)+(22.2.95)+(22.2.98) that the process (ϕt)0≤t≤T has the mar-ginal distribution

Pπ(ϕt ∈ dx) = q(π; t, x) dx (22.2.100)

where the transition density function q for x > 0 is given by

q(π; t, x) = π g(π; t, x) + (1−π)∫ t

0

λe−λs h(s; π; t, x) ds (22.2.101)

+ (1−π) e−λt g(π; t, x)

with g , h , g from (22.2.93), (22.2.96), (22.2.97) respectively.

Hence by (22.0.14) we easily find that the process (πt)0≤t≤T has the marginaldistribution

Pπ(πt ∈ dx) = p(π; t, x) dx (22.2.102)

where the transition density function p for 0 < x < 1 is given by

p(π; t, x) =1

(1 − x)2q(π; t,

x

1 − x

). (22.2.103)

This completes the Appendix.

Notes. The quickest detection problems considered in this chapter belong tothe class of the “disorder/change point” problems that can be described as follows.

We have two “statistically different” processes X1 = (X1t )t≥0 and X2 =

(X2t )t≥0 that form the observable process X = (Xt)t≥0 as

Xt =

X1

t if t < θ,

X2t−θ if t ≥ θ

(22.2.104)

where θ is either a random variable or an unknown parameter that we want toestimate on basis of the observations of X . There are two formulations of theproblem:

(a) If we have all observations of Xt on an admissible time interval [0, T ]and we try to construct an FX

T -measurable estimate θ = θ(Xs, s ≤ T ) of θ , thenwe speak of a “change-point” problem. It is clear that this “a posteriori” problemis essentially a classical estimation problem of mathematical statistics.


(b) We speak of a “disorder” problem if observations are arriving sequentiallyin time and we want to construct an alarm time τ (i.e. stopping time) that insome sense is “as close as possible” to the disorder time θ (when a “good” processX1 turns into a “bad” process X2 ).

Mathematical formulations of the “disorder” problem first of all depend onassumptions about the disorder time θ . Parametric formulations assume simplythat θ is an unknown (unobservable) parameter taking values in a subset of R+ .Bayesian formulations (which we address in the present monograph) assume thatθ is a random variable with distribution Fθ . In our text we suppose that Fθ isan exponential distribution on [0,∞) and we consider two formulations of the“quickest detection” problem: “infinite horizon” and “finite horizon”. In the firstcase we admit for τ all values from R+ = [0,∞) . In the second case we admitfor τ only values from the time interval [0, T ] (it explains the terminology “finitehorizon”).

We refer to [113] for many theoretical investigations of the “disorder/changepoint” problems as well as for a number of important applications (detection of“breaks” in geological data; quickest detection of the beginning of earthquakes,tsunamis, and general “spontaneously appearing effects”, see also [26]). Applica-tions in financial data analysis (detection of arbitrage) are recently discussed in[198]. For quickest detection problems with exponential penalty for delay see [173]and [12]. See also [200] for the criterion infτ E|τ − θ| .

From the standpoint of applications it is also interesting to consider problemswhere the disorder appears on a finite time interval or a decision should be madebefore a certain finite time. Similar to the problems of testing statistical hypotheseson finite time intervals, the corresponding quickest detection problems in the finitehorizon formulation are more difficult than in the case of infinite horizon (becauseadditional “sufficient” statistics time t should be taken into account for finitehorizon problems). Clearly, among all processes that can be considered in theproblem, the Wiener process and the Poisson process take a central place. Oncethese problems are understood sufficiently well, the study of problems includingother processes may follow a similar line of arguments.

Shiryaev in [188, 1961] (see also [187], [189]–[193], [196, Chap. IV]) derivedan explicit solution of the Bayesian and variational problem for a Wiener processwith infinite horizon by reducing the initial optimal stopping problem to a free-boundary problem for a differential operator (see also [208]). Some particular casesof the Bayesian problem for a Poisson process with infinite horizon were solvedby Gal’chuk and Rozovskii [73] and Davis [35]. A complete solution of the latterproblem was given in [169] by reducing the initial optimal stopping problem to afree-boundary problem for a differential-difference operator (see Subsection 24.1below). The main aim of Subsection 22.2 above (following [72]) is to derive asolution of the Bayesian and variational problem for a Wiener process with finitehorizon.


23. Sequential testing of a Poisson process

In this section we continue our study of sequential testing problems consideredin Section 21 above. Instead of the Wiener process we now deal with the Poissonprocess.


1. Description of the problem. Suppose that at time t = 0 we begin to observea Poisson process X = (Xt)t≥0 with intensity λ > 0 which is either λ0 orλ1 where λ0 < λ1 . Assuming that the true value of λ is not known to us, ourproblem is then to decide as soon as possible and with a minimal error probability(both specified later) if the true value of λ is either λ0 or λ1 .

Depending on the hypotheses about the unknown intensity λ , this problemadmits two formulations. The Bayesian formulation relies upon the hypothesisthat an a priori probability distribution of λ is given to us, and that λ takeseither of the values λ0 and λ1 at time t = 0 according to this distribution. Thevariational formulation (sometimes also called a fixed error probability formula-tion) involves no probabilistic assumptions on the unknown intensity λ .

2. Solution of the Bayesian problem. In the Bayesian formulation of theproblem (see [196, Chap. 4]) it is assumed that at time t = 0 we begin observinga trajectory of the point process X = (Xt)t≥0 with the compensator A = (At)t≥0

(see [128, Chap. 18]) where At = λt and a random intensity λ = λ(ω) takes twovalues λ1 and λ0 with probabilities π and 1−π . (We assume that λ1 > λ0 > 0and π ∈ [0, 1] .)

2.1. For a precise probability-statistical description of the Bayesian sequentialtesting problem it is convenient to assume that all our considerations take placeon a probability-statistical space (Ω,F ; Pπ , π ∈ [0, 1]) where Pπ has the specialstructure

Pπ = πP1 + (1−π)P0 (23.1.1)

for π ∈ [0, 1] . We further assume that the F0 -measurable random variableλ = λ(ω) takes two values λ1 and λ0 with probabilities Pπ(λ = λ1) = π andPπ(λ = λ0) = 1−π . Concerning the observable point process X = (Xt)t≥0 , weassume that Pπ(X ∈ · | λ = λi) = Pi(X ∈ · ) , where Pi(X ∈ · ) coincides withthe distribution of a Poisson process with intensity λi for i = 0, 1 .

Probabilities π and 1−π play a role of a priori probabilities of the statis-tical hypotheses

H1 : λ = λ1, (23.1.2)H0 : λ = λ0. (23.1.3)

2.2. Based upon information which is continuously updated through obser-vation of the point process X , our problem is to test sequentially the hypotheses

Section 23. Sequential testing of a Poisson process 335

H1 and H0 . For this it is assumed that we have at our disposal a class of sequen-tial decision rules (τ, d) consisting of stopping times τ = τ(ω) with respect to(FX

t )t≥0 where FXt = σXs : s ≤ t , and FX

τ -measurable functions d = d(ω)which take values 0 and 1 . Stopping the observation of X at time τ , the termi-nal decision function d indicates that either the hypothesis H1 or the hypothesisH0 should be accepted; if d = 1 we accept H1 , and if d = 0 we accept that H0

is true.

2.3. Each decision rule (τ, d) implies losses of two kinds: the loss due to acost of the observation, and the loss due to a wrong terminal decision. The averageloss of the first kind may be naturally identified with cEπ(τ) , and the average lossof the second kind can be expressed as aPπ(d = 0, λ = λ1) + bPπ(d = 1, λ = λ0) ,where c,a, b>0 are some constants. It will be clear from (23.1.8) below that thereis no restriction to assume that c = 1 , as the case of general c > 0 follows byreplacing a and b with a/c and b/c respectively. Thus, the total average lossof the decision rule (τ, d) is given by

Lπ(τ, d) = Eπ

(τ + a1(d=0,λ=λ1) + b1(d=1,λ=λ0)

). (23.1.4)

Our problem is then to compute

V (π) = inf(τ,d)

Lπ(τ, d) (23.1.5)

and to find the optimal decision rule (τ∗, d∗) , called the π-Bayes decision rule,at which the infimum in (23.1.5) is attained.

Observe that for any decision rule (τ, d) we have

aPπ(d = 0, λ = λ1) + bPπ(d = 1, λ = λ0) = aπα(d) + b(1−π)β(d) (23.1.6)

where α(d) = P1(d = 0) is called the probability of an error of the first kind, andβ(d) = P0(d = 1) is called the probability of an error of the second kind.

2.4. The problem (23.1.5) can be reduced to an optimal stopping problem forthe a posteriori probability process defined by

πt = Pπ

(λ = λ1 | FX

t

)(23.1.7)

with π0 = π under Pπ . Standard arguments (see [196, pp. 166–167]) show that

V (π) = infτ

Eπ

(τ + ga,b(πτ )

)(23.1.8)

where ga,b(π) = aπ ∧ b(1− π) ( recall that x ∧ y = minx, y ), the optimalstopping time τ∗ in (23.1.8) is also optimal in (23.1.5), and the optimal decisionfunction d∗ is obtained by setting

d∗ =

1 if πτ∗ ≥ b/(a+b),0 if πτ∗ < b/(a+b).

(23.1.9)


Our main task in the sequel is therefore reduced to solving the optimal stoppingproblem (23.1.8).

2.5. Another natural process, which is in a one-to-one correspondence withthe process (πt)t≥0 , is the likelihood ratio process; it is defined as the Radon–Nikodym density

ϕt =d(P1 |FX

t )d(P0 |FX

t )(23.1.10)

where Pi |FXt denotes the restriction of Pi to FX

t for i = 0, 1 . Since

πt = πd(P1 |FX

t )d(Pπ |FX

t )(23.1.11)

where Pπ |FXt = π P1 |FX

t + (1−π)P0 |FXt , it follows that

πt =(

π

1−πϕt

)/(1 +

π

1−πϕt

)(23.1.12)

as well as thatϕt =

1−π

π

πt

1−πt. (23.1.13)

Moreover, the following explicit expression is known to be valid (see e.g. [51] or[128, Theorem 19.7]):

ϕt = exp(

Xt logλ1

λ0− (λ1 −λ0)t

). (23.1.14)

This representation may now be used to reveal the Markovian structure inthe problem. Since the process X = (Xt)t≥0 is a time-homogeneous Markovprocess having stationary independent increments (Levy process) under both P0

and P1 , from the representation (23.1.14), and due to the one-to-one corre-spondence (23.1.12), we see that (ϕt)t≥0 and (πt)t≥0 are time-homogeneousMarkov processes under both P0 and P1 with respect to natural filtrationswhich clearly coincide with (FX

t )t≥0 . Using then further that Eπ(H | FXt ) =

E1(H | FXt )πt + E0(H | FX

t ) (1−πt) for any (bounded) measurable H , it followsthat (πt)t≥0 , and thus (ϕt)t≥0 as well, is a time-homogeneous Markov processunder each Pπ for π ∈ [0, 1] . (Observe, however, that although the same argu-ment shows that X is a Markov process under each Pπ for π ∈ (0, 1) , it is nota time-homogeneous Markov process unless π equals 0 or 1 .) Note also directlyfrom (23.1.7) that (πt)t≥0 is a martingale under each Pπ for π ∈ [0, 1] . Thus, theoptimal stopping problem (23.1.8) falls into the class of optimal stopping prob-lems for Markov processes (cf. Chapter I), and we therefore proceed by finding theinfinitesimal operator of (πt)t≥0 . A slight modification of the arguments aboveshows that all these processes possess a strong Markov property actually.


2.6. By Ito’s formula (page 67) one can verify (cf. [106, Ch. I, § 4]) that pro-cesses (ϕt)t≥0 and (πt)t≥0 solve the following stochastic equations respectively:

dϕt =(

λ1

λ0− 1

)ϕt− d

(Xt −λ0 t), (23.1.15)

dπt =(λ1 −λ0) πt−(1−πt−)λ1 πt− + λ0 (1−πt−)

(dXt −

(λ1 πt− + λ0 (1−πt−)

)dt)

(23.1.16)

(cf. formula (19.86) in [128]). The equation (23.1.16) may now be used to determinethe infinitesimal operator of the Markov process (πt,FX

t , Pπ)t≥0 for π ∈ [0, 1] .For this, let f = f(π) from C1[0, 1] be given. Then by Ito’s formula (page 67) wefind

f(πt) = f(π0) (23.1.17)

+∫ t

0

f ′(πs−) dπs +∑

0<s≤t

(f(πs) − f(πs−) − f ′(πs−) ∆πs

)= f(π0) +

∫ t

0

f ′(πs−)(− (λ1 −λ0) πs−(1−πs−)

)ds

+∑

0<s≤t

(f(πs) − f(πs−)

)= f(π0) +

∫ t

0

f ′(πs−)(− (λ1 − λ0) πs− (1 − πs−)

)ds

+∫ t

0

∫ 1

0

(f(πs− + y) − f(πs−)

)µπ(ds, dy)

= f(π0) +∫ t

0

f ′(πs−)(− (λ1 − λ0) πs− (1 − πs−)

)ds

+∫ t

0

∫ 1

0

(f(πs−+ y) − f(πs−)

)νπ(ds, dy)

+∫ t

0

∫ 1

0

(f(πs−+ y) − f(πs−)

) (µπ(ds, dy) − νπ(ds, dy)

)= f(π0) +

∫ t

0

(Lf)(πs−) ds + Mt

where µπ is the random measure of jumps of the process (πt)t≥0 and νπ is acompensator of µπ (see e.g. [129, Chap. 3] or [106, Chap. II]), the operator L isgiven as in (23.1.19) below, and M = (Mt)t≥0 defined as

Mt =∫ t

0

∫ 1

0

(f(πs−+y) − f(πs−)

) (µπ(ds, dy) − νπ(ds, dy)

)(23.1.18)


is a local martingale with respect to (FXt )t≥0 and Pπ for every π ∈ [0, 1] . It

follows from (23.1.17) that the infinitesimal operator of (πt)t≥0 acts on f ∈C1[0, 1] like

(Lf)(π) = −(λ1 −λ0)π(1 − π)f ′(π) (23.1.19)

+(λ1π + λ0(1−π)

)(f

(λ1π

λ1π + λ0 (1−π)

)− f(π)

).

2.7. Looking back at (23.1.5) and using explicit expressions (23.1.4) and(23.1.6) with (23.1.1), it is easily verified (cf. [123, p. 105]) that the value functionπ → V (π) is concave on [0, 1] , and thus it is continuous on (0, 1) . Evidently,this function is pointwise dominated by π → ga,b(π) . From these facts and fromthe general theory of optimal stopping for Markov processes (cf. Chapter I) wemay guess that the value function π → V (π) from (23.1.8) should solve the fol-lowing free-boundary problem (for a differential-difference equation defined by theinfinitesimal operator):

(LV )(π) = −1, A∗ < π < B∗, (23.1.20)V (π) = aπ ∧ b(1−π), π /∈ (A∗, B∗), (23.1.21)V (A∗+) = V (A∗), V (B∗−) = V (B∗) (continuous fit), (23.1.22)V ′(A∗) = a (smooth fit) (23.1.23)

for some 0 < A∗< b/(a+b) < B∗< 1 to be found. Observe that (23.1.21) containstwo conditions relevant for the system: (i) V (A∗) = aA∗ and (ii) V (π) = b(1−π)for π ∈ [B∗, S(B∗)] with S = S(π) from (23.1.24) below. These conditions arein accordance with the fact that if the process (πt)t≥0 starts or ends up at someπ outside (A∗, B∗) , we must stop it instantly.

Note from (23.1.16) that the process (πt)t≥0 moves continuously towards0 and only jumps towards 1 at times of jumps of the point process X . Thisprovides some intuitive support for the principle of smooth fit to hold at A∗ (cf.Subsection 9.1). However, without a concavity argument it is not a priori clearwhy the condition V (B∗−) = V (B∗) should hold at B∗ . As Figure VI.3 shows,this is a rare property shared only by exceptional pairs (A, B) (cf. Subsection9.2), and one could think that once A∗ is fixed through the “smooth fit”, theunknown B∗ will be determined uniquely through the “continuous fit”. Whilethis train of thoughts sounds perfectly logical, we shall see quite opposite belowthat the equation (23.1.19) dictates our travel to solution from B∗ to A∗ .

Our next aim is to show that the three conditions in (23.1.22) and (23.1.23)are sufficient to determine a unique solution of the free-boundary problem whichin turn leads to the solution of the optimal stopping problem (23.1.8).

2.8. Solution of the free-boundary problem (23.1.20)–(23.1.23). Consider theequation (23.1.20) on (0, B] with some B >b/(a+b) given and fixed. Introduce


0.3 0.7

3

4

0.3 0.7

3

4

0.3 0.7

1

0.3 0.7

1

0.3 0.7

1

0.3 0.7

1

(1) (2)

(3) (4)

(5) (6)

Figure VI.3: In view of the problem (23.1.8) and its decompositionvia (23.1.4) and (23.1.6) with (23.1.1), we consider τ = inf t ≥ 0 : πt ∈(A,B) for (πt)t≥0 from (23.1.7)+(23.1.12)+(23.1.14) with π ∈ (A, B)given and fixed, so that π0 = π under P0 and P1 ; the computer drawingsabove show the following functions respectively: (1) π → P1(πτ = A) ;(2) π → P0(πτ ≥ B) ; (3) π → E1τ ; (4) π → E0τ ; (5) π → πE1τ+ (1− π)E0τ + aπP1(πτ = A) + b(1− π) P0(πτ ≥ B) = Eπ(τ +ga,b(πτ )) ;(6) π → Eπ(τ +ga,b(πτ )) and π → ga,b(π) , where A = 0.3 , B = 0.7 ,λ0 = 1 , λ1 = e and a = b = 8 . Functions (1)–(4) are found by solvingsystems analogous to the system (23.1.64)–(23.1.66); their discontinuityat B should be noted, as well as the discontinuity of their first derivativeat B1 = 0.46 . . . from (23.1.25); observe that the function (5) is a super-position of functions (1)–(4), and thus the same discontinuities carry overto the function (5), unless something special occurs. The crucial fact tobe observed is that if the function (5) is to be the value function (23.1.8),and thus extended by the gain function π → ga,b(π) outside (A,B) ,then such an extension would generally be discontinuous at B and havea discontinuous first derivative at A ; this is depicted in the final picture(6). It is a matter of fact that the optimal A∗ and B∗ are to be chosenin such a way that both of these discontinuities disappear; these are theprinciples of continuous and smooth fit respectively. Observe that in thiscase the discontinuity of the first derivative of (5) also disappears at B1 ,and the extension obtained is C1 everywhere but at B∗ where it is onlyC0 generally (see Figure VI.5 below).


the “step” function

S(π) =λ1 π

λ1 π + λ0 (1−π)(23.1.24)

for π ≤ B . Observe that π → S(π) is increasing, and find points · · · < B2 <B1 < B0 := B such that S(Bn) = Bn−1 for n ≥ 1 . It is easily verified that

Bn =(λ0)nB

(λ0)nB + (λ1)n(1−B)(n = 0, 1, 2, . . . ). (23.1.25)

Denote In = (Bn, Bn−1] for n ≥ 1 , and introduce the “distance” function

d(π, B) = 1 +

[log(

B

1−B

1−π

π

)/log(

λ1

λ0

)](23.1.26)

for π ≤ B , where [x] denotes the integer part of x . Observe that d is definedto satisfy

π ∈ In ⇐⇒ d(π, B) = n (23.1.27)

for all 0 < π ≤ B .

Consider the equation (23.1.20) on I1 upon setting V (π) = b(1− π) forπ ∈ (B, S(B)] ; this is then a first-order linear differential equation which can besolved explicitly, and imposing a continuity condition at B which is in agreementwith (23.1.22), we obtain a unique solution π → V (π; B) on I1 ; move thenfurther and consider the equation (23.1.20) on I2 upon using the solution foundon I1 ; this is then a first-order linear differential equation which can be solvedexplicitly, and imposing a continuity condition over I2 ∪ I1 at B1 , we obtain aunique solution π → V (π; B) on I2 ; continuing this process by induction, wefind the following formula:

V (π; B) =(1−π)γ1

πγ0

n−1∑k=0

(Cn−k

βk

k!logk

((λ1

λ0

)k−1π

1−π

))(23.1.28)

−(

nλ1 −λ0

λ0λ1+ b

)π +

(n

λ0+ b

)for π ∈ In , where C1, . . . , Cn are constants satisfying the following recurrentrelation:

Cp+1 =p−1∑k=0

(Cp−k

(f

(p)k − f

(p)k+1

))+

(Bp)γ0

(1−Bp)γ1

(λ1 −λ0

λ0λ1Bp − 1

λ0

)(23.1.29)

for p = 0, 1, . . . , n − 1 , with

f(p)k =

βk

k!logk

((λ1

λ0

)k−p−1B

1−B

)(23.1.30)


and where we set

γ0 =λ0

λ1 −λ0, γ1 =

λ1

λ1 −λ0, β =

1(λ1 −λ0)

(λ0)γ1

(λ1)γ0. (23.1.31)

Making use of the distance function (23.1.26), we may now write down theunique solution of (23.1.20) on (0, B] satisfying (23.1.21) on [B, S(B)] and thesecond part of (23.1.22) at B :

V (π; B) =(1−π)γ1

πγ0

d(π,B)−1∑k=0

(Cd(π,B)−k

βk

k!logk

((λ1

λ0

)k−1π

1−π

))(23.1.32)

−(

d(π, B)λ1 −λ0

λ0λ1+ b

)π +

(d(π, B)

λ0+ b

)for 0 < π ≤ B . It is clear from our construction above that π → V (π; B) is C1

on (0, B) and C0 at B .

Observe that when computing the first derivative of π → V (π; B) , we cantreat d(π, B) in (23.1.32) as not depending on π . This then gives the followingexplicit expression:

V ′(π; B) =(1−π)γ1−1

πγ0+1(23.1.33)

×d(π,B)−1∑

k=0

(Cd(π,B)−k

βk

k!logk

((λ1

λ0

)k−1π

1−π

)

×(

k/

log((

λ1

λ0

)k−1π

1−π

)− (π + γ0)

))−(

d(π, B)λ1 −λ0

λ0λ1+ b

)for 0 < π ≤ B .

Setting C = b/(a + b) elementary calculations show that π → V (π; B) isconcave on (0, B) , as well as that V (π; B) → −∞ as π ↓ 0 , for all B ∈ [C, 1] .Moreover, it is easily seen from (23.1.28) (with n = 1 ) that V (π; 1) < 0 for all0 < π < 1 . Thus, if for some B > C , close to C , it happens that π → V (π; B)crosses π → aπ when π moves to the left from B , then a uniqueness argumentpresented in Remark 23.2 below (for different B ’s the curves π → V (π; B) donot intersect) shows that there exists B∗ ∈ (C, 1) , obtained by moving B fromB to 1 or vice versa, such that for some A∗ ∈ (0, C) we have V (A∗; B∗) = aA∗and V ′(A∗; B∗) = a (see Figure VI.4). Observe that the first identity capturespart (i) of (23.1.22), while the second settles (23.1.23).

These considerations show that the system (23.1.20)–(23.1.23) has a unique(nontrivial) solution consisting of A∗ , B∗ and π → V (π; B∗) , if and only if

limB↓C

V ′(B−; B) < a. (23.1.34)


A* B*

π

1

(0,0)

π ga,b(π)

1

Figure VI.4: A computer drawing of “continuous fit” solutions π →V (π; B) of (23.1.20), satisfying (23.1.21) on [B, S(B)] and the secondpart of (23.1.22) at B , for different B in (b/(a+b), 1) ; in this particularcase we took B = 0.95, 0.80, 0.75, . . . , 0.55 , with λ0 = 1 , λ1 = 5 anda = b = 2 . The unique B∗ is obtained through the requirement that themap π → V (π;B∗) hits “smoothly” the gain function π → ga,b(π) atA∗ ; as shown above, this happens for A∗ = 0.22 . . . and B∗ = 0.70 . . . ;such obtained A∗ and B∗ are a unique solution of the system (23.1.38)–(23.1.39). The solution π → V (π;B∗) leads to the explicit form of thevalue function (23.1.8) as shown in Figure VI.5 below.

Geometrically this is the case when for B > C , close to C , the solution π →V (π; B) intersects π → aπ at some π 1a

+1b. (23.1.35)

In this process one should observe that B1 from (23.1.25) tends to a numberstrictly less than C when B ↓ C , so that all calculations are actually performedon I1 .

Thus, the condition (23.1.35) is necessary and sufficient for the existence ofa unique nontrivial solution of the system (23.1.20)–(23.1.23); in this case the


optimal A∗ and B∗ are uniquely determined as the solution of the system oftranscendental equations V (A∗; B∗) = aA∗ and V ′(A∗; B∗) = a , where π →V (π; B) and π → V ′(π; B) are given by (23.1.32) and (23.1.33) respectively;once A∗ and B∗ are fixed, the solution π → V (π; B∗) is given for π ∈ [A∗, B∗]by means of (23.1.32).

2.9. Solution of the optimal stopping problem (23.1.8). We shall now showthat the solution of the free-boundary problem (23.1.20)–(23.1.23) found abovecoincides with the solution of the optimal stopping problem (23.1.8). This in turnleads to the solution of the Bayesian problem (23.1.5).

Theorem 23.1. (I): Suppose that the condition (23.1.35) holds. Then the π-Bayesdecision rule (τ∗, d∗) in the problem (23.1.5) of testing two simple hypotheses H1

and H0 is explicitly given by (see Remark 23.3 below):

τ∗ = inft≥0 : πt /∈ (A∗, B∗)

, (23.1.36)

d∗ =

1 (accept H1) if πτ∗ ≥B∗,0 (accept H0) if πτ∗ = A∗

(23.1.37)

where the constants A∗ and B∗ satisfying 0 < A∗ < b/(a+b) < B∗ < 1 areuniquely determined as solutions of the system of transcendental equations:

V (A∗; B∗) = aA∗, (23.1.38)V ′(A∗; B∗) = a (23.1.39)

with π → V (π; B) and π → V ′(π; B) in (23.1.32) and (23.1.33) respectively.

(II): In the case when the condition (23.1.35) fails to hold, the π-Bayes de-cision rule is trivial : Accept H1 if π > b/(a+b) , and accept H0 if π < b/(a+b);either decision is equally good if π = b/(a+b) .

Proof. (I): 1. We showed above that the free-boundary problem (23.1.20)–(23.1.23) is solvable if and only if (23.1.35) holds, and in this case the solutionπ → V∗(π) is given explicitly by π → V (π; B∗) in (23.1.32) for A∗ ≤ π ≤ B∗ ,where A∗ and B∗ are a unique solution of (23.1.38)–(23.1.39).

In accordance with the interpretation of the free-boundary problem, we ex-tend π → V∗(π) to the whole of [0, 1] by setting V∗(π) = aπ for 0 ≤ π < A∗and V∗(π) = b(1−π) for B∗ < π ≤ 1 (see Figure VI.5). Note that π → V∗(π)is C1 on [0, 1] everywhere but at B∗ where it is C0 . To complete the proof itis enough to show that such defined map π → V∗(π) equals the value functiondefined in (23.1.8), and that τ∗ defined in (23.1.36) is an optimal stopping time.

2. Since π → V∗(π) is not C1 only at one point at which it is C0 , recallingalso that π → V∗(π) is concave, we can apply Ito’s formula (page 67) to V∗(πt) .In exactly the same way as in (23.1.17) this gives

V∗(πt) = V∗(π) +∫ t

0

(LV∗)(πs−) ds + Mt (23.1.40)


A*B*

π(0,0)

π ga,b(π)

1

1

π V(π)

Figure VI.5: A computer drawing of the value function (23.1.8) in thecase λ0 = 1 , λ1 = 5 and a = b = 2 as indicated in Figure VI.4above. The interval (A∗, B∗) is the set of continued observation of theprocess (πt)t≥0 , while its complement in [0, 1] is the stopping set. Thus,as indicated in (23.1.36), the observation should be stopped as soon asthe process (πt)t≥0 enters [0, 1] \ (A∗, B∗) , and this stopping time isoptimal in the problem (23.1.8). The optimal decision function is thengiven by (23.1.37).

where M = (Mt)t≥0 is a martingale given by

Mt =∫ t

0

(V∗(πs−+ ∆πs

)− V∗(πs−))

dXs (23.1.41)

and Xt = Xt −∫ t

0 Eπ(λ | FXs−) ds = Xt −

∫ t

0 (λ1πs− + λ0(1− πs−)) ds is the so-called innovation process (see e.g. [128, Theorem 18.3]) which is a martingale withrespect to (FX

t )t≥0 and Pπ whenever π ∈ [0, 1] . Note in (23.1.40) that we mayextend V ′

∗ arbitrarily to B∗ as the time spent by the process (πt)t≥0 at B∗ isof Lebesgue measure zero.

3. Recall that (LV∗)(π) = −1 for π ∈ (A∗, B∗) , and note that due to thesmooth fit (23.1.23) we also have (LV∗)(π) ≥ −1 for all π ∈ [0, 1] \(A∗, B∗] .


To verify this claim first note that (LV∗)(π) = 0 for π ∈ (0, S−1(A∗)) ∪(B∗, 1) , since Lf ≡ 0 if f(π) = aπ or f(π) = b(1− π) . Observe also that(LV∗)(S−1(A∗)) = 0 and (LV∗)(A∗) = −1 both due to the smooth fit (23.1.23).Thus, it is enough to verify that (LV∗)(π) ≥ −1 for π ∈ (S−1(A∗), A∗) .

For this, consider the equation LV = −1 on (S−1(A∗), A∗] upon imposingV (π) = V (π; B∗) for π ∈ (A∗, S(A∗)] , and solve it under the initial conditionV (A∗) = V (A∗; B∗) + c where c ≥ 0 . This generates a unique solution π →Vc(π) on (S−1(A∗), A∗] , and from (23.1.28) we read that Vc(π) = V (π; B∗) +Kc(1−π)γ1/πγ0 for π ∈ (S−1(A∗), A∗] where Kc = c(A∗)γ0/(1−A∗)γ1 . (Observethat the curves π → Vc(π) do not intersect on (S−1(A∗), A∗] for different c ’s.)Hence we see that there exists c0 > 0 large enough such that for each c > c0

the curve π → Vc(π) lies strictly above the curve π → aπ on (S−1(A∗), A∗] ,and for each c < c0 the two curves intersect. For c ∈ [0, c0) let πc denote the(closest) point (to A∗ ) at which π → Vc(π) intersects π → aπ on (S−1(A∗), A∗] .Then π0 = A∗ and πc decreases (continuously) in the direction of S−1(A∗)when c increases from 0 to c0 . Observe that the points πc are “good” pointssince by Vc(πc) = aπc = V∗(πc) with V ′

c (πc) > a = V ′∗(πc) and Vc(S(πc)) =

V (S(πc); B∗) = V∗(S(πc)) we see from (23.1.19) that (LV∗)(πc) ≥ (LVc)(πc) =−1 . Thus, if we show that πc reaches S−1(A∗) when c ↑ c0 , then the proof ofthe claim will be complete. Therefore assume on the contrary that this is not thecase. Then Vc1(S−1(A∗)−) = aS−1(A∗) for some c1 < c0 , and Vc(S−1(A∗)−) >aS−1(A∗) for all c > c1 . Thus by choosing c > c1 close enough to c1 , we see thata point πc > S−1(A∗) arbitrarily close to S−1(A∗) is obtained at which Vc(πc) =aπc = V∗(πc) with V ′

c (πc) < a = V ′∗(πc) and Vc(S(πc)) = V (S(πc); B∗) =V∗(S(πc)) , from where we again see by (23.1.19) that (LV∗)(πc) ≤ (LVc)(πc) =−1 . This however leads to a contradiction because π → (LV∗)(π) is continuousat S−1(A∗) (due to the smooth fit) and (LV∗)(S−1(A∗)) = 0 as already statedearlier. Thus, we have (LV∗)(π) ≥ −1 for all π ∈ [0, 1] (upon setting V ′

∗(B∗) :=0for instance).

4. Recall further that V∗(π) ≤ ga,b(π) for all π ∈ [0, 1] . Moreover, sinceπ → V∗(π) is bounded, and (Xt −λi t )t≥0 is a martingale under Pi for i = 0, 1 ,it is easily seen from (23.1.41) with (23.1.17) upon using the optional samplingtheorem (page 60), that EπMτ = 0 whenever τ is a stopping time of X suchthat Eπτ < ∞ . Thus, taking the expectation on both sides in (23.1.40), we obtain

V∗(π) ≤ Eπ

(τ + ga,b(πτ )

)(23.1.42)

for all such stopping times, and hence V∗(π) ≤ V (π) for all π ∈ [0, 1] .

5. On the other hand, the stopping time τ∗ from (23.1.36) clearly satisfiesV∗(πτ∗) = ga,b(πτ∗) . Moreover, a direct analysis of τ∗ based on (23.1.12)–(23.1.14)(see Remark 23.3 below), together with the fact that for any Poisson processN = (Nt)t≥0 the exit time of the process (Nt − µt)t≥0 from [A, B] has a finiteexpectation for any real µ , shows that Eπτ∗ < ∞ for all π ∈ [0, 1] . Taking then


the expectation on both sides in (23.1.40), we get

V∗(π) = Eπ

(τ∗ + ga,b(πτ∗)

)(23.1.43)

for all π ∈ [0, 1] . This fact and the consequence of (23.1.42) stated above showthat V∗ = V , and that τ∗ is an optimal stopping time. The proof of the first partis complete.

(II): Although, in principle, it is clear from our construction above that thesecond part of the theorem holds as well, we shall present a formal argument forcompleteness.

Suppose that the π-Bayes decision rule is not trivial. In other words, thismeans that V (π)<ga,b(π) for some π ∈ (0, 1) . Since π → V (π) is concave, thisimplies that there are 0 < A∗ < b/(a+b) < B∗ < 1 such that τ∗ = inf t >0 : πt /∈ (A∗, B∗) is optimal for the problems (23.1.8) and (23.1.5) respectively,with d∗ from (23.1.9) in the latter case. Thus V (π) = Eπ(τ∗ + ga,b(πτ∗)) forπ ∈ [0, 1] , and therefore by the general Markov processes theory, and due to thestrong Markov property of (πt)t≥0 , we know that π → V (π) solves (23.1.20)and satisfies (23.1.21) and (23.1.22); a priori we do not know if the smooth fitcondition (23.1.23) is satisfied. Nevertheless, these arguments show the existenceof a solution to (23.1.20) on (0, B∗] which is b(1−B∗) at B∗ and which crossesπ → aπ at (some) A∗ <b/(a+b) . But then the same uniqueness argument usedin paragraph 2.8 above (see Remark 23.2 below) shows that there must existpoints A∗≤A∗ and B∗≥B∗ such that the solution π → V (π; B∗) of (23.1.20)satisfying V (B∗; B∗) = b(1− B∗) hits π → aπ smoothly at A∗ . The first part ofthe proof above then shows that the stopping time τ∗ = inf t>0 : πt /∈ (A∗, B∗) is optimal. As this stopping time is known to be Pπ -a.s. pointwise the smallestpossible optimal stopping time (cf. Chapter I or see the proof of Theorem 23.4below), this shows that τ∗ cannot be optimal unless the smooth fit conditionholds at A∗ , that is, unless A∗ = A∗ and B∗ = B∗ . In any case, however, thisargument implies the existence of a nontrivial solution to the system (23.1.20)–(23.1.23), and since this fact is equivalent to (23.1.35) as shown above, we see thatcondition (23.1.35) cannot be violated.

Observe that we have actually proved that if the optimal stopping prob-lem (23.1.8) has a nontrivial solution, then the principle of smooth fit holds atA∗ . An alternative proof of the statement could be done by using Lemma 3 in[196, p. 118]. The proof of the theorem is complete.

Remark 23.2. The following probabilistic argument can be given to show that thetwo curves π → V (π, B′) and π → V (π, B′′) from (23.1.32) do not intersect on(0, B′] whenever 0<B′<B′′≤1 .

Assume that the two curves do intersect at some Z < B′ . Let π → απ + βdenote the tangent of the map V ( · ; B′) at Z . Define a map π → g(π) bysetting g(π) = (απ + β) ∧ b(1− π) for π ∈ [0, 1] , and consider the optimal


stopping problem (23.1.8) with g instead of ga,b . Let V = V (π) denote thevalue function. Consider also the map π → V∗(π) defined by V∗(π) = V (π; B′)for π ∈ [Z, B′] and V∗(π) = g(π) for π ∈ [0, 1] \ [Z, B′] . As π → V∗(π) is C0 atB′ and C1 at Z , then in exactly the same way as in paragraphs 3– 5 of theproof above we find that V∗(π) = V (π) for all π ∈ [0, 1] . However, if we considerthe stopping time σ∗ = inf t>0 : πt /∈ (Z, B′′) , then it follows in the same wayas in paragraph 5 of the proof above that V (π; B′′) = Eπ(σ∗ + g(πσ∗)) for allπ ∈ [Z, B′′] . As V (π; B′′) < V∗(π) for π ∈ (Z, B′] , this is a contradiction. Thus,the curves do not intersect.

Remark 23.3. 1. Observe that the optimal decision rule (23.1.36)–(23.1.37) can beequivalently rewritten as follows:

τ∗ = inft≥0 : Zt /∈ (A∗, B∗)

, (23.1.44)

d∗ =

1 (accept H1) if Zτ∗ ≥B∗,0 (accept H0) if Zτ∗ = A∗

(23.1.45)

where we use the following notation:

Zt = Xt − µt, (23.1.46)

A∗ = log(

A∗1−A∗

1−π

π

)/log(

λ1

λ0

), (23.1.47)

B∗ = log(

B∗1−B∗

1−π

π

)/log(

λ1

λ0

), (23.1.48)

µ =(λ1 −λ0

)/log(

λ1

λ0

). (23.1.49)

2. The representation (23.1.44)–(23.1.45) reveals the structure and applicabil-ity of the optimal decision rule in a clearer manner. The result proved above showsthat the following sequential procedure is optimal: While observing Xt , monitorZt , and stop the observation as soon as Zt enters either (−∞, A∗] or [B∗,∞) ;in the first case conclude λ = λ0 , in the second conclude λ = λ1 . In this processthe condition (23.1.35) must be satisfied, and the constants A∗ and B∗ shouldbe determined as a unique solution of the system (23.1.38)–(23.1.39). This systemcan be successfully treated by means of standard numerical methods if one mimicsour travel from B∗ to A∗ in the construction of our solution in paragraph 2.8above. A pleasant fact is that only a few steps by (23.1.24) will be often neededto recapture A∗ if one starts from B∗ .

3. Note that the same problem of testing two statistical hypotheses for aPoisson process was treated by different methods in [179]. One may note that thenecessary and sufficient condition (23.1.35) of Theorem 23.1 is different from thecondition aλ1 + b(λ0+λ1) < b/a found in [179].


3. Solution of the variational problem. In the variational formulation of theproblem it is assumed that the sequentially observed process X = (Xt)t≥0 is aPoisson process with intensity λ0 or λ1 , and no probabilistic assumption is madeabout the outcome of λ0 and λ1 at time 0 . To formulate the problem we shalladopt the setting and notation from the previous part. Thus Pi is a probabilitymeasure on (Ω,F) under which X = (Xt)t≥0 is a Poisson process with intensityλi for i = 0, 1 .

3.1. Given the numbers α, β > 0 such that α + β < 1 , let ∆(α, β) denotethe class of all decision rules (τ, d) satisfying

α(d) ≤ α and β(d) ≤ β (23.1.50)

where α(d) = P1(d = 0) and β(d) = P0(d = 1) . The variational problem is thento find a decision rule (τ , d ) in the class ∆(α, β) such that

E0τ ≤ E0τ and E1τ ≤ E1τ (23.1.51)

for any other decision rule (τ, d) from the class ∆(α, β) . Note that the mainvirtue of the requirement (23.1.51) is its simultaneous validity for both P0 andP1 .

Our main aim below is to show how the solution of the variational problemtogether with a precise description of all admissible pairs (α, β) can be obtainedfrom the Bayesian solution (Theorem 23.1). The sequential procedure which leadsto the optimal decision rule (τ , d) in this process is a SPRT (sequential probabilityratio test). We now describe a procedure of passing from the Bayesian solution tothe variational solution.

3.2. It is useful to note that the explicit procedure of passing from theBayesian solution to the variational solution presented in the next three stepsis not confined to a Poissonian case but is also valid in greater generality includingthe Wiener case (for details in the case of discrete time see [123]).

Step 1 (Construction): Given α, β > 0 with α + β < 1 , find constants Aand B satisfying A<0<B such that the stopping time

τ = inft≥0 : Zt /∈ (A, B)

(23.1.52)

satisfies the following identities:

P1

(Zτ = A

)= α, (23.1.53)

P0

(Zτ ≥ B

)= β (23.1.54)

where (Zt)t≥0 is as in (23.1.46). Associate with τ the following decision function:

d =

1 (accept H1) if Zτ ≥ B,

0 (accept H0) if Zτ = A.(23.1.55)


We will actually see below that not for all values α and β do such A and Bexist; a function G : (0, 1) → (0, 1) is displayed in (23.1.73) such that the solution(A, B) to (23.1.53)–(23.1.54) exists only for β ∈ (0, G(α)) if α ∈ (0, 1) . Suchvalues α and β will be called admissible.

Step 2 (Embedding): Once A and B are found for admissible α and β ,we may respectively identify them with A∗ and B∗ from (23.1.47) and (23.1.48).Then, for any π ∈ (0, 1) given and fixed, we can uniquely determine A∗ and B∗satisfying 0 < A∗ < B∗ < 1 such that (23.1.47) and (23.1.48) hold with π = π .Once A∗ and B∗ are given, we can choose a > 0 and b > 0 in the Bayesianproblem (23.1.4)–(23.1.5) such that the optimal stopping time in (23.1.8) is exactlythe exit time τ∗ of (πt)t≥0 from (A∗, B∗) as given in (23.1.36). Observe that thisis possible to achieve since the optimal A∗ and B∗ range through all (0, 1) whena and b satisfying (23.1.35) range through (0,∞) . (For this, let any B∗ ∈ (0, 1)be given and fixed, and choose a > 0 and b > 0 such that B∗ = b/(a + b)with λ1 − λ0 = 1/a + 1/b . Then consider the solution V ( · ; B∗) := Vb( · ; B∗)of (23.1.20) on (0, B∗) upon imposing Vb(π; B∗) = b(1−π) for π ∈ [B∗, S(B∗)]where b ≥ b . To each such a solution there corresponds a > 0 such that π → aπhits π → Vb(π; B∗) smoothly at some A = A(b) . When b increases from bto ∞ , then A(b) decreases from B∗ to zero. This is easily verified by a simplecomparison argument upon noting that π → Vb(π; B∗) stays strictly above π →V (π; B∗) + Vb(B∗; B∗) on (0, B∗) (recall the idea used in Remark 23.3 above).As each A(b) obtained (in the pair with B∗ ) is optimal (recall the argumentsused in paragraphs 3– 5 of the proof of Theorem 23.1), the proof of the claimis complete.)

Step 3 (Verification): Consider the process (πt)t≥0 defined by (23.1.12)+(23.1.14) with π = π , and denote by (τ∗, d∗) the optimal decision rule(23.1.36)–(23.1.37) associated with it. From our construction above note that τfrom (23.1.52) actually coincides with τ∗ , as well as that πτ∗ = A∗ = Zbτ = Aand πτ∗ ≥B∗ = Zτ ≥B . Thus (23.1.53) and (23.1.54) show that

P1

(d∗= 0

)= α, (23.1.56)

P0

(d∗= 1

)= β (23.1.57)

for the admissible α and β . If now any decision rule (τ, d) from ∆(α, β) is given,then either P1(d = 0) = α and P0(d = 1) = β , or at least one strict inequalityholds. In both cases, however, from (23.1.4)–(23.1.6) and (23.1.56)–(23.1.57) weeasily see that Eπ τ∗ ≤ Eπτ , since otherwise τ∗ would not be optimal. Sinceτ∗ = τ , it follows that Eπ τ ≤ Eπτ , and letting π first go to 0 and then to 1 , weobtain (23.1.51) in the case when E0τ < ∞ and E1τ < ∞ . If either E0τ or E1τequals ∞ , then (23.1.51) follows by the same argument after a simple truncation(e.g. if E0τ < ∞ but E1τ = ∞ , choose n ≥ 1 such that P0(τ >n) ≤ ε , apply thesame argument to τn := τ ∧n and dn := d1τ≤n+ 1τ>n , and let ε go to zero


in the end.) This solves the variational problem posed above for all admissible αand β .

3.3. The preceding arguments also show:

If either P1(d = 0) < α or P0(d = 1) < β for some (τ, d)∈ ∆(α, β) with admissible α and β , then at least onestrict inequality in (23.1.51) holds.

(23.1.58)

Moreover, since τ∗ is known to be Pπ -a.s. the smallest possible optimal stoppingtime (cf. Chapter I or see the proof of Theorem 23.4 below), from the argumentsabove we also get

If P1(d = 0) = α and P0(d = 1) = β for some (τ, d) ∈∆(α, β) with admissible α and β , and both equalities in(23.1.51) hold, then τ = τ P0 -a.s. and P1 -a.s.

(23.1.59)

The property (23.1.59) characterizes τ as a unique stopping time of the decisionrule with maximal admissible error probabilities having both P0 and P1 expecta-tion at minimum.

3.4. It remains to determine admissible α and β in (23.1.53) and (23.1.54)above. For this, consider τ defined in (23.1.52) for some A < 0 < B , and notefrom (23.1.14) that ϕt = exp

(Zt log(λ1/λ0)

). By means of (23.1.10) we find

P1

(Zτ = A

)= P1

(ϕτ = exp

A log

(λ1

λ0

))(23.1.60)

= exp

A log(

λ1

λ0

)P0

(Zτ = A

)= exp

A log

(λ1

λ0

)(1 − P0

(Zτ ≥B

)).

Using (23.1.53)–(23.1.54), from (23.1.60) we see that

A = log(

α

1−β

)/log(

λ1

λ0

). (23.1.61)

To determine B , let Pz0 be a probability measure under which X = (Xt)t≥0

is a Poisson process with intensity λ0 and Z = (Zt)t≥0 starts at z . It is easilyseen that the infinitesimal operator of Z under (Pz

0 )z∈R acts like

(L0f)(z) = −µf ′(z) + λ0

(f(z+1)− f(z)

). (23.1.62)

In view of (23.1.54), introduce the function

u(z) = Pz0

(Zτ ≥B

). (23.1.63)


Strong Markov arguments then show that z → u(z) solves the following system:

(L0u)(z) = 0 if z∈(A, B)\B−1, (23.1.64)u(A) = 0, (23.1.65)u(z) = 1 if z≥B. (23.1.66)

The solution of this system is given in (4.15) of [51]. To display it, introducethe function

F (x; B) =δ(x,B)∑k=0

(−1)k

k!

((B−x− k

)ρ e−ρ

)k

(23.1.67)

for x≤B , where we denote

δ(x, B) = −[x−B+1], (23.1.68)

ρ = log(

λ1

λ0

)/(λ1

λ0− 1

). (23.1.69)

Setting Jn = [B−n−1, B−n) for n ≥ 0 , observe that δ(x, B) = n if and onlyif x ∈ Jn .

It is then easily verified that the solution of the system (23.1.64)–(23.1.66) isgiven by

u(z) = 1 − e−ρ(z−A) F (z; B)F (A; B)

(23.1.70)

for A ≤ z < B . Note that z → u(z) is C1 everywhere in (A, B) but at B−1where it is only C0 ; note also that u(A+) = u(A) = 0 , but u(B−) < u(B) = 1(see Figure VI.6).

Going back to (23.1.54), and using (23.1.70), we see that

P0

(Zτ ≥B

)= 1 − eρA F (0; B)

F (A; B). (23.1.71)

Letting B ↓ 0 in (23.1.71), and using the fact that the expression (23.1.71) iscontinuous in B and decreases to 0 as B ↑ ∞ , we clearly obtain a necessary andsufficient condition on β to satisfy (23.1.54), once A = A(α, β) is fixed through(23.1.61); as F (0; 0) = 1 , this condition reads

β < 1 − eρA(α,β)

F(A(α, β); 0

) . (23.1.72)

Note, however, if β increases, then the function on the right-hand side in (23.1.72)decreases, and thus there exists a unique β∗ = β∗(α) > 0 at which equality in


-1 1 2

1

z P0 (Zτ ≥ B)z

z

Figure VI.6: A computer drawing of the map u(z) = Pz0(Zτ ≥ B) from

(23.1.63) in the case A = −1 , B = 2 and λ0 = 0.5 . This map is a uniquesolution of the system (23.1.64)–(23.1.66). Its discontinuity at B shouldbe noted, as well as the discontinuity of its first derivative at B − 1 .Observe also that u(A+) = u(A) = 0 . The case of general A , B andλ0 looks very much the same.

(23.1.72) is attained. (This value can easily be computed by means of standardnumerical methods.) Setting

G(α) = 1 − eρA(α,β∗(α))

F(A(α, β∗(α)); 0

) (23.1.73)

we see that admissible α and β are characterized by 0 < β < G(α) (see FigureVI.7). In this case A is given by (23.1.61), and B is uniquely determined fromthe equation

F (0; B) − (1−β) F (A; B) e−ρA = 0. (23.1.74)

The set of all admissible α and β will be denoted by A . Thus, we have

A =(α, β) : 0 < α < 1, 0 < β < G(α)

. (23.1.75)

3.5. The preceding considerations may be summarised as follows (see alsoRemark 23.5 below).

Theorem 23.4. In the problem (23.1.50)–(23.1.51) of testing two simple hypotheses(23.1.2)–(23.1.3) based upon sequential observations of the Poisson process X =(Xt)t≥0 under P0 or P1, there exists a unique decision rule (τ , d ) ∈ ∆(α, β)satisfying (23.1.51) for any other decision rule (τ, d)∈∆(α, β) whenever (α, β) ∈A . The decision rule (τ , d ) is explicitly given by (23.1.52)+(23.1.55) with A


α G(α)

α + β = 1

1

1α

β

(0,0)

Figure VI.7: A computer drawing of the map α → G(α) from (23.1.73)in the case λ0 = 1 and λ1 = 3 . The area A which lies below the graphof G determines the set of all admissible α and β . The case of generalλ0 and λ1 looks very much the same; it can also be shown that G(0+)decreases if the difference λ1 −λ0 increases, as well as that G(0+) in-creases if both λ0 and λ1 increase so that the difference λ1 −λ0 remainsconstant; in all cases G(1−) = 0 . It may seem somewhat surprising thatG(0+) < 1 ; observe, however, this is in agreement with the fact that(Zt)t≥0 from (23.1.46) is a supermartingale under P0 . (A little peak onthe graph, at α = 0.19 . . . and β = 0.42 . . . in this particular case, cor-responds to the disturbance when A from (23.1.61) passes through −1while B = 0+ ; it is caused by a discontinuity of the first derivative of themap from (23.1.71) at B − 1 (see Figure VI.6).)

in (23.1.61) and B from (23.1.74), it satisfies (23.1.58), and is characterized by(23.1.59).

Proof. It only remains to prove (23.1.59). For this, in the notation used above,assume that τ is a stopping time of X satisfying the hypotheses of (23.1.59).Then clearly τ is an optimal stopping time in (23.1.8) for π = π with a and bas in Step 2 above.


Recall that V∗(π) ≤ ga,b(π) for all π , and observe that τ can be written as

τ = inft≥0 : V∗(πt) ≥ ga,b(πt)

(23.1.76)

where π → V∗(π) is the value function (23.1.8) appearing in the proof of Theorem23.1. Supposing now that Pπ(τ < τ ) > 0 , we easily find by (23.1.76) that

Eπ

(τ + ga,b(πτ )

)> Eπ

(τ + V∗(πτ )

). (23.1.77)

On the other hand, it is clear from (23.1.40) with LV∗ ≥ −1 that (t+V∗(πt) )t≥0

is a submartingale. Thus by the optional sampling theorem (page 60) it followsthat

Eπ

(τ + V∗(πτ )

) ≥ V∗(π). (23.1.78)

However, from (23.1.77) and (23.1.78) we see that τ cannot be optimal, and thuswe must have Pπ(τ ≥ τ ) = 1 . Moreover, since it follows from our assumption thatEπτ = Eπτ , this implies that τ = τ Pπ -a.s. Finally, as Pi Pπ for i = 0, 1 , wesee that τ = τ both P0 -a.s. and P1 -a.s. The proof of the theorem is complete.

Observe that the sequential procedure of the optimal decision rule (τ , d )from Theorem 23.4 is precisely the SPRT. The explicit formulae for E0τ and E1τare given in (4.22) of [51].

Remark 23.5. If (α, β) /∈ A , that is, if β ≥ G(α) for some α, β > 0 such thatα+β <1 , then no decision rule given by the SPRT-form (23.1.52)+(23.1.55) cansolve the variational problem (23.1.50)–(23.1.51).

To see this, let such (α, β∗) /∈ A be given, and let (τ, d) be a decision rulesatisfying (23.1.52)+(23.1.55) for some A< 0 < B . Denote β = P0(Zτ ≥B) andchoose α to satisfy (23.1.61). Then β < G(α) ≤ β∗ by definition of the mapG . Given β′ ∈ (β, G(α)) , let B′ be taken to satisfy (23.1.54) with β′ , and letα′ be determined from (23.1.61) with β′ so that A remains unchanged. Clearly0 < B′ < B and 0 < α′ < α , and (23.1.53) holds with A and α′ respectively.But then (τ ′, d′) satisfying (23.1.52)+(23.1.55) with A < 0 < B′ still belongs to∆(α, β∗) , while clearly τ ′ < τ both under P0 and P1 . This shows that (τ, d)does not solve the variational problem.

The preceding argument shows that the admissible class A from (23.1.75)is exactly the class of all error probabilities (α, β) for which the SPRT is opti-mal. A pleasant fact is that A always contains a neighborhood around (0, 0) in[0, 1]×[0, 1] , which is the most interesting case from the standpoint of statisticalapplications.

Notes. The main aim of this section (following [168]) was to present an explicitsolution of the problem of testing two statistical hypotheses about the intensityof an observed Poisson process in the context of a Bayesian formulation, and thenapply this result to deduce the optimality of the method (SPRT) in the context of a

Section 24. Quickest detection of a Poisson process 355

variational formulation, providing a precise description of the set of all admissibleprobabilities of a wrong decision (“errors of the first and second kind”).

Despite the fact that the Bayesian approach to sequential analysis of problemson testing two statistical hypotheses has gained a considerable interest in the lastfifty or so years (see e.g. [216], [217], [18], [123], [31], [196], [203]), it turns out that avery few problems of that type have been solved explicitly (by obtaining a solutionin closed form). In this respect the case of testing two simple hypotheses about themean value of a Wiener process with drift is exceptional, as the explicit solutionto the problem has been obtained in both Bayesian and variational formulation(cf. Section 21). These solutions (including the proof of the optimality of theSPRT) were found by reducing the initial problem to a free-boundary problem(for a second-order differential operator) which could be solved explicitly. It isclear from the material above that the Poisson free-boundary problem is moredelicate, since in this case one needs to deal with a differential-difference operator,the appearance of which is a consequence of the discontinuous character of theobserved (Poisson) process.

The variational problem formulation (23.1.51) is due to Wald [216]. In thepapers [218] and [219] Wald and Wolfowitz proved the optimality of the SPRT inthe case of i.i.d. observations and under special assumptions on the admissibilityof (α, β) (see [218], [219], [5], [123] for more details and compare it with the ad-missability notion given above). In the paper [51] Dvoretzky, Kiefer and Wolfowitzconsidered the problem of optimality of the SPRT in the case of continuous timeand satisfied themselves with the remark that “a careful examination of the resultsof [218] and [219] shows that their conclusions in no way require that the processesbe discrete in time” omitting any further detail and concentrating their attentionon the problem of finding the error probabilities α(d) and β(d) with expectationsE0τ and E1τ for the given SPRT (τ, d) defined by “stopping boundaries” A andB in the case of a Wiener or Poisson process.

The SPRT is known to be optimal in the variational formulation for a largeclass of observable processes (see [51], [96], [17]). For the general problem of theminimax optimality of the SPRT (in the sense (23.1.51)) in the case of continuoustime see [96].

24. Quickest detection of a Poisson process

In this section we continue our study of quickest detection problems consideredin Section 22 above. Instead of the Wiener process we now deal with the Poissonprocess.


1. Description of the problem. The Poisson disorder problem is less formally statedas follows. Suppose that at time t = 0 we begin observing a trajectory of the


Poisson process X = (Xt)t≥0 whose intensity changes from λ0 to λ1 at somerandom (unknown) time θ which is assumed to take value 0 with probabilityπ , and is exponentially distributed with parameter λ given that θ > 0 . Basedupon the information which is continuously updated through our observation ofthe trajectory of X , our problem is to terminate the observation (and declarethe alarm) at a time τ∗ which is as close as possible to θ (measured by a costfunction with parameter c > 0 specified below).

1.1. The problem can be formally stated as follows. Let Nλ0 = (Nλ0t )t≥0 ,

Nλ1 = (Nλ1t )t≥0 and L = (Lt)t≥0 be three independent stochastic processes

defined on a probability-statistical space (Ω,F ; Pπ , π ∈ [0, 1]) such that:

Nλ0 is a Poisson process with intensity λ0 >0; (24.1.1)

Nλ1 is a Poisson process with intensity λ1 >0; (24.1.2)L is a continuous Markov chain with two states λ0 and λ1,initial distribution [1− π; π], and transition-probability matrix[e−λt, 1− e−λt; 0, 1] for t > 0 where λ > 0.

(24.1.3)

Thus Pπ(L0 = λ1) = 1 − Pπ(L0 = λ0) = π , and given that L0 = λ0 , thereis a single passage of L from λ0 to λ1 at a random time θ > 0 satisfyingPπ(θ > t) = e−λt for all t > 0 .

The process X = (Xt)t≥0 observed is given by

Xt =∫ t

0

I(Ls−= λ0) dNλ0s +

∫ t

0

I(Ls−= λ1) dNλ1s (24.1.4)

and we set FXt = σXs : 0 ≤ s ≤ t for t ≥ 0 . Denoting θ = inf t ≥ 0 : Lt =

λ1 we see that Pπ(θ = 0) = π and Pπ(θ > t | θ > 0) = e−λt for all t > 0 . Itis assumed that the time θ of “disorder” is unknown (i.e. it cannot be observeddirectly).

The Poisson disorder problem (or the quickest detection problem for the Pois-son process) seeks to find a stopping time τ∗ of X that is “as close as possible”to θ as a solution of the following optimal stopping problem:

V (π) = infτ

(Pπ(τ <θ) + cEπ(τ − θ)+

)(24.1.5)

where Pπ(τ <θ) is interpreted as the probability of a “false alarm”, Eπ(τ − θ)+

is interpreted as the “average delay” in detecting the occurrence of “disorder”correctly, c > 0 is a given constant, and the infimum in (24.1.5) is taken overall stopping times τ of X (compare this with the “Wiener disorder problem” inSection 22 above). A stopping time of X means a stopping time with respect tothe natural filtration (FX

t )t≥0 generated by X . The same terminology will beused for other processes in the sequel as well.


1.2. Introducing the a posteriori probability process

πt = Pπ(θ ≤ t | FXt ) (24.1.6)

for t ≥ 0 , it is easily seen that Pπ(τ < θ) = Eπ(1 − πτ ) and Eπ(τ − θ)+ =Eπ

( ∫ τ

0πt dt

)for all stopping times τ of X , so that (24.1.5) can be rewritten as

follows:

V (π) = infτ

Eπ

((1−πτ ) + c

∫ τ

0

πt dt

)(24.1.7)

where the infimum is taken over all stopping times τ of (πt)t≥0 (as shown fol-lowing (24.1.12) below).

Define the likelihood ratio process

ϕt =πt

1 − πt. (24.1.8)

Similarly to the case of a Wiener process (see (22.0.9)) we find that

ϕt = eλtZt

(ϕ0 + λ

∫ t

0

e−λs

Zsds

)(24.1.9)

where the likelihood process

Zt =dP0

dP∞ (t, X) =d(P0 |FX

t )d(P∞ |FX

t )= exp

(log(

λ1

λ0

)Xt − (λ1 −λ0)t

)(24.1.10)

and the measures P0 and P∞ (as well as Ps ) are defined analogously to theWiener process case (thus Ps is the probability law (measure) of the process Xgiven that θ = s for s ∈ [0,∞] ). From (24.1.9)–(24.1.10) by Ito’s formula (page67) one finds that the processes (ϕt)t≥0 and (πt)t≥0 solve the following stochasticequations respectively:

dϕt = λ(1+ϕt) dt +(

λ1

λ0− 1

)ϕt− d

(Xt −λ0t), (24.1.11)

dπt = λ(1−πt) dt +(λ1 −λ0)πt−(1−πt−)λ1 πt− + λ0 (1−πt−)

(24.1.12)

×(dXt −

(λ1 πt− + λ0 (1−πt−)

)dt).

It follows that (ϕt)t≥0 and (πt)t≥0 are time-homogeneous (strong) Markov pro-cesses under Pπ with respect to the natural filtrations which clearly coincide with(FX

t )t≥0 respectively. Thus, the infimum in (24.1.7) may indeed be viewed astaken over all stopping times τ of (πt)t≥0 , and the optimal stopping problem(24.1.7) falls into the class of optimal stopping problems for Markov processes (cf.Chapter I). We thus proceed by finding the infinitesimal operator of the Markovprocess (πt)t≥0 .


1.3. Noting that

Pπ = π P0 + (1−π)∫ ∞

0

λe−λsPs ds (24.1.13)

it follows that the so-called innovation process (Xt)t≥0 defined by

Xt = Xt −∫ t

0

Eπ(Ls | FXs−) ds = Xt −

∫ t

0

(λ1πs− + λ0(1−πs−)

)ds (24.1.14)

is a martingale under Pπ with respect to (FXt )t≥0 for π ∈ [0, 1] . Moreover, from

(24.1.12) and (24.1.14) we get

dπt = λ(1−πt) dt +(λ1 −λ0)πt−(1−πt−)λ1 πt− + λ0 (1−πt−)

dXt. (24.1.15)

This implies that the infinitesimal operator of (πt)t≥0 acts on f ∈ C1[0, 1] ac-cording to the rule

(Lf)(π) =(λ − (λ1 −λ0)π

)(1−π) f ′(π) (24.1.16)

+(λ1π + λ0 (1−π)

)(f

(λ1 π

λ1 π + λ0 (1−π)

)− f(π)

).

Note that for λ = 0 the equations (24.1.11)–(24.1.12) and (24.1.16) reduce to(23.1.15)–(23.1.16) and (23.1.19) respectively.

1.4. Using (24.1.13) it is easily verified that the following facts are valid:

The map π → V (π) is concave (continuous) and decreasingon [0, 1];

(24.1.17)

The stopping time τ∗ = inf t ≥ 0 : πt ≥ B∗ is optimal inthe problem (24.1.5)+(24.1.7), where B∗ is the smallest π

from [0, 1] satisfying V (π) = 1 − π.

(24.1.18)

Thus V (π) < 1 − π for all π ∈ [0, B∗) and V (π) = 1 − π for all π ∈ [B∗, 1] . Itshould be noted in (24.1.18) that πt = ϕt/(1+ϕt) , and hence by (24.1.9)–(24.1.10)we see that πt is a (path-dependent) functional of the process X observed upto time t . Thus, by observing a trajectory of X it is possible to decide when tostop in accordance with the rule τ∗ given in (24.1.18).

The question arises, however, to determine the optimal threshold B∗ interms of the four parameters λ0, λ1, λ, c as well as to compute the value V (π)for π ∈ [0, B∗) (especially for π = 0 ). We tackle these questions by forming afree-boundary problem.

2. The free-boundary problem. Being aided by the general optimal stoppingtheory of Markov processes (cf. Chapter I), and making use of the preceding facts,


we are naturally led to formulate the following free-boundary problem for π →V (π) and B∗ defined above:

(LV )(π) = −cπ (0 < π < B∗), (24.1.19)V (π) = 1 − π (B∗ ≤ π ≤ 1), (24.1.20)V (B∗−) = 1−B∗ (continuous fit). (24.1.21)

In some cases (specified below) the following condition will be satisfied aswell:

V ′(B∗) = −1 (smooth fit). (24.1.22)

However, we will also see below that this condition may fail.Finally, it is easily verified by passing to the limit for π ↓ 0 that each

continuous solution of the system (24.1.19)–(24.1.20) must necessarily satisfy

V ′(0+) = 0 (normal entrance) (24.1.23)

whenever V (0+) is finite. This condition proves useful in the case when λ1 < λ0 .

2.1. Solving the free-boundary problem. It turns out that the case λ1 < λ0

is much different from the case λ1 > λ0 . Thus assume first that λ1 > λ0 andconsider the equation (24.1.19) on (0, B] for some 0 π for all 0 < π < 1 and findpoints · · · < B2 < B1 < B0 := B such that S(Bn) = Bn−1 for n ≥ 1 . It iseasily verified that

Bn =(λ0)nB

(λ0)nB + (λ1)n(1−B)(n = 0, 1, 2, . . .). (24.1.25)

Denote In = (Bn, Bn−1] for n ≥ 1 , and introduce the “distance” function

d(π, B) = 1 +

[log(

B

1−B

1−π

π

)/log(

λ1

λ0

)](24.1.26)

for π ≤ B (cf. (23.1.26)), where [x] denotes the integer part of x . Observe thatd is defined to satisfy

π ∈ In ⇐⇒ d(π, B) = n (24.1.27)

for all 0 < π ≤ B .

Now consider the equation (24.1.19) first on I1 upon setting V (π) = 1 − πfor π ∈ (B, S(B)] . This is then a first-order linear differential equation which can


be solved explicitly. Imposing a continuity condition at B (which is in agreementwith (24.1.21) above) we obtain a unique solution π → V (π; B) on I1 . It ispossible to verify that the following formula holds:

V (π; B) = c1(B)Vg(π) + Vp,1(π; B) (π ∈ I1) (24.1.28)

where π → Vp,1(π; B) is a (bounded) particular solution of the nonhomogeneousequation in (24.1.19):

Vp,1(π; B) = − λ0(λ1 − c)λ1(λ0+λ)

π +λ0λ1+λc

λ1(λ0+λ)(24.1.29)

and π → Vg(π) is a general solution of the homogeneous equation in (24.1.19):

Vg(π) =

⎧⎪⎪⎨⎪⎪⎩(1−π)γ1

|λ− (λ1 −λ0)π |γ0, if λ = λ1 −λ0,

(1−π) exp( λ1

(λ1 −λ0)(1−π)

), if λ = λ1 −λ0,

(24.1.30)

where γ1 = λ1/(λ1 −λ0 −λ) and γ0 = (λ0+λ)/(λ1 −λ0 −λ) , and the constantc1(B) is determined by the continuity condition V (B−; B) = 1 − B leading to

c1(B) = − 1Vg(B)

(λ1λ+λ0c

λ1(λ0+λ)B − λ(λ1 − c)

λ1(λ0+λ)

)(24.1.31)

where Vg(B) is obtained by replacing π in (24.1.30) by B . [We observe from(24.1.29)–(24.1.31) however that the continuity condition at B cannot be metwhen B equals B from (24.1.34) below unless B equals λ(λ1 − c)/(λλ1+cλ0)from (24.1.41) below (the latter is equivalent to c = λ1 − λ0 − λ ). Thus, ifB = B = λ(λ1 − c)/(λλ1 +cλ0) then there is no solution π → V (π; B) on I1

that satisfies V (π; B) = 1−π for π ∈ (B, S(B)] and is continuous at B . It turnsout, however, that this analytic fact has no significant implication for the solutionof (24.1.5)+(24.1.7).]

Next consider the equation (24.1.19) on I2 upon using the solution foundon I1 and setting V (π) = c1(B) Vg(π) + Vp,1(π; B) for π ∈ (B1, S(B1)] . This isthen again a first-order linear differential equation which can be solved explicitly.Imposing a continuity condition over I2 ∪ I1 at B1 (which is in agreement with(24.1.17) above) we obtain a unique solution π → V (π; B) on I2 . It turns out,however, that the general solution of this equation cannot be expressed in termsof elementary functions (unless λ = 0 as shown in Subsection 23.1 above) but oneneeds, for instance, the Gauss hypergeometric function. As these expressions areincreasingly complex to record, we omit the explicit formulae in the sequel.

Continuing the preceding procedure by induction as long as possible (consid-ering the equation (24.1.19) on In upon using the solution found on In−1 and


imposing a continuity condition over In ∪ In−1 at Bn−1 ) we obtain a uniquesolution π → V (π; B) on In given as

V (π; B) = cn(B)Vg(π) + Vp,n(π; B) (π ∈ In) (24.1.32)

where π → Vp,n(π; B) is a (bounded) particular solution, π → Vg(π) is a generalsolution given by (24.1.30), and B → cn(B) is a function of B (and the fourparameters). [We will see however in Theorem 24.1 below that in the case B >

B > 0 with B from (24.1.34) below the solution (24.1.32) exists for π ∈ (B, B]but explodes at B unless B = B∗ . ]

The key difference in the case λ1 < λ0 is that S(π) < π for all 0 < π < 1so that we need to deal with points B := B0 < B1 < B2 < · · · such thatS(Bn) = Bn−1 for n ≥ 1 . Then the facts (24.1.25)–(24.1.27) remain preservedprovided that we set In = [Bn−1, Bn) for n ≥ 1 . In order to prescribe the initialcondition when considering the equation (24.1.19) on I1 , we can take B = ε > 0small and make use of (24.1.23) upon setting V (π) = v for all π ∈ [S(B), B)where v ∈ (0, 1) is a given number satisfying V (B) = v . Proceeding by inductionas earlier (considering the equation (24.1.19) on In upon using the solution foundon In−1 and imposing a continuity condition over In−1∪In at Bn−1 ) we obtaina unique solution π → V (π; ε, v) on In given as

V (π; ε, v) = cn(ε)Vg(π) + Vp,n(π; ε, v) (π ∈ In) (24.1.33)

where π → Vp,n(π; ε, v) is a particular solution, π → Vg(π) is a general solutiongiven by (24.1.30), and ε → cn(ε) is a function of ε (and the four parameters).We shall see in Theorem 24.1 below how these solutions can be used to determinethe optimal π → V (π) and B∗ .

2.2. Two key facts about the solution. Both of these facts hold only in thecase when λ1 > λ0 and they will be used in the proof of Theorem 24.1 statedbelow. The first fact to be observed is that

B =λ

λ1 −λ0(24.1.34)

is a singularity point of the equation (24.1.19) whenever λ < λ1 − λ0 . This isclearly seen from (24.1.30) where Vg(π) → ∞ for π → B . The second fact ofinterest is that

B =λ

λ + c(24.1.35)

is a smooth-fit point of the system (24.1.19)–(24.1.21) whenever λ1 > λ0 andc = λ1 − λ0 − λ , i.e. V ′(B−; B) = −1 in the notation of (24.1.32) above. Thiscan be verified by (24.1.28) using (24.1.29)–(24.1.31). It means that B is theunique point which in addition to (24.1.19)–(24.1.21) has the power of satisfyingthe smooth-fit condition (24.1.22).


It may also be noted in the verification above that the equation V ′(B−; B) =−1 has no solution when c = λ1 − λ0 − λ as the only candidate B := B = Bsatisfies

V ′(B−; B) = −λ0

λ1. (24.1.36)

This identity follows readily from (24.1.28)–(24.1.31) upon noticing that c1(B) =0 . Thus, when c runs from +∞ to λ1−λ0−λ , the smooth-fit point B runs from0 to the singularity point B , and once B has reached B for c = λ1 − λ0 − λ ,the smooth-fit condition (24.1.22) breaks down and gets replaced by the condition(24.1.36) above. We will soon attest below that in all these cases the smooth-fitpoint B is actually equal to the optimal-stopping point B∗ from (24.1.18) above.

Observe that the equation (24.1.19) has no singularity points when λ1 < λ0 .This analytic fact reveals a key difference between the two cases.

3. Conclusions. In parallel to the two analytic properties displayed above webegin this part by stating the relevant probabilistic properties of the a posterioriprobability process.

3.1. Sample-path properties of (πt)t≥0 . First consider the case λ1 > λ0 .Then from (24.1.12) we see that (πt)t≥0 can only jump towards 1 (at times ofthe jumps of the process X ). Moreover, the sign of the drift term λ(1−π)−(λ1−λ0)π(1−π) = (λ1 −λ0)(B −π)(1−π) is determined by the sign of B−π . Hencewe see that (πt)t≥0 has a positive drift in [0, B) , a negative drift in (B, 1] , and azero drift at B . Thus, if (πt)t≥0 starts or ends up at B , it is trapped there untilthe first jump of the process X occurs. At that time (πt)t≥0 finally leaves B

by jumping towards 1 . This also shows that once (πt)t≥0 leaves [0, B) it nevercomes back. The sample-path behaviour of (πt)t≥0 when λ1 > λ0 is depicted inFigure VI.8 (Part i).

Next consider the case λ1 < λ0 . Then from (24.1.12) we see that (πt)t≥0

can only jump towards 0 (at times of the jumps of the process X ). Moreover,the sign of the drift term λ(1 − π)− (λ1 − λ0)π(1 − π) = (λ + (λ0 − λ1)π)(1 − π)is always positive. Thus (πt)t≥0 always moves continuously towards 1 and canonly jump towards 0 . The sample-path behaviour of (πt)t≥0 when λ1 < λ0 isdepicted in Figure VI.8 (Part ii).

3.2. Sample-path behaviour and the principles of smooth and continuous fit.With a view to (24.1.18), and taking 0 < B < 1 given and fixed, we shall nowexamine the manner in which the process (πt)t≥0 enters [B, 1] if starting atB−dπ where dπ is infinitesimally small (or equivalently enters (B, 1] if startingat B ). Our previous analysis then shows the following (see Figure VI.8).

If λ1 > λ0 and B λ0 and B > B , then the only way for


λ1 λ0>(i)

0 1

0 1B

B

• • • • • •

• •

0 1

λ1 λ0<(ii)

• • •

Figure VI.8: Sample-path properties of the a posteriori probability pro-cess (πt)t≥0 from (24.1.6)+(24.1.12). The point bB is a singularity point(24.1.34) of the free-boundary equation (24.1.19).

(πt)t≥0 to enter [B, 1] is by jumping over B . (Jumping exactly at B happenswith probability zero.)

The case λ1 > λ0 and B = B is special. If starting outside [B, 1] then(πt)t≥0 travels towards B by either moving continuously or by jumping. However,the closer (πt)t≥0 gets to B the smaller the drift to the right becomes, and ifthere were no jump over B eventually, the process (πt)t≥0 would never reach Bas the drift to the right tends to zero together with the distance of (πt)t≥0 toB . This fact can be formally verified by analysing the explicit representation of(ϕt)t≥0 in (24.1.9)–(24.1.10) and using that πt = ϕt/(1+ϕt) for t ≥ 0 . Thus, inthis case as well, the only way for (πt)t≥0 to enter [B, 1] after starting at B−dπ

is by jumping over to (B, 1] .

We will demonstrate below that the sample-path behaviour of the process(πt)t≥0 during the entrance of [B∗, 1] has a precise analytic counterpart in termsof the free-boundary problem (24.1.19). If the process (πt)t≥0 may enter [B∗, 1]by passing through B∗ continuously, then the smooth-fit condition (24.1.22) holds


at B∗ ; if, however, the process (πt)t≥0 enters [B∗, 1] exclusively by jumpingover B∗ , then the smooth-fit condition (24.1.22) breaks down. In this case thecontinuous-fit condition (24.1.21) still holds at B∗ , and the existence of a singu-larity point B can be used to determine the optimal B∗ as shown below.

Due to the fact that the times of jumps of the process (πt)t≥0 are ‘sufficientlyapart’ it is evident that the preceding two sample-path behaviors can be rephrasedin terms of regularity of the boundary point B∗ as discussed in Section 7 above.

3.3. The preceding considerations may now be summarized as follows.

Theorem 24.1. Consider the Poisson disorder problem (24.1.5) and the equivalentoptimal-stopping problem (24.1.7) where the process (πt)t≥0 from (24.1.6) solves(24.1.12) and λ0, λ1, λ, c > 0 are given and fixed.

Then there exists B∗ ∈ (0, 1) such that the stopping time

τ∗ = inf t ≥ 0 : πt ≥ B∗ (24.1.37)

is optimal in (24.1.5) and (24.1.7). Moreover, the optimal cost function π → V (π)from (24.1.5)+(24.1.7) solves the free-boundary problem (24.1.19)–(24.1.21), andthe optimal threshold B∗ is determined as follows.

(i): If λ1 > λ0 and c > λ1 − λ0 − λ , then the smooth-fit condition (24.1.22)holds at B∗ , and the following explicit formula is valid :

B∗ =λ

λ + c. (24.1.38)

In this case B∗ λ0 and c = λ1 − λ0 − λ , then the smooth-fit condition breaksdown at B∗ and gets replaced by the condition (24.1.36) above ( V ′(B∗−) =−λ0/λ1 ) . The optimal threshold B∗ is still given by (24.1.38), and in this caseB∗ = B (see Figure VI.10).

(iii): If λ1 > λ0 and c < λ1 −λ0 −λ , then the smooth-fit condition does nothold at B∗ , and the optimal threshold B∗ is determined as a unique solution in(B, 1) of the following equation:

cd( bB,B∗)(B∗) = 0 (24.1.39)

where the map B → d(B, B) is defined in (24.1.26), and the map B → cn(B) isdefined by (24.1.31) and (24.1.32) above (see Figure VI.11). In particular, when csatisfies

λ1λ0 (λ1 −λ0 −λ)λ1λ0 + (λ1 −λ0)(λ−λ0)

≤ c < λ1 −λ0 −λ, (24.1.40)


then the following explicit formula is valid :

B∗ =λ (λ1 − c)λλ1+cλ0

(24.1.41)

which in the case c = λ1 − λ0 − λ reduces again to (24.1.38) above.

In the cases (i)–(iii) the optimal cost function π → V (π) from (24.1.5)+(24.1.7) is given by (24.1.32) with B∗ in place of B for all 0 < π ≤ B∗ (withV (0) = V (0+) ) and V (π) = 1 − π for B∗ ≤ π ≤ 1 .

(iv): If λ1 < λ0 then the smooth-fit condition holds at B∗ , and the optimalthreshold B∗ can be determined using the normal entrance condition (24.1.23) asfollows (see Figure VI.12). For ε > 0 small let vε denote a unique number in(0, 1) for which the map π → V (π; ε, vε) from (24.1.33) hits the map π → 1 − πsmoothly at some Bε

∗ from (0, 1) . Then we have

B∗ = limε↓0

Bε∗, (24.1.42)

V (π) = limε↓0

V (π; ε, vε) (24.1.43)

for all 0 < π ≤ B∗ (with V (0) = V (0+) ) and V (π) = 1 − π for B∗ ≤ π ≤ 1 .

Proof. We have already established in (24.1.18) above that τ∗ from (24.1.37) isoptimal in (24.1.5) and (24.1.7) for some B∗ ∈ [0, 1] to be found. It thus follows bythe strong Markov property of the process (πt)t≥0 together with (24.1.17) abovethat the optimal cost function π → V (π) from (24.1.5)+(24.1.7) solves the free-boundary problem (24.1.19)–(24.1.21). Some of these facts will also be reprovedbelow.

First consider the case λ1 > λ0 . In paragraph 2.1 above it was shown thatfor each given and fixed B ∈ (0, B) the problem (24.1.19)–(24.1.21) with B inplace of B∗ has a unique continuous solution given by the formula (24.1.32).Moreover, this solution is (at least) C1 everywhere but possibly at B where itis (at least) C0 . As explained following (24.1.31) above, these facts also hold forB = B when B equals λ(λ1 − c)/(λλ1+cλ0) from (24.1.41) above. We will nowshow how the optimal threshold B∗ is determined among all these candidates Bwhen c ≥ λ1 − λ0 − λ .

(i)+(ii): Since the innovation process (24.1.14) is a martingale under Pπ withrespect to (FX

t )t≥0 , it follows by (24.1.15) that

πt = π + λ

∫ t

0

(1−πs−) ds + Mt (24.1.44)


1

1

B*

π

π → 1- π

π → V(π)

(i) λ1 λ0>

1

1

B*

π

π → 1- π

(ii)

B

λ1 λ0>

π →V(π;B)

π →V(π;B)

Figure VI.9: A computer drawing of the maps π → V (π;B) from (24.1.32)for different B from (0, 1) in the case λ1 = 4 , λ0 = 2 , λ = 1 , c = 2 .

The singularity point bB from (24.1.34) equals 1/2 , and the smooth-

fit point eB from (24.1.35) equals 1/3 . The optimal threshold B∗ co-

incides with the smooth-fit point eB . The value function π → V (π)from (24.1.5)+(24.1.7) equals π → V (π;B∗) for 0 ≤ π ≤ B∗ and1 − π for B∗ ≤ π ≤ 1 . (This is presented in Part (i) above.) The so-lutions π → V (π; B) for B > B∗ are ruled out since they fail to satisfy0 ≤ V (π) ≤ 1 − π for all π ∈ [0, 1] . (This is shown in Part (ii) above.)The general case λ1 > λ0 with c > λ1 − λ0 − λ looks very much thesame.


where M = (Mt)t≥0 is a martingale under Pπ with respect to (FXt )t≥0 . Hence

by the optional sampling theorem (page 60) we easily find

Eπ

((1−πτ

)+ c

∫ τ

0

πt dt

)(24.1.45)

= (1−π) + (λ+c) Eπ

(∫ τ

0

(πt − λ

λ+c

)dt

)for all stopping times τ of (πt)t≥0 . Recalling the sample-path behaviour of(πt)t≥0 in the case λ1 > λ0 as displayed in paragraph 3.1 above (cf. Figure VI.8(Part i)), and the definition of V (π) in (24.1.7) together with the fact that B =λ/(λ+ c) ≤ B when c ≥ λ1−λ0−λ , we clearly see from (24.1.45) that it is neveroptimal to stop (πt)t≥0 in [0, B) , as well as that (πt)t≥0 must be stopped imme-diately after entering [B, 1] as it will never return to the “favourable” set [0, B)again. This proves that B equals the optimal threshold B∗ , i.e. that τ∗ from(24.1.37) with B∗ from (24.1.38) is optimal in (24.1.5) and (24.1.7). The claimabout the breakdown of the smooth-fit condition (24.1.22) when c = λ1 − λ0 − λhas been already established in paragraph 2.2 above (cf. Figure VI.10).

(iii): It was shown in paragraph 2.1 above that for each given and fixedB ∈ (B, 1) the problem (24.1.19)–(24.1.21) with B in place of B∗ has a uniquecontinuous solution on (B, 1] given by the formula (24.1.32). We will now showthat there exists a unique point B∗ ∈ (B, 1) such that limπ↓ bB V (π; B) = ±∞if B ∈ (B, B∗) ∪ (B∗, 1) and limπ↓ bB V (π; B∗) is finite. This point is the optimalthreshold, i.e. the stopping time τ∗ from (24.1.37) is optimal in (24.1.5) and(24.1.7). Moreover, the point B∗ can be characterized as a unique solution of theequation (24.1.39) in (B, 1) .

In order to verify the preceding claims we will first state the following obser-vation which proves useful. Setting g(π) = 1 − π for 0 < π < 1 we have

(Lg)(π) ≥ −cπ ⇐⇒ π ≥ B (24.1.46)

where B is given in (24.1.35). This is verified straightforwardly using (24.1.16).

Now since B is a singularity point of the equation (24.1.19) (recall our dis-cussion in paragraph 2.2 above), and moreover π → V (π) from (24.1.5)+(24.1.7)solves (24.1.19)–(24.1.21), we see that the optimal threshold B∗ from (24.1.18)must satisfy (24.1.39). This is due to the fact that a particular solution π →Vp,n(π; B∗) for n = d(B, B∗) in (24.1.32) above is taken bounded. The key re-maining fact to be established is that there cannot be two (or more) points in(B, 1) satisfying (24.1.39).


1

1

π

π → 1- ππ → V(π)

B*

B

=smooth fitλ1 λ0-- λc >( )

continuous fitλ1 λ0-- λc < )(

breakdown point

λ1 λ0>

Figure VI.10: A computer drawing of the value functions π → V (π)from (24.1.5)+(24.1.7) in the case λ1 = 4 , λ0 = 2 , λ = 1 andc = 1.4, 1.3, 1.2, 1.1, 1, 0.9, 0.8, 0.7, 0.6 . The given V (π) equals V (π; B∗)from (24.1.32) for all 0 < π ≤ B∗ where B∗ as a function of c is givenby (24.1.38) and (24.1.41). The smooth-fit condition (24.1.22) holds in thecases c = 1.4, 1.3, 1.2, 1.1 . The point c = 1 is a breakdown point whenthe optimal threshold B∗ equals the singularity point bB from (24.1.34),and the smooth-fit condition gets replaced by the condition (24.1.36) with

B = B∗ = bB = 0.5 in this case. For c = 0.9, 0.8, 0.7, 0.6 the smooth-fitcondition (24.1.22) does not hold. In these cases the continuous-fit condi-tion (24.1.21) is satisfied. Moreover, numerical computations suggest that

the mapping B∗ → V ′(B∗−; B∗) which equals −1 for 0 < B∗ < bB and

jumps to −λ0/λ1 = −0.5 for B∗ = bB is decreasing on [ bB, 1) and tendsto a value slightly larger than −0.6 when B∗ ↑ 1 that is c ↓ 0 . Thegeneral case λ1 > λ0 looks very much the same.


(ii)

1

1

π

π → 1- π

B

λ1 λ0>

π →V(π;B)

B*

1

1

π

π → 1- π

(i) λ1 λ0>

B

π →V(π;B)

Figure VI.11: A computer drawing of the maps π → V (π; B) from(24.1.32) for different B from (0, 1) in the case λ1 = 4 , λ0 = 2 ,

λ = 1 , c = 2/5 . The singularity point bB from (24.1.34) equals 1/2 .The optimal threshold B∗ can be determined from the fact that all so-lutions π → V (π; B) for B > B∗ hit zero for some π > bB , and all

solutions π → V (π; B) for B < B∗ hit 1− π for some π > bB . (This isshown in Part (i) above.) A simple numerical method based on the preced-ing fact suggests the estimates 0.750 < B∗ < 0.752 . The value functionπ → V (π) from (24.1.5)+(24.1.7) equals π → V (π; B∗) for 0 ≤ π ≤ B∗and 1 − π for B∗ ≤ π ≤ 1 . The solutions π → V (π; B) for B ≤ bB areruled out since they fail to be concave. (This is shown in Part (ii) above.)The general case λ1 > λ0 with c < λ1 − λ0 − λ looks very much thesame.


Assume on the contrary that there are two such points B1 and B2 . We mayhowever assume that both B1 and B2 are larger than B since for B ∈ (B, B)the solution π → V (π; B) is ruled out by the fact that V (π; B) > 1 − π forπ ∈ (B − ε, B) with ε > 0 small. This fact is verified directly using (24.1.28)–(24.1.31). Thus, each map π → V (π; Bi) solves (24.1.19)–(24.1.21) on (0, Bi] andis continuous (bounded) at B for i = 1, 2 . Since S(π) > π for all 0 < π < 1when λ1 > λ0 , it follows easily from (24.1.16) that each solution π → V (π; Bi)of (24.1.19)–(24.1.21) must also satisfy −∞ < V (0+; Bi) < +∞ for i = 1, 2 .

In order to make use of the preceding fact we shall set hβ(π) = (1 +(β − 1)B) − βπ for 0 ≤ π ≤ B and hβ(π) = 1 − π for B ≤ π ≤ 1 . Sinceboth maps π → V (π; Bi) are bounded on (0, B) we can fix β > 0 large enoughso that V (π; Bi) ≤ hβ(π) for all 0 < π ≤ B and i = 1, 2 . Consider then theauxiliary optimal stopping problem

W (π) := infτ

Eπ

(hβ(πτ ) + c

∫ τ

0

πt dt

)(24.1.47)

where the supremum is taken over all stopping times τ of (πt)t≥0 . Extend themap π → V (π; Bi) on [Bi, 1] by setting V (π; Bi) = 1 − π for Bi ≤ π ≤ 1 anddenote the resulting (continuous) map on [0, 1] by π → Vi(π) for i = 1, 2 . Thenπ → Vi(π) satisfies (24.1.19)–(24.1.21), and since Bi ≥ B , we see by means of(24.1.46) that the following condition is also satisfied:

(LVi)(π) ≥ −cπ (24.1.48)

for π ∈ [Bi, 1] and i = 1, 2 . We will now show that the preceding two facts havethe power of implying that Vi(π) = W (π) for all π ∈ [0, 1] with either i ∈ 1, 2given and fixed.

It follows by Ito’s formula (page 67) that

Vi(πt) = Vi(π) +∫ t

0

(LVi)(πs−) ds + Mt (24.1.49)

where M = (Mt)t≥0 is a martingale ( under Pπ ) given by

Mt =∫ t

0

(Vi

(πs−+ ∆πs

)− Vi(πs−))

dXs (24.1.50)

and Xt = Xt −∫ t

0

(λ1πs− + λ0(1− πs−)

)ds is the innovation process. By the

optional sampling theorem (page 60) it follows from (24.1.49) using (24.1.48) andthe fact that Vi(π) ≤ hβ(π) for all π ∈ [0, 1] that Vi(π) ≤ W (π) for all π ∈[0, 1] . Moreover, defining τi = inf t ≥ 0 : πt ≥ Bi it is easily seen e.g. by(24.1.44) that Eπτi < ∞ . Using then that π → Vi(π) is bounded on [0, 1] , itfollows easily by the optional sampling theorem (page 60) that Eπ(Mτi) = 0 . Since


1

1

π → 1- π

λ1 λ0<

π

π → V(π)

π →V(π;ε,v)

π →V(π;ε,vε)

B*

≈εB*

Figure VI.12: A computer drawing of the maps π → V (π; ε, v) from(24.1.33) for different v from (0, 1) with ε = 0.001 in the case λ1 = 2 ,λ0 = 4 , λ = 1 , c = 1 . For each ε > 0 there is a unique numbervε ∈ (0, 1) such that the map π → V (π; ε, vε) hits the map π → 1 − πsmoothly at some Bε

∗ ∈ (0, 1) . Letting ε ↓ 0 we obtain Bε∗ → B∗

and V (π; ε, vε) → V (π) for all π ∈ [0, 1] where B∗ is the optimalthreshold from (24.1.18) and π → V (π) is the value function from(24.1.5)+(24.1.7).

moreover Vi(πτi) = hβ(πτi) and (LVi)(πs−) = −cπs− for all s ≤ τi , we see from(24.1.49) that the inequality Vi(π) ≤ W (π) derived above is actually equality forall π ∈ [0, 1] . This proves that V (π; B1) = V (π; B2) for all π ∈ [0, 1] , or inother words, that there cannot be more than one point B∗ in (B, 1) satisfying(24.1.39). Thus, there is only one solution π → V (π) of (24.1.19)–(24.1.21) whichis finite at B (see Figure VI.11), and the proof of the claim is complete.

(iv): It was shown in paragraph 2.1 above that the map π → V (π; ε, v) from(24.1.33) is a unique continuous solution of the equation (LV )(π) = −cπ forε < π < 1 satisfying V (π) = v for all π ∈ [S(ε), ε] . It can be checked using(24.1.30) that

Vp,1(π; ε, v) =cλ0

λ1(λ0+λ)π +

cλ

λ1(λ0+λ)+ v, (24.1.51)

c1(ε) = − 1Vg(ε)

(cλ0

λ1(λ0+λ)ε +

cλ

λ1(λ0+λ)

)(24.1.52)


for π ∈ I1 = [ε, ε1) where S(ε1) = ε . Moreover, it may be noted directly from(24.1.16) above that L(f+c) = L(f) for every constant c , and thus V (π; ε, v) =V (π; ε, 0) + v for all π ∈ [S(ε), 1) . Consequently, the two maps π → V (π; ε, v′)and π → V (π; ε, v′′) do not intersect in [S(ε), 1) when v′ and v′′ are different.

Each map π → V (π; ε, v) is concave on [S(ε), 1) . This fact can be provedby a probabilistic argument using (24.1.13) upon considering the auxiliary optimalstopping problem (24.1.47) where the map π → hβ(π) is replaced by the concavemap hv(π) = v ∧ (1 − π) . [It is a matter of fact that π → W (π) from (24.1.47)is concave on [0, 1] whenever π → hβ(π) is so.] Moreover, using (24.1.30) and(24.1.51)–(24.1.52) in (24.1.33) with n = 1 it is possible to see that for v closeto 0 we have V (π; ε, v) < 0 for some π > ε , and for v close to 1 we haveV (π; ε, v) > 1 − π for some π > ε (see Figure VI.12). Thus a simple concavityargument implies the existence of a unique point Bε

∗ ∈ (0, 1) at which π →V (π; ε, vε) for some vε ∈ (0, 1) hits π → 1 − π smoothly. The key nontrivialpoint in the verification that V (π; ε, vε) equals the value function W (π) of theoptimal stopping problem (24.1.47) with π → hvε(π) in place of π → hβ(π) is toestablish that (L(V ( · ; ε, vε)))(π) ≥ −cπ for all π ∈ (Bε

∗, S−1(Bε

∗)) . Since Bε∗

is a smooth-fit point, however, this can be done using the same method which weapplied in paragraph 3 of the proof of Theorem 23.1. Moreover, when ε ↓ 0 thenclearly (24.1.42) and (24.1.43) are valid (recall (24.1.17) and (24.1.23) above), andthe proof of the theorem is complete.

Notes. The Poisson disorder problem seeks to determine a stopping timewhich is as close as possible to the (unknown) time of “disorder” when the intensityof an observed Poisson process changes from one value to another. The problem wasfirst studied in [73] where a solution has been found in the case when λ+c ≥ λ1 >λ0 . This result has been extended in [35] to the case when λ + c ≥ λ1 − λ0 > 0 .Many other authors have also studied the problem from a different standpoint (seee.g. [131]). The main purpose of the present section (following [169]) is to describethe structure of the solution in the general case. The method of proof consistsof reducing the initial (optimal stopping) problem to a free-boundary differential-difference problem. The key point in the solution is reached by specifying whenthe principle of smooth fit breaks down and gets superseded by the principle ofcontinuous fit. This can be done in probabilistic terms (by describing the samplepath behaviour of the a posteriori probability process) and in analytic terms (viathe existence of a singularity point of the free-boundary equation).

The Wiener process version of the disorder problem (where the drift changes)appeared earlier (see [188]) and is now well understood (cf. Section 22 above).The method of proof consists of reducing the initial optimal stopping problem toa free-boundary differential problem which can be solved explicitly. The principleof smooth fit plays a key role in this context.


In this section we adopt the same methodology as in the Wiener processcase. A discontinuous character of the observed (Poisson) process in the presentcase, however, forces us to deal with a differential-difference equation forming afree-boundary problem which is more delicate. This in turn leads to a new effectof the breakdown of the smooth fit principle (and its replacement by the principleof continuous fit), and the key issue in the solution is to understand and specifywhen exactly this happens. This can be done, on one hand, in terms of the aposteriori probability process (i.e. its jump structure and sample path behaviour),and on the other hand, in terms of a singularity point of the equation from the free-boundary problem. Moreover, it turns out that the existence of such a singularitypoint makes explicit computations feasible.

The facts on the principles of continuous and smooth fit presented abovecomplement and further extend our findings in Section 23 above.

For more general problems of Poisson disorder see [9] and [38] and the refer-ences therein.

Chapter VII.

Optimal stopping in mathematical finance

25. The American option


1. According to theory of modern finance (see e.g. [197]) the arbitrage-free priceof the American put option with infinite horizon (perpetual option) is given by

V (x) = supτ

Ex

(e−rτ(K −Xτ )+

)(25.1.1)

where the supremum is taken over all stopping times τ of the geometric Brownianmotion X = (Xt)t≥0 solving

dXt = rXt dt + σXt dBt (25.1.2)

with X0 = x > 0 under Px . We recall that B = (Bt)t≥0 is a standard Brownianmotion process started at zero, r > 0 is the interest rate, K > 0 is the strike(exercise) price, and σ > 0 is the volatility coefficient.

The equation (25.1.2) under Px has a unique (strong) solution given by

Xt = x exp(σBt + (r−σ2/2)t

)(25.1.3)

for t ≥ 0 and x > 0 . The process X is strong Markov (diffusion) with theinfinitesimal generator given by

LX = rx∂

∂x+

σ2

2x2 ∂2

∂x2. (25.1.4)

The aim of this subsection is to compute the arbitrage-free price V from (25.1.1)and to determine the optimal exercise time τ∗ (at which the supremum in (25.1.1)is attained).

376 Chapter VII. Optimal stopping in mathematical finance


From (25.1.1) and (25.1.3) we see that the closer X gets to 0 the less likelythat the gain will increase upon continuation. This suggests that there exists apoint b ∈ (0, K) such that the stopping time

τb = inf t ≥ 0 : Xt ≤ b (25.1.5)

is optimal in the problem (25.1.1). [In (25.1.5) we use the standard conventionthat inf(∅) = ∞ (see Remark 25.2 below).]

Standard arguments based on the strong Markov property (cf. Chapter III)lead to the following free-boundary problem for the unknown value function Vand the unknown point b :

LXV = rV for x > b, (25.1.6)

V (x) = (K − x)+ for x = b, (25.1.7)V ′(x) = −1 for x = b (smooth fit), (25.1.8)

V (x) > (K − x)+ for x > b, (25.1.9)

V (x) = (K − x)+ for 0 < x < b. (25.1.10)

3. To solve the free-boundary problem note that the equation (25.1.6) us-ing (25.1.4) reads

Dx2V ′′ + rxV ′ − rV = 0 (25.1.11)

where we set D = σ2/2 . One may now recognize (25.1.11) as the Cauchy–Eulerequation. Let us thus seek a solution in the form

V (x) = xp. (25.1.12)

Inserting (25.1.12) into (25.1.11) we get

p2 −(1− r

D

)p − r

D= 0. (25.1.13)

The quadratic equation (25.1.13) has two roots, p1 = 1 and p2 = −r/D . Thusthe general solution of (25.1.11) can be written as

V (x) = C1 x + C2 x−r/D (25.1.14)

where C1 and C2 are undetermined constants. From the fact that V (x) ≤ Kfor all x > 0 , we see that C1 must be zero. Thus (25.1.7) and (25.1.8) become

Section 25. The American option 377

two algebraic equations in two unknowns C2 and b (free-boundary). Solving thissystem one gets

C2 =D

r

( K

1 + D/r

)1+r/D

, (25.1.15)

b =K

1 + D/r. (25.1.16)

Inserting (25.1.15) into (25.1.14) upon using that C1 = 0 we conclude that

V (x) =

⎧⎨⎩Dr

(K

1+D/r

)1+r/D

x−r/D if x ∈ [b,∞),

K − x if x ∈ (0, b].(25.1.17)

Note that V is C2 on (0, b) ∪ (b,∞) but only C1 at b . Note also that V isconvex on (0,∞) .

4. In this way we have arrived at the two conclusions of the following theorem.

Theorem 25.1. The arbitrage-free price V from (25.1.1) is given explicitly by(25.1.17) above. The stopping time τb from (25.1.5) with b given by (25.1.16)above is optimal in the problem (25.1.1).

Proof. To distinguish the two functions let us denote the value function from(25.1.1) by V∗(x) for x > 0 . We need to prove that V∗(x) = V (x) for all x > 0where V (x) is given by (25.1.17) above.

1. The properties of V stated following (25.1.17) above show that Ito’sformula (page 67) can be applied to e−rtV (Xt) in its standard form (cf. Subsection3.5). This gives

e−rtV (Xt) = V (x) +∫ t

0

e−rs(LXV − rV )(Xs)I(Xs = b) ds (25.1.18)

+∫ t

0

e−rsσXsV′(Xs) dBs.

Setting G(x) = (K − x)+ we see that (LXG − rG)(x) = −rK < 0 so thattogether with (25.1.6) we have

(LXV − rV ) ≤ 0 (25.1.19)

everywhere on (0,∞) but b . Since Px(Xs = b) = 0 for all s and all x , we seethat (25.1.7), (25.1.9)–(25.1.10) and (25.1.18)–(25.1.19) imply that

e−rt(K − Xt)+ ≤ e−rtV (Xt) ≤ V (x) + Mt (25.1.20)


Mt =∫ t

0

e−rsσXsV′(Xs) dBs. (25.1.21)


(Using that |V ′(x)| ≤ 1 for all x > 0 it is easily verified by standard means thatM is a martingale.)

Let (τn)n≥1 be a localization sequence of (bounded) stopping times for M(for example τn ≡ n will do). Then for every stopping time τ of X we have by(25.1.20) above

e−r(τ∧τn)(K − Xτ∧τn)+ ≤ V (x) + Mτ∧τn (25.1.22)

for all n ≥ 1 . Taking the Px -expectation, using the optional sampling theorem(page 60) to conclude that ExMτ∧τn = 0 for all n , and letting n → ∞ , we findby Fatou’s lemma that

Ex

(e−rτ (K − Xτ )+

) ≤ V (x). (25.1.23)

Taking the supremum over all stopping times τ of X we find that V∗(x) ≤ V (x)for all x > 0 .

2. To prove the reverse inequality (equality) we observe from (25.1.18) uponusing (25.1.6) (and the optional sampling theorem as above) that

Ex

(e−r(τb∧τn)V (Xτb∧τn)

)= V (x) (25.1.24)

for all n ≥ 1 . Letting n → ∞ and using that e−rτbV (Xτb) = e−rτb(K − Xτb

)+

(both expressions being 0 when τb = ∞ ), we find by the dominated convergencetheorem that

Ex

(e−rτb(K − Xτb

)+)

= V (x). (25.1.25)

This shows that τb is optimal in (25.1.1). Thus V∗(x) = V (x) for all x > 0 andthe proof is complete.

Remark 25.2. It is evident from the definition of τb in (25.1.5) and the explicitrepresentation of X in (25.1.3) that τb is not always finite. Using the well-knownDoob formula (see e.g. [197, Chap. VIII, § 2a, (51)])

P(

supt≥0

(Bt − αt) ≥ β)

= e−2αβ (25.1.26)

for α > 0 and β > 0 , it is straightforwardly verified that

Px(τb < ∞) =

⎧⎨⎩ 1 if r ≤ D or x ∈ (0, b],(bx

)(r/D)−1

if r > D and x ∈ (b,∞)(25.1.27)

for x > 0 .



1. The arbitrage-free price of the American put option with finite horizon (cf.(25.1.1) above) is given by


Et,x

(e−rτ(K −Xt+τ )+

)(25.2.1)

where τ is a stopping time of the geometric Brownian motion X = (Xt+s)s≥0

solvingdXt+s = rXt+s ds + σXt+s dBs (25.2.2)

with Xt = x > 0 under Pt,x . We recall that B = (Bs)s≥0 denotes a standardBrownian motion process started at zero, T > 0 is the expiration date (maturity),r > 0 is the interest rate, K > 0 is the strike (exercise) price, and σ > 0 is thevolatility coefficient. Similarly to (25.1.2) the strong solution of (25.2.2) under Pt,x

is given byXt+s = x exp

(σBs + (r−σ2/2)s

)(25.2.3)

whenever t ≥ 0 and x > 0 are given and fixed. The process X is strong Markov(diffusion) with the infinitesimal generator given by

LX = rx∂

∂x+

σ2

2x2 ∂2

∂x2. (25.2.4)

We refer to [197] for more information on the derivation and economic meaningof (25.2.1).

2. Let us determine the structure of the optimal stopping time in the prob-lem (25.2.1).

(i) First note that since the gain function G(x) = (K−x)+ is continuous, itis possible to apply Corollary 2.9 (Finite horizon) with Remark 2.10 and concludethat there exists an optimal stopping time in the problem (25.2.1). From our earlierconsiderations we may therefore conclude that the continuation set equals

C = (t, x) ∈ [0, T )× (0,∞) : V (t, x) > G(x) (25.2.5)

and the stopping set equals

D = (t, x) ∈ [0, T ]× (0,∞) : V (t, x) = G(x) . (25.2.6)

It means that the stopping time τD defined by

τD = inf 0 ≤ s ≤ T − t : Xt+s ∈ D (25.2.7)



(ii) We claim that all points (t, x) with x ≥ K for 0 ≤ t < T belongto the continuation set C . Indeed, this is easily verified by considering τε =inf 0 ≤ s ≤ T − t : Xt+s ≤ K − ε for 0 < ε < K and noting that Pt,x(0 <τε < T − t) > 0 if x ≥ K with 0 ≤ t < T . The strict inequality implies thatEt,x(e−rτε(K − Xt+τε)+) > 0 so that (t, x) with x ≥ K for 0 ≤ t < T cannotbelong to the stopping set D as claimed.

(iii) Recalling the solution to the problem (25.2.1) in the case of infinitehorizon, where the stopping time τ∗ = inf s > 0 : Xs ≤ A∗ is optimal and0 < A∗ < K is explicitly given by Theorem 25.1 above, we see that all points(t, x) with 0 < x ≤ A∗ for 0 ≤ t ≤ T belong to the stopping set D . Moreover,since x → V (t, x) is convex on (0,∞) for each 0 ≤ t ≤ T given and fixed(the latter is easily verified using (25.2.1) and (25.2.3) above), it follows directlyfrom the previous two conclusions about C and D that there exists a functionb : [0, T ] → R satisfying 0 < A∗ ≤ b(t) < K for all 0 ≤ t < T (later we will seethat b(T ) = K as well) such that the continuation set C equals

C = (t, x) ∈ [0, T )× (0,∞) : x > b(t) (25.2.8)

and the stopping set D is the closure of the set

D = (t, x) ∈ [0, T ]× (0,∞) : x < b(t) (25.2.9)

joined with remaining points (T, x) for x ≥ b(T ) . (Below we will show that Vis continuous so that C is open.)

(iv) Since the problem (25.2.1) is time-homogeneous, in the sense that thegain function G(x) = (K−x)+ is a function of space only (i.e. does not dependon time), it follows that the map t → V (t, x) is decreasing on [0, T ] for eachx ∈ (0,∞) . Hence if (t, x) belongs to C for some x ∈ (0,∞) and we take anyother 0 ≤ t′ < t ≤ T , then V (t′, x) − G(x) ≥ V (t, x) − G(x) > 0 , showingthat (t′, x) belongs to C as well. From this we may conclude that the boundaryt → b(t) in (25.2.8) and (25.2.9) is increasing on [0, T ] .

3. Let us show that the value function (t, x) → V (t, x) is continuous on[0, T ]× (0,∞) .

For this, it is enough to prove that

x → V (t, x) is continuous at x0, (25.2.10)t → V (t, x) is continuous at t0 uniformly over x ∈ [x0 − δ, x0 + δ] (25.2.11)

for each (t0, x0) ∈ [0, T ]× (0,∞) with some δ > 0 small enough (it may dependon x0 ).

Since (25.2.10) follows from the fact that x → V (t, x) is convex on (0,∞) ,it remains to establish (25.2.11).


For this, let us fix arbitrary 0 ≤ t1 < t2 ≤ T and x ∈ (0,∞) , and letτ1 = τ∗(t1, x) denote the optimal stopping time for V (t1, x) . Set τ2 = τ1∧(T−t2)and note, since t → V (t, x) is decreasing on [0, T ] , that upon denoting St =exp(σBt + γt) with γ = r − σ2/2 we have

0 ≤ V (t1, x) − V (t2, x) (25.2.12)

≤ E(e−rτ1(K − xSτ1)

+)− E

(e−rτ2(K − xSτ2)

+)

≤ E(e−rτ2

[(K − xSτ1)

+ − (K − xSτ2)+])

≤ xE (Sτ2 − Sτ1)+

where we use that τ2 ≤ τ1 and that (K − y)+−(K − z)+ ≤ (z−y)+ for y, z ∈ R .

Set Zt = σBt + γt and recall that stationary independent increments ofZ = (Zt)t≥0 imply that (Zτ2+t −Zτ2)t≥0 is a version of Z , i.e. the two processeshave the same law. Using that τ1 − τ2 ≤ t2 − t1 hence we get

E (Sτ2 − Sτ1)+ = E

(E((Sτ2 − Sτ1)

+ | Fτ2

))(25.2.13)

= E(Sτ2 E

((1 − Sτ1/Sτ2)

+ | Fτ2

))= E

(Sτ2 E

((1 − eZτ1−Zτ2 )+ | Fτ2

))= E (Sτ2)E

(1 − eZτ1−Zτ2

)+= E (Sτ2)E

(1 − inf

0≤t≤t2−t1eZτ2+t−Zτ2

)= E (Sτ2)E

(1 − inf

0≤t≤t2−t1eZt

)=: E (Sτ2)L(t2 − t1)

where we also used that Zτ1 −Zτ2 is independent from Fτ2 . By basic propertiesof Brownian motion it is easily seen that L(t2 − t1) → 0 as t2 − t1 → 0 .

Combining (25.2.12) and (25.2.13) we find by the martingale property of(exp(σBt − (σ2/2)t)

)t≥0

that

0 ≤ V (t1, x) − V (t2, x) ≤ xE (Sτ2)L(t2 − t1) ≤ x erT L(t2 − t1) (25.2.14)

from where (25.2.11) becomes evident. This completes the proof.

4. In order to prove that the smooth-fit condition (25.2.28) holds, i.e. thatx → V (t, x) is C1 at b(t) , let us fix a point (t, x) ∈ (0, T ) × (0,∞) lying onthe boundary b so that x = b(t) . Then x < K and for all ε > 0 such thatx + ε < K we have

V (t, x + ε) − V (t, x)ε

≥ G(x + ε) − G(x)ε

= −1 (25.2.15)


∂+V

∂x(t, x) ≥ G′(x) = −1 (25.2.16)


where the right-hand derivative exists (and is finite) by virtue of the convexity ofthe mapping x → V (t, x) on (0,∞) . (Note that the latter will also be provedindependently below.)

To prove the converse inequality, let us fix ε > 0 such that x + ε < K , andconsider the stopping time τε = τ∗(t, x + ε) being optimal for V (t, x + ε) . Thenwe have

V (t, x + ε) − V (t, x) (25.2.17)

≤ E(e−rτε(K − (x + ε)Sτε)

+)− E

(e−rτε(K − xSτε)

+)

≤ E(e−rτε

[(K − (x + ε)Sτε)

+ − (K − xSτε)+]I((x + ε)Sτε < K

))= −ε E

(e−rτεSτεI

((x + ε)Sτε < K

)).

Using that s → − γσ s is a lower function of B at zero and the fact that the

optimal boundary s → b(s) is increasing on [t, T ] , it is not difficult to verify thatτε → 0 P-a.s. as ε ↓ 0 . In particular, this implies that

E(e−rτεSτε I((x+ε)Sτε < K)

)→ 1 (25.2.18)

as ε ↓ 0 by the dominated convergence theorem.

Combining (25.2.17) and (25.2.18) we see that

∂+V

∂x(t, x) ≤ G′(x) = −1 (25.2.19)

which together with (25.2.16) completes the proof.

5. We proceed to prove that the boundary b is continuous on [0, T ] andthat b(T ) = K .

(i) Let us first show that the boundary b is right-continuous on [0, T ] .For this, fix t ∈ (0, T ] and consider a sequence tn ↓ t as n → ∞ . Since bis increasing, the right-hand limit b(t+) exists. Because (tn, b(tn)) ∈ D for alln ≥ 1 , and D is closed, we get that (t, b(t+)) ∈ D . Hence by (25.2.9) we seethat b(t+) ≤ b(t) . The reverse inequality follows obviously from the fact that bis increasing on [0, T ] , thus proving the claim.

(ii) Suppose that at some point t∗ ∈ (0, T ) the function b makes a jump,i.e. let b(t∗) > b(t∗−) . Let us fix a point t′ < t∗ close to t∗ and consider thehalf-open set R ⊆ C being a curved trapezoid formed by the vertices (t′, b(t′)) ,(t∗, b(t∗−)) , (t∗, x′) and (t′, x′) with any x′ fixed arbitrarily in the interval(b(t∗−), b(t∗)) .

Recall that the strong Markov property (cf. Chapter III) implies that thevalue function V is C1,2 in C . Note also that the gain function G is C2 in


R so that by the Newton–Leibniz formula using (25.2.27) and (25.2.28) it followsthat

V (t, x) − G(x) =∫ x

b(t)

∫ u

b(t)

(Vxx(t, v) − Gxx(v)) dv du (25.2.20)

for all (t, x) ∈ R . Moreover, the strong Markov property (cf. Chapter III) impliesthat the value function V solves the equation (25.2.26) from where using thatt → V (t, x) and x → V (t, x) are decreasing so that Vt ≤ 0 and Vx ≤ 0 in C ,we obtain

Vxx(t, x) =2

σ2x2

(rV (t, x) − Vt(t, x) − rxVx(t, x)

)(25.2.21)

=2

σ2x2r(K − x)+ ≥ c > 0

for each (t, x) ∈ R where c > 0 is small enough.

Hence by (25.2.20) using that Gxx = 0 in R we get

V (t′, x′) − G(x′) ≥ c(x′ − b(t′))2

2−→ c

(x′ − b(t∗))2

2> 0 (25.2.22)

as t′ ↑ t∗ . This implies that V (t∗, x′) > G(x′) which contradicts the fact that(t∗, x′) belong to the stopping set D . Thus b(t∗+) = b(t∗) showing that b iscontinuous at t∗ and thus on [0, T ] as well.

(iii) We finally note that the method of proof from the previous part (ii) alsoimplies that b(T ) = K . To see this, we may let t∗ = T and likewise supposethat b(T ) < K . Then repeating the arguments presented above word by word wearrive at a contradiction with the fact that V (t, x) = G(x) for all x ∈ [b(T ), K] .

6. Summarizing the facts proved in paragraphs 1–5 above we may concludethat the following hitting time is optimal in the problem (25.2.1):

τb = inf 0 ≤ s ≤ T − t : Xt+s ≤ b(t+s) (25.2.23)

(the infimum of an empty set being equal to T −t ) where the boundary b satisfiesthe properties

b : [0, T ] → (0, K] is continuous and increasing, (25.2.24)b(T ) = K. (25.2.25)

(see Figure VII.1).

Standard arguments based on the strong Markov property (cf. Chapter III)lead to the following free-boundary problem for the unknown value function V


β

Κ

x

Tτb

t Xt→ t b(t)→

Figure VII.1: A computer drawing of the optimal stopping boundary bfrom Theorem 25.3. The number β is the optimal stopping point in thecase of infinite horizon (Theorem 25.1).

and the unknown boundary b :

Vt + LXV = rV in C, (25.2.26)

V (t, x) = (K − x)+ for x = b(t), (25.2.27)Vx(t, x) = −1 for x = b(t) (smooth fit), (25.2.28)

V (t, x) > (K − x)+ in C, (25.2.29)

V (t, x) = (K − x)+ in D (25.2.30)

where the continuation set C is defined in (25.2.8) above and the stopping set Dis the closure of the set D in (25.2.9) above.

7. The following properties of V and b were verified above:

V is continuous on [0, T ]×R+, (25.2.31)

V is C1,2 on C (and C1,2 on D), (25.2.32)x → V (t, x) is decreasing and convex with Vx(t, x) ∈ [−1, 0], (25.2.33)

t → V (t, x) is decreasing with V (T, x) = (K −x)+, (25.2.34)t → b(t) is increasing and continuous with 0 < b(0+) < K (25.2.35)and b(T−) = K.


Note also that (25.2.28) means that x → V (t, x) is C1 at b(t) .

Once we know that V satisfying (25.2.28) is “sufficiently regular” (cf. foot-note 14 in [27] when t → V (t, x) is known to be C1 for all x ), we can apply Ito’sformula (page 67) to e−rsV (t+s, Xt+s) in its standard form and take the Pt,x -expectation on both sides in the resulting identity. The martingale term then van-ishes by the optional sampling theorem (page 60) using the final part of (25.2.33)above, so that by (25.2.26) and (25.2.27)+(25.2.30) upon setting s = T − t (beingthe key advantage of the finite horizon) one obtains the early exercise premiumrepresentation of the value function

V (t, x) = e−r(T−t)Et,xG(XT ) (25.2.36)

−∫ T−t

0

e−ruEt,x

(H(t−u, Xt+u) I

(Xt+u ≤ b(t+u)

))du

= e−r(T−t)Et,xG(XT ) + rK

∫ T−t

0

e−ruPt,x

(Xt+u ≤ b(t+u)

)du

for all (t, x) ∈ [0, T ]×R+ where we set G(x) = (K −x)+ and H = Gt+LXG−rGso that H = −rK for x < b(t) .

A detail worth mentioning in this derivation (see (25.2.47) below) is that(25.2.36) follows from (3.5.9) with F (t+s, Xt+s) = e−rsV (t+s, Xt+s) withoutknowing a priori that t → V (t, x) is C1 at b(t) as required under the condition of“sufficiently regular” recalled prior to (25.2.36) above. This approach is more directsince the sufficient conditions (3.5.10)–(3.5.13) for (3.5.9) are easier verified thansufficient conditions [such as b is C1 or (locally) Lipschitz] for t → V (t, x) to beC1 at b(t) . This is also more in the spirit of the free-boundary equation (25.2.39)to be derived below where neither differentiability nor a Lipschitz property of bplays a role in the formulation.

Since V (t, x) = G(x) = (K −x)+ in D by (25.2.27)+(25.2.30), we see that(25.2.36) reads

K − x = e−r(T−t) Et,x(K −XT )+ (25.2.37)

+ rK

∫ T−t

0

e−ruPt,x

(Xt+u ≤ b(t+u)

)du

for all x ∈ (0, b(t)] and all t ∈ [0, T ] .

8. A natural candidate equation is obtained by inserting x = b(t) in (25.2.37).This leads to the free-boundary equation (cf. Subsection 14.1)

K − b(t) = e−r(T−t) Et,b(t)(K −XT )+ (25.2.38)

+ rK

∫ T−t

0

e−ru Pt,b(t)

(Xt+u ≤ b(t+u)

)du


which upon using (25.2.3) more explicitly reads as follows:

K − b(t) (25.2.39)

= e−r(T−t)

∫ K

0

Φ(

1σ√

T − t

(log

K − z

b(t)−(r− σ2

2

)(T − t)

))dz

+ rK

∫ T−t

0

e−ru Φ(

1σ√

u

(log

b(t+u)b(t)

−(r− σ2

2

)u

))du

for all t ∈ [0, T ] where Φ(x) = (1/√

2π)∫ x

−∞ e−z2/2dz for x ∈ R . It is a nonlinearVolterra integral equation of the second kind (see [212]).

9. The main result of the present subsection may now be stated as follows(see also Remark 25.5 below).

Theorem 25.3. The optimal stopping boundary in the American put problem(25.2.1) can be characterized as the unique solution of the free-boundary equa-tion (25.2.39) in the class of continuous increasing functions c : [0, T ] → R satis-fying 0 < c(t) < K for all 0 < t < T .

Proof. The fact that the optimal stopping boundary b solves (25.2.38) i.e. (25.2.39)was derived above. The main emphasis of the theorem is thus on the claim ofuniqueness. Let us therefore assume that a continuous increasing c : [0, T ] → R

solving (25.2.39) is given such that 0 < c(t) < K for all 0 < t < T , and let usshow that this c must then coincide with the optimal stopping boundary b . Theproof of this implication will be presented in the nine steps as follows.

1. In view of (25.2.36) and with the aid of calculations similar to thoseleading from (25.2.38) to (25.2.39), let us introduce the function

U c(t, x) (25.2.40)

= e−r(T−t) Et,xG(XT ) + rK

∫ T−t

0

e−ru Pt,x

(Xt+u ≤ c(t+u)

)du

= e−r(T−t) U c1(t, x) + rK U c

2(t, x)

where U c1 and U c

2 are defined as follows:

U c1(t, x) =

∫ K

0

Φ(

1σ√

T − t

(log

K − z

x−γ (T − t)

))dz, (25.2.41)

U c2(t, x) =

∫ T

t

e−r(v−t) Φ(

1σ√

v− t

(log

c(v)x

−γ (v− t)))

dv (25.2.42)

for all (t, x) ∈ [0, T )×(0,∞) upon setting γ = r−σ2/2 and substituting v = t+u .


Denoting ϕ = Φ′ we then have

∂U c1

∂x(t, x) = − 1

σx√

T − t

∫ K

0

ϕ

(1

σ√

T − t

(log

K − z

x−γ (T − t)

))dz, (25.2.43)

∂U c2

∂x(t, x) = − 1

σx

∫ T

t

e−r(v−t)

√v− t

ϕ

(1

σ√

v− t

(log

c(v)x

−γ (v− t)))

dv (25.2.44)

for all (t, x) ∈ [0, T ) × (0,∞) where the interchange of differentiation and inte-gration is justified by standard means. From (25.2.43) and (25.2.44) we see that∂U c

1/∂x and ∂U c2/∂x are continuous on [0, T )×(0,∞) , which in view of (25.2.40)

implies that U cx is continuous on [0, T )× (0,∞) .

2. In accordance with (25.2.36) define a function V c : [0, T ) × (0,∞) → R

by setting V c(t, x) = U c(t, x) for x > c(t) and V c(t, x) = G(x) for x ≤ c(t)when 0 ≤ t < T . Note that since c solves (25.2.39) we have that V c is continuouson [0, T )× (0,∞) , i.e. V c(t, x) = U c(t, x) = G(x) for x = c(t) when 0 ≤ t < T .Let C1 and C2 be defined by means of c as in (3.5.3) and (3.5.4) with [0, T )instead of R+ , respectively.

Standard arguments based on the Markov property (or a direct verification)show that V c i.e. U c is C1,2 on C1 and that

V ct + LXV c = rV c in C1. (25.2.45)

Moreover, since U cx is continuous on [0, T )× (0,∞) we see that V c

x is continuouson C1 . Finally, since 0 < c(t) < K for 0 < t < T we see that V c i.e. G is C1,2

on C2 .

3. Summarizing the preceding conclusions one can easily verify that with(t, x) ∈ [0, T ) × (0,∞) given and fixed, the function F : [0, T − t) × (0,∞) → R

defined byF (s, y) = e−rsV c(t+s, xy) (25.2.46)

satisfies (3.5.10)–(3.5.13) (in the relaxed form) so that (3.5.9) can be applied. Inthis way we get

e−rsV c(t+s, Xt+s) = V c(t, x) (25.2.47)

+∫ s

0

e−ru(V c

t +LXV c − rV c)(t+u, Xt+u) I(Xt+u = c(t+u)) du

+ M cs +

12

∫ s

0

e−ru∆xV cx (t+u, c(t+u)) dc

u(X)

where M cs =

∫ s

0 e−ruV cx (t+u, Xt+u)σXt+u I(Xt+u = c(t+u)) dBu and we set

∆xV cx (v, c(v)) = V c

x (v, c(v)+)−V cx (v, c(v)−) for t ≤ v ≤ T . Moreover, it is easily

seen from (25.2.43) and (25.2.44) that (M cs )0≤s≤T−t is a martingale under Pt,x

so that Et,xM cs = 0 for each 0 ≤ s ≤ T − t .


4. Setting s = T − t in (25.2.47) and then taking the Pt,x -expectation,using that V c(T, x) = G(x) for all x > 0 and that V c satisfies (25.2.45) in C1 ,we get

e−r(T−t)Et,xG(XT ) = V c(t, x) (25.2.48)

+∫ T−t

0

e−ruEt,x

(H(t+u, Xt+u) I(Xt+u ≤ c(t+u))

)du

+12

∫ T−t

0

e−ru∆xV cx (t+u, c(t+u)) duEt,x(c

u(X))

for all (t, x) ∈ [0, T )× (0,∞) where H = Gt + LXG − rG = −rK for x ≤ c(t) .From (25.2.48) we thus see that

V c(t, x) = e−r(T−t)Et,xG(XT ) (25.2.49)

+ rK

∫ T−t

0

e−ruPt,x(Xt+u ≤ c(t+u)) du

− 12

∫ T−t

0

e−ru∆xV cx (t+u, c(t+u)) duEt,x(c

u(X))

for all (t, x) ∈ [0, T ) × (0,∞) . Comparing (25.2.49) with (25.2.40), and recallingthe definition of V c in terms of U c and G , we get∫ T−t

0

e−ru∆xV cx (t+u, c(t+u))duEt,x(c

u(X)) (25.2.50)

= 2(U c(t, x)−G(x)

)I(x ≤ c(t))

for all 0 ≤ t < T and x > 0 , where I(x ≤ c(t)) equals 1 if x ≤ c(t) and 0 ifx > c(t) .


x → V c(t, x) is C1 at c(t) (25.2.51)

for each 0 ≤ t < T given and fixed, then it will follow that

U c(t, x) = G(x) for all 0 < x ≤ c(t). (25.2.52)

On the other hand, if we know that (25.2.52) holds, then using the general fact

∂

∂x

(U c(t, x) − G(x)

)∣∣∣x=c(t)

= V cx (t, c(t)+) − V c

x (t, c(t)−) (25.2.53)

= ∆xV cx (t, c(t))

for all 0 ≤ t < T , we see that (25.2.51) holds too (since U cx is continuous). The

equivalence of (25.2.51) and (25.2.52) just explained then suggests that instead of


dealing with the equation (25.2.50) in order to derive (25.2.51) above (which wasthe content of an earlier proof) we may rather concentrate on establishing (25.2.52)directly. [To appreciate the simplicity and power of the probabilistic argument tobe given shortly below one may differentiate (25.2.50) with respect to x , computethe left-hand side explicitly (taking care of a jump relation), and then try to provethe uniqueness of the zero solution to the resulting (weakly singular) Volterraintegral equation using any of the known analytic methods (see e.g. [212]).]

6. To derive (25.2.52) first note that standard arguments based on theMarkov property (or a direct verification) show that U c is C1,2 on C2 and that

U ct + LXU c − rU c = −rK in C2 . (25.2.54)

Since F in (25.2.46) with U c instead of V c is continuous and satisfies (3.5.10)–(3.5.13) (in the relaxed form), we see that (3.5.9) can be applied just as in (25.2.47),and this yields

e−rsU c(t+s, Xt+s) (25.2.55)

= U c(t, x) − rK

∫ s

0

e−ruI(Xt+u ≤ c(t+u)) du + M cs

upon using (25.2.45) and (25.2.54) as well as that ∆xU cx(t+u, c(t+u)) = 0 for all

0 ≤ u ≤ s since U cx is continuous. In (25.2.55) we have M c

s =∫ s

0 e−ruU cx(t+u,

Xt+u)σXt+u I(Xt+u = c(t+u)) dBu and (M cs )0≤s≤T−t is a martingale under

Pt,x .

Next note that (3.5.9) applied to F in (25.2.46) with G instead of V c yields

e−rsG(Xt+s) = G(x) − rK

∫ s

0

e−ruI(Xt+u < K) du (25.2.56)

+ MKs +

12

∫ s

0

e−ru dKu (X)

upon using that Gt + LXG − rG equals −rK on (0, K) and 0 on (K,∞) aswell as that ∆xGx(t+u, K) = 1 for 0 ≤ u ≤ s . In (25.2.56) we have MK

s =∫ s

0 e−ru G′(Xt+u)σXt+u I(Xt+u = K) dBu = − ∫ s

0 e−ru σXt+u I(Xt+u < K) dBu

and (MKs )0≤s≤T−t is a martingale under Pt,x .

For 0 < x ≤ c(t) consider the stopping time

σc = inf 0 ≤ s ≤ T − t : Xt+s ≥ c(t+s) . (25.2.57)

Then using that U c(t, c(t)) = G(c(t)) for all 0 ≤ t < T since c solves (25.2.9),and that U c(T, x) = G(x) for all x > 0 by (25.2.40), we see that U c(t +σc, Xt+σc) = G(Xt+σc) . Hence from (25.2.55) and (25.2.56) using the optional


sampling theorem (page 60) we find

U c(t, x) = Et,x

(e−rσcU c(t+σc, Xt+σc)

)(25.2.58)

+ rK Et,x

(∫ σc

0

e−ruI(Xt+u ≤ c(t+u)) du

)= Et,x

(e−rσcG(Xt+σc)

)+ rK Et,x

(∫ σc

0


)= G(x) − rK Et,x

(∫ σc

0

e−ruI(Xt+u <K) du

)+ rK Et,x

(∫ σc

0


)= G(x)

since Xt+u < K and Xt+u ≤ c(t + u) for all 0 ≤ u < σc . This estab-lishes (25.2.52) and thus (25.2.51) holds as well as explained above.

7. Consider the stopping time

τc = inf 0 ≤ s ≤ T − t : Xt+s ≤ c(t+s) . (25.2.59)

Note that (25.2.47) using (25.2.45) and (25.2.51) reads

e−rsV c(t+s, Xt+s) = V c(t, x) (25.2.60)

+∫ s

0

e−ruH(t+u, Xt+u) I(Xt+u≤c(t+u)) du + M cs

where H = Gt+LXG−rG = −rK for x ≤ c(t) and (M cs )0≤s≤T−t is a martingale

under Pt,x . Thus Et,xM cτc

= 0 , so that after inserting τc in place of s in (25.2.60),it follows upon taking the Pt,x -expectation that

V c(t, x) = Et,x

(e−rτc(K −Xt+τc)

+)

(25.2.61)

for all (t, x) ∈ [0, T )× (0,∞) where we use that V c(t, x) = G(x) = (K −x)+ forx ≤ c(t) or t = T . Comparing (25.2.61) with (25.2.1) we see that

V c(t, x) ≤ V (t, x) (25.2.62)

for all (t, x) ∈ [0, T )× (0,∞) .

8. Let us now show that c ≥ b on [0, T ] . For this, recall that by the samearguments as for V c we also have

e−rsV (t+s, Xt+s) = V (t, x) (25.2.63)

+∫ s

0

e−ruH(t+u, Xt+u) I(Xt+u≤b(t+u)) du + M bs


where H = Gt+LXG−rG = −rK for x ≤ b(t) and (M bs )0≤s≤T−t is a martingale

under Pt,x . Fix (t, x) ∈ (0, T ) × (0,∞) such that x < b(t) ∧ c(t) and considerthe stopping time

σb = inf 0 ≤ s ≤ T − t : Xt+s ≥ b(t+s) . (25.2.64)

Inserting σb in place of s in (25.2.60) and (25.2.63) and taking the Pt,x -expec-tation, we get

Et,x

(e−rσbV c(t+σb, Xt+σb

))

= G(x) (25.2.65)

− rK Et,x

(∫ σb

0


),

Et,x

(e−rσbV (t+σb, Xt+σb

))

= G(x) − rK Et,x

(∫ σb

0

e−ru du

). (25.2.66)

Hence by (25.2.62) we see that

Et,x

(∫ σb

0

e−ruI(Xt+u≤c(t+u)) du

)≥ Et,x

(∫ σb

0

e−ru du

)(25.2.67)

from where it follows by the continuity of c and b that c(t) ≥ b(t) for all0 ≤ t ≤ T .

9. Finally, let us show that c must be equal to b . For this, assume thatthere is t ∈ (0, T ) such that c(t) > b(t) , and pick x ∈ (b(t), c(t)) . Under Pt,x con-sider the stopping time τb from (25.2.23). Inserting τb in place of s in (25.2.60)and (25.2.63) and taking the Pt,x -expectation, we get

Et,x

(e−rτbG(Xt+τb

))

= V c(t, x) (25.2.68)

− rK Et,x

(∫ τb

0


),

Et,x

(e−rτbG(Xt+τb

))

= V (t, x). (25.2.69)


Et,x

(∫ τb

0


)≤ 0 (25.2.70)

from where it follows by the continuity of c and b that such a point x cannotexist. Thus c must be equal to b , and the proof is complete.

Remark 25.4. The fact that U c defined in (25.2.40) must be equal to G be-low c when c solves (25.2.39) is truly remarkable. The proof of this fact givenabove (paragraphs 2– 6 ) follows the way which led to its discovery. A shorter


but somewhat less revealing proof can also be obtained by introducing U c asin (25.2.40) and then verifying directly (using the Markov property only) that

e−rsU c(t+s, Xt+s) + rK

∫ s

0

e−ruI(Xt+u≤c(t+u)) du (25.2.71)

is a martingale under Pt,x for 0 ≤ s ≤ T − t . In this way it is possible tocircumvent the material from paragraphs 2– 4 and carry out the rest of the proofstarting with (25.2.56) onward. Moreover, it may be noted that the martingaleproperty of (25.2.71) does not require that c is increasing (but only measurable).This shows that the claim of uniqueness in Theorem 25.3 holds in the class ofcontinuous (or left-continuous) functions c : [0, T ] → R such that 0 < c(t) < Kfor all 0 < t < T . It also identifies some limitations of the approach based onthe local time-space formula (cf. Subsection 3.5) as initially undertaken (where cneeds to be of bounded variation).

Remark 25.5. Note that in Theorem 25.3 above we do not assume that the solutionstarts (ends) at a particular point. The equation (25.2.39) is highly nonlinear andseems to be out of the scope of any existing theory on nonlinear integral equations(the kernel having four arguments). Similar equations arise in the first-passageproblem for Brownian motion (cf. Subsection 14.2).

Notes. According to theory of modern finance (see e.g. [197]) the arbitrage-free price of the American put option with a strike price K coincides withthe value function V of the optimal stopping problem with the gain functionG = (K −x)+ . The optimal stopping time in this problem is the first time whenthe price process (geometric Brownian motion) falls below the value of a time-dependent boundary b . When the option’s expiration date T is finite, the math-ematical problem of finding V and b is inherently two-dimensional and thereforeanalytically more difficult (for infinite T the problem is one-dimensional and bis constant).

The first mathematical analysis of the problem is due to McKean [133] whoconsidered a “discounted” American call with the gain function G = e−βt(x−K)+

and derived a free-boundary problem for V and b . He further expressed V interms of b so that b itself solves a countable system of nonlinear integral equations(p. 39 in [133]). The approach of expressing V in terms of b was in line with theideas coming from earlier work of Kolodner [114] on free-boundary problems inmathematical physics (such as Stefan’s ice melting problem). The existence anduniqueness of a solution to the system for b derived by McKean was left openin [133].

McKean’s work was taken further by van Moerbeke [215] who derived asingle nonlinear integral equation for b (pp. 145–146 in [215]). The connectionto the physical problem is obtained by introducing the auxiliary function V =∂(V−G)/∂t so that the “smooth-fit condition” from the optimal stopping problem


translates into the “condition of heat balance” (i.e. the law of conservation ofenergy) in the physical problem. A motivation for the latter may be seen from thefact that in the mathematical physics literature at the time it was realized that theexistence and local uniqueness of a solution to such nonlinear integral equationscan be proved by applying the contraction principle (fixed point theorem), firstfor a small time interval and then extending it to any interval of time by induction(see [137] and [70]). Applying this method, van Moerbeke has proved the existenceand local uniqueness of a solution to the integral equations of a general optimalstopping problem (see Sections 3.1 and 3.2 in [215]) while the proof of the sameclaim in the context of the discounted American call [133] is merely indicated (seeSection 4.4 in [215]). One of the technical difficulties in this context is that thederivative b′ of the optimal boundary b is not bounded at the initial point T asused in the general proof (cf. Sections 3.1 and 3.2 in [215]).

The fixed point method usually results in a long and technical proof withan indecisive end where the details are often sketched or omitted. Another conse-quence of the approach is the fact that the integral equations in [133] and [215]involve both b and its derivative b′ , so that either the fixed point method resultsin proving that b is differentiable, or this needs to be proved a priori if the existenceis claimed simply by identifying b with the boundary of the set where V = G .The latter proof, however, appears difficult to give directly, so that if one is onlyinterested in the actual values of b which indicate when to stop, it seems thatthe differentiability of b plays a minor role. Finally, since it is not obvious to see(and it was never explicitly addressed) how the “condition of heat balance” relatesto the economic mechanism of “no-arbitrage” behind the American option, one isled to the conclusion that the integral equations derived by McKean and van Mo-erbeke, being motivated purely by the mathematical tractability arising from thework in mathematical physics, are perhaps more complicated then needed fromthe standpoint of optimal stopping.

This was to be confirmed in the beginning of the 1990’s when Kim [110], Jacka[102] and Carr, Jarrow, Myneni [27] independently arrived at a nonlinear integralequation for b that is closely linked to the early exercise premium representationof V having a clear economic meaning (see Section 1 in [27] and Corollary 3.1in [142]). In fact, the equation is obtained by inserting x = b(t) in this represen-tation, and for this reason it is called the free-boundary equation (see (25.2.39)above). The early exercise premium representation for V follows transparentlyfrom the free-boundary formulation (given that the smooth-fit condition holds)and moreover corresponds to the decomposition of the superharmonic function Vinto its harmonic and its potential part (the latter being the basic principle ofoptimal stopping established in the works of Snell [206] and Dynkin [52]).

The superharmonic characterization of the value function V (cf. ChapterI) implies that e−rsV (t− s, Xt+s) is the smallest supermartingale dominatinge−rs(K −Xt+s)+ on [0, T − t] , i.e. that V (t, x) is the smallest superharmonicfunction (relative to ∂/∂t + LX − rI ) dominating (K −x)+ on [0, T ]×R+ . The


two requirements (i.e. smallest and superharmonic) manifest themselves in thesingle analytic condition of smooth fit (25.2.28).

The derivation of the smooth-fit condition given in Myneni [142] upon inte-grating the second formula on p. 15 and obtaining the third one seems to violatethe Newton–Leibniz formula unless x → V (t, x) is smooth at b(t) so that there isnothing to prove. Myneni writes that this proof is essentially from McKean [133].A closer inspection of his argument on p. 38 in [133] reveals the same difficulty.Alternative derivations of the smooth-fit principle for Brownian motion and dif-fusions are given in Grigelionis & Shiryaev [88] and Chernoff [30] by a Taylorexpansion of V at (t, b(t)) and in Bather [11] and van Moerbeke [215] by aTaylor expansion of G at (t, b(t)) . The latter approach seems more satisfactorygenerally since V is not known a priori. Jacka [104] (Corollary 7) develops adifferent approach which he applies in [102] (Proposition 2.8) to verify (25.2.28).

It follows from the preceding that the optimal stopping boundary b satis-fies the free-boundary equation, however, as pointed out by Myneni [142] (p. 17)“the uniqueness and regularity of the stopping boundary from this integral equa-tion remain open”. This attempt is in line with McKean [133] (p. 33) who wrotethat “another inviting unsolved problem is to discuss the integral equation for thefree-boundary of section 6”, concluding the paper (p. 39) with the words “eventhe existence and uniqueness of solutions is still unproved”. McKean’s integralequations [133] (p. 39) are more complicated (involving b′ as well) than the equa-tion (25.2.37). Thus the simplification of his equations to the equations (25.2.37)and finally the equation (25.2.39) may be viewed as a step to the solution of theproblem. Theorem 4.3 of Jacka [102] states that if c : [0, T ] → R is a “left-continuous” solution of (25.2.37) for all x ∈ (0, c(t)] satisfying 0 < c(t) < K forall t ∈ (0, T ) , then c is the optimal stopping boundary b . Since (25.2.37) isa different equation for each new x ∈ (0, c(t)] , we see that this assumption ineffect corresponds to c solving a countable system of nonlinear integral equations(obtained by letting x in (0, c(t)] run through rationals for instance). From thestandpoint of numerical calculation it is therefore of interest to reduce the numberof these equations.

The main purpose of the present section (following [164]) is to show thatthe question of Myneni can be answered affirmatively and that the free-boundaryequation alone does indeed characterize the optimal stopping boundary b . The keyargument in the proof is based on the local time-space formula [163] (see Subsection3.5). The same method of proof can be applied to more general continuous Markovprocesses (diffusions) in problems of optimal stopping with finite horizon. Forexample, in this way it is also possible to settle the somewhat more complicatedproblem of the Russian option with finite horizon [165] (see Section 26 below).

With reference to [133] and [215] it is claimed in [142] (and used in someother papers too) that b is C1 but we could not find a complete/transparentproof in either of these papers (nor anywhere else). If it is known that b is C1 ,then the proof above shows that C in (25.2.32) can be replaced by C , implyingalso that s → V (s, b(t)) is C1 at t . For both, in fact, it is sufficient to know

Section 26. The Russian option 395

that b is (locally) Lipschitz, but it seems that this fact is no easier to establishdirectly, and we do not know of any transparent proof.

For more information on the American option problem we refer to the surveypaper [142], the book [197] and Sections 2.5–2.8 in the book [107] where furtherreferences can also be found. For a numerical discussion of the free-boundaryequation and possible improvements along these lines see e.g. [93]. For asymptoticsof the optimal stopping boundary see [121], and for a proof that it is convex see[58]. For random walks and Levy processes see [33], [140] and [2].

26. The Russian option


1. The arbitrage-free price of the Russian option with infinite horizon (perpetualoption) is given by

V = supτ

E(e−(r+λ)τMτ

)(26.1.1)

where the supremum is taken over all stopping times τ of the geometric Brownianmotion S = (St)t≥0 solving

dSt = rSt dt + σSt dBt (S0 = s) (26.1.2)

and M = (Mt)t≥0 is the maximum process given by

Mt =(

max0≤u≤t

Su

)∨ m (26.1.3)

where m≥ s > 0 are given and fixed. We recall that B = (Bt)t≥0 is a standardBrownian motion process started at zero, r > 0 is the interest rate, λ > 0 is thediscounting rate, and σ > 0 is the volatility coefficient.

2. The problem (26.1.1) is two-dimensional since the underlying Markov pro-cess may be identified with (S, M) . Using the same method as in Section 13it is possible to solve the problem (26.1.1) explicitly. Instead we will follow adifferent route to the explicit solution using a change of measure (cf. Subsec-tion 15.3) which reduces the initial two-dimensional problem to an equivalentone-dimensional problem (cf. Subsection 6.2). This reduction becomes especiallyhandy in the case when the horizon in (26.1.1) is finite (cf. Subsection 26.2 below).

Recalling that the strong solution of (26.1.2) is given by (26.1.9) below andwriting Mτ in (26.1.1) as Sτ (Mτ/Sτ ) , we see by change of measure that

V = s supτ

E(e−λτXτ ) (26.1.4)

where we setXt =

Mt

St(26.1.5)


and P is a probability measure satisfying dP = exp(σBt − (σ2/2) t

)dP when

restricted to FBt = σ(Bs : 0 ≤ s ≤ t) for t ≥ 0 . By Girsanov’s theorem (see

[106] or [197]) we see that the process B = (Bt)t≥0 given by Bt = Bt − σt isa standard Brownian motion under P for t ≥ 0 . By Ito’s formula (page 67) onefinds that the process X = (Xt)t≥0 solves

dXt = −rXt dt + σXt dBt + dRt (X0 = x) (26.1.6)

under P where B = −B is a standard Brownian motion, and we set

Rt =∫ t

0

I(Xs = 1)dMs

Ss(26.1.7)

for t ≥ 0 . It follows that X is a diffusion process in [1,∞) having 1 as aboundary point of instantaneous reflection. The infinitesimal generator of X istherefore given by

LX = −rx∂

∂x+

σ2

2x2 ∂2

∂x2in (1,∞), (26.1.8)

∂

∂x= 0 at 1+.

The latter means that the infinitesimal generator of X is acting on a space of C2

functions f defined on [1,∞) such that f ′(1+) = 0.

3. For further reference recall that the strong solution of (26.1.2) is given by

St = s exp(

σBt +(r− σ2

2

)t

)= s exp

(σBt +

(r+

σ2

2

)t

)(26.1.9)

for t ≥ 0 where B and B are standard Brownian motions with respect to Pand P respectively. When dealing with the process X on its own, however, notethat there is no restriction to assume that s = 1 and m = x with x ≥ 1 .

4. Summarizing the preceding facts we see that the Russian option problemwith infinite horizon reduces to solving the following optimal stopping problem:

V (x) = supτ

Ex

(e−λτXτ

)(26.1.10)

where τ is a stopping time of the diffusion process X satisfying (26.1.5)–(26.1.8)above and X0 = x under Px with x ≥ 1 given and fixed.



From (26.1.6) and (26.1.10) we see that the further away X gets from 1 theless likely that the gain will increase upon continuation. This suggests that thereexists a point b ∈ (1,∞] such that the stopping time

τb = inf t ≥ 0 : Xt ≥ b (26.1.11)

is optimal in the problem (26.1.10).

Standard arguments based on the strong Markov property (cf. Chapter III)lead to the following free-boundary problem for the unknown value function Vand the unknown point b :

LXV = λV for x ∈ [1,∞), (26.1.12)V (x) = x for x = b, (26.1.13)V ′(x) = 1 for x = b (smooth fit), (26.1.14)V ′(x) = 0 for x = 1 (normal reflection), (26.1.15)V (x) > x for x ∈ [1, b), (26.1.16)V (x) = x for x ∈ (b,∞). (26.1.17)

6. To solve the free-boundary problem (26.1.12)–(26.1.17) note that the equa-tion (26.1.12) using (26.1.8) reads as

Dx2V ′′ − rxV ′ − λV = 0 (26.1.18)

where we set D = σ2/2 . One may now recognize (26.1.18) as the Cauchy–Eulerequation. Let us thus seek a solution in the form

V (x) = xp. (26.1.19)


p2 −(1+

r

D

)p − λ

D= 0. (26.1.20)

The quadratic equation (26.1.20) has two roots:

p1,2 =

(1 + r

D

)±√(1 + rD

)2 + 4λD

2. (26.1.21)

Thus the general solution of (26.1.18) can be written as

V (x) = C1xp1 + C2x

p2 (26.1.22)

where C1 and C2 are undetermined constants. The three conditions (26.1.13)–(26.1.15) can be used to determine C1 , C2 and b (free boundary) uniquely. This


gives

C1 = − p2

p1bp2−1 − p2bp1−1, (26.1.23)

C2 =p1

p1bp2−1 − p2bp1−1, (26.1.24)

b =(

p1(p2 −1)p2(p1 −1)

)1/(p1−p2)

. (26.1.25)

Note that C1 > 0 , C2 > 0 and b > 1 . Inserting (26.1.23) and (26.1.24) into(26.1.22) we obtain

V (x) =

⎧⎨⎩1

p1bp2−1 − p2bp1−1

(p1x

p2 − p2xp1)

if x ∈ [1, b],

x if x ∈ [ b,∞)(26.1.26)

where b is given by (26.1.25). Note that V is C2 on [1, b)∪ (b,∞) but only C1

at b . Note also that V is convex and increasing on [1,∞) and that (26.1.16) issatisfied.

7. In this way we have arrived at the conclusions in the following theorem.

Theorem 26.1. The arbitrage-free price V from (26.1.10) is given explicitly by(26.1.26) above. The stopping time τb from (26.1.11) with b given by (26.1.25)above is optimal in the problem (26.1.10).

Proof. To distinguish the two functions let us denote the value function from(26.1.10) by V∗(x) for x ≥ 1 . We need to prove that V∗(x) = V (x) for all x ≥ 1where V (x) is given by (26.1.26) above.

1. The properties of V stated following (26.1.26) above show that Ito’s for-mula (page 67) can be applied to e−λtV (Xt) in its standard form (cf. Subsection3.5). This gives

e−λtV (Xt) = V (x) +∫ t

0

e−λs (LXV −λV )(Xs) I(Xs = b) ds (26.1.27)

+∫ t

0

e−λsV ′(Xs) dRs +∫ t

0

e−λsσXsV′(Xs) dBs

= V (x) +∫ t

0

e−λs(LXV −λV )(Xs)I(Xs = b) ds

+∫ t

0

e−λsσXsV′(Xs) dBs

upon using (26.1.7) and (26.1.15) to conclude that the integral with respect todRs equals zero. Setting G(x) = x we see that (LXG−λG)(x) = −(r+λ)x < 0so that together with (26.1.12) we have

(LXV −λV ) ≤ 0 (26.1.28)


everywhere on [1,∞) but b . Since Px(Xs = b) = 0 for all s and all x , we seethat (26.1.13), (26.1.16)–(26.1.17) and (26.1.27)–(26.1.28) imply that

e−λtXt ≤ e−λtV (Xt) ≤ V (x) + Mt (26.1.29)


Mt =∫ t

0

e−λsσXsV′(Xs) dBs. (26.1.30)

(Using that 0 ≤ V ′(x) ≤ 1 for all x ≥ 1 it is easily verified by standard meansthat M is a martingale.)

Let (τn)n≥1 be a localizations sequence of (bounded) stopping times for M(for example τn ≡ n will do). Then for every stopping time τ of X we haveby (26.1.29) above:

e−λ(τ∧τn)Xτ∧τn ≤ V (x) + Mτ∧τn (26.1.31)

for all n ≥ 1 . Taking the Px -expectation, using the optional sampling theorem(page 60) to conclude that ExMτ∧τn = 0 for all n , and letting n → ∞ , we findby Fatou’s lemma that

Ex

(e−λτXτ

) ≤ V (x). (26.1.32)

Taking the supremum over all stopping times τ of X we conclude that V∗(x) ≤V (x) for all x ∈ [1,∞) .

2. To prove the reverse inequality (equality) we may observe from (26.1.27)upon using (26.1.12) (and the optional sampling theorem as above) that

Ex

(e−λ(τb∧τn)V (Xτb∧τn)

)= V (x) (26.1.33)

for all n ≥ 1 . Letting n → ∞ and using that e−λτbV (Xτb) = e−λτbXτb

, we findby the dominated convergence theorem that

Ex

(e−λτbXτb

)= V (x). (26.1.34)

This shows that τb is optimal in (26.1.10). Thus V∗(x) = V (x) for all x ∈ [1,∞)and the proof is complete.

Remark 26.2. In the notation of Theorem 26.1 above set

u(x) = Exτb (26.1.35)

for x ∈ [1, b] . Standard arguments based on the strong Markov property (cf.Section 7) imply that u solves

LXu = −1 on (1, b), (26.1.36)u(b) = 0, (26.1.37)u′(1) = 0. (26.1.38)


The general solution of (26.1.1) is given by

u(x) = C1 + C2 x1+r/D +1

r+Dlog x. (26.1.39)

Using (26.1.2) and (26.1.3) we find

C1 =b1+r/D

(1 + r/D)(r + D)− 1

r+Dlog(1 +

r

D

), (26.1.40)

C2 = − 1(1 + r/D)(r + D)

. (26.1.41)

It can easily be verified using standard means (Ito’s formula and the optionalsampling theorem) that (26.1.39) with (26.1.40) and (26.1.41) give the correctexpression for (26.1.35). In particular, this also shows that τb < ∞ Px -a.s. forevery x ∈ [1,∞) , where b > 1 is given and fixed (arbitrary). Thus τb in (26.1.11)is indeed a (finite) stopping time under every Px with x ∈ [1,∞) .


1. The arbitrage-free price of the Russian option with finite horizon (cf. (26.1.1)above) is given by

V = sup0≤τ≤T

E(e−rτMτ

)(26.2.1)

where the supremum is taken over all stopping times τ of the geometric Brownianmotion S = (St)0≤t≤T solving

dSt = rSt dt + σSt dBt (S0 = s) (26.2.2)

and M = (Mt)0≤t≤T is the maximum process given by

Mt =(

max0≤u≤t

Su

)∨ m (26.2.3)

where m ≥ s > 0 are given and fixed. We recall that B = (Bt)t≥0 is a standardBrownian motion process started at zero, T > 0 is the expiration date (maturity),r > 0 is the interest rate, and σ > 0 is the volatility coefficient.

The first part of this subsection is analogous to the first part of the previoussubsection (cf. paragraphs 1–3) and we will briefly repeat all the details merelyfor completeness and ease of reference.

2. For the purpose of comparison with the infinite-horizon results from theprevious subsection we will also introduce a discounting rate λ ≥ 0 so that Mτ

in (26.2.1) is to be replaced by e−λτMτ . By change of measure as in (26.1.4)above it then follows that

V = s sup0≤τ≤T

E(e−λτXτ

)(26.2.4)


where we set

Xt =Mt

St(26.2.5)

and P is defined following (26.1.5) above so that Bt = Bt − σt is a standardBrownian motion under P for 0 ≤ t ≤ T . As in (26.1.6) above one finds that Xsolves

dXt = −rXt dt + σXt dBt + dRt (X0 = x) (26.2.6)

under P where B = −B is a standard Brownian motion, and we set

Rt =∫ t

0

I(Xs = 1)dMs

Ss(26.2.7)

for 0 ≤ t ≤ T . Recall that X is a diffusion process in [1,∞) with 1 beinginstantaneously reflecting, and the infinitesimal generator of X is given by

LX = −rx∂

∂x+

σ2

2x2 ∂2

∂x2in (1,∞), (26.2.8)

∂

∂x= 0 at 1+ .

For more details on the derivation of (26.2.4)–(26.2.8) see the text of (26.1.4)-(26.1.8) above.

3. For further reference recall that the strong solution of (26.2.2) is given by

St = s exp(σBt +

(r− σ2

2

)t)

= s exp(σBt +

(r+

σ2

2

)t)

(26.2.9)

for 0 ≤ t ≤ T where B and B are standard Brownian motions under P and Prespectively. Recall also when dealing with the process X on its own that thereis no restriction to assume that s = 1 and m = x with x ≥ 1 .

4. Summarizing the preceding facts we see that the Russian option problemwith finite horizon reduces to solving the following optimal stopping problem:


Et,x

(e−λτXt+τ

)(26.2.10)

where τ is a stopping time of the diffusion process X satisfying (26.2.5)–(26.2.8)above and Xt = x under Pt,x with (t, x) ∈ [0, T ]× [1,∞) given and fixed.

5. Standard Markovian arguments (cf. Chapter III) indicate that V from(26.2.10) solves the following free-boundary problem:


Vt + LXV = λV in C, (26.2.11)V (t, x) = x for x = b(t) or t = T , (26.2.12)Vx(t, x) = 1 for x = b(t) (smooth fit), (26.2.13)Vx(t, 1+) = 0 (normal reflection), (26.2.14)V (t, x) > x in C, (26.2.15)V (t, x) = x in D (26.2.16)

where the continuation set C and the stopping set D (as the closure of the setD below) are defined by

C = (t, x) ∈ [0, T )×[1,∞) : x < b(t), (26.2.17)D = (t, x) ∈ [0, T )×[1,∞) : x > b(t) (26.2.18)

and b : [0, T ] → R is the (unknown) optimal stopping boundary, i.e. the stoppingtime

τb = inf 0≤s≤T − t : Xt+s≥b(t+s) (26.2.19)

is optimal in the problem (26.2.10).

6. It will follow from the result of Theorem 26.3 below that the free-boundaryproblem (26.2.11)–(26.2.16) characterizes the value function V and the optimalstopping boundary b in a unique manner. Our main aim, however, is to followthe train of thought where V is first expressed in terms of b , and b itself isshown to satisfy a nonlinear integral equation. A particularly simple approach forachieving this goal in the case of the American put option has been exposed inSubsection 25.2 above and we will take it up in the present subsection as well.We will moreover see (as in the case of the American put option above) that thenonlinear equation derived for b cannot have other solutions.

7. Below we will make use of the following functions:

F (t, x) = E0,x(Xt) =∫ ∞

1

∫ m

0

(m∨x

s

)f(t, s, m) ds dm, (26.2.20)

G(t, x, y) = E0,x

(Xt I(Xt≥y)

)(26.2.21)

=∫ ∞

1

∫ m

0

(m∨x

s

)I((

m∨xs

)≥y)

f(t, s, m) ds dm

for t > 0 and x, y ≥ 1 , where (s, m) → f(t, s, m) is the probability densityfunction of (St, Mt) under P with S0 = M0 = 1 given by (see e.g. [107, p. 368]):

f(t, s, m) =2

σ3√

2πt3log(m2/s)

smexp

(− log2(m2/s)

2σ2t+

β

σlog s− β2

2t

)(26.2.22)

for 0 < s ≤ m and m ≥ 1 with β = r/σ + σ/2 , and is equal to 0 otherwise.

8. The main result of the present subsection may now be stated as follows.


Theorem 26.3. The optimal stopping boundary in the Russian option problem(26.2.10) can be characterized as the unique continuous decreasing solution b :[0, T ] → R of the nonlinear integral equation

b(t) = e−λ(T−t)F (T − t, b(t)) + (r+λ)∫ T−t

0

e−λu G(u, b(t), b(t+u)) du (26.2.23)

satisfying b(t) > 1 for all 0 < t < T . [The solution b satisfies b(T−) = 1 andthe stopping time τb from (26.2.19) is optimal in (26.2.10) (see Figure VII.2). ]

The arbitrage-free price of the Russian option (26.2.10) admits the following“early exercise premium” representation:

V (t, x) = e−λ(T−t)F (T − t, x) + (r+λ)∫ T−t

0

e−λu G(u, x, b(t+u)) du (26.2.24)

for all (t, x) ∈ [0, T ] × [1,∞) . [Further properties of V and b are exhibited inthe proof below. ]

1

α

x

τb T

Mt

St=

t Xt→

t b(t)→•

•

Figure VII.2: A computer drawing of the optimal stopping boundary bfrom Theorem 26.3. The number α is the optimal stopping point in thecase of infinite horizon (Theorem 26.1). If the discounting rate λ is zero,then α is infinite (i.e. it is never optimal to stop), while b is still finiteand looks as above.

Proof. The proof will be carried out in several steps. We begin by stating somegeneral remarks which will be freely used below without further mention.

It is easily seen that E (max 0≤t≤T Xt) < ∞ so that V (t, x) < ∞ for all(t, x) ∈ [0, T ] × [1,∞) . Recall that it is no restriction to assume that s = 1 and


m = x so that Xt = (Mt∨x)/St with S0 = M0 = 1 . We will write Xxt instead of

Xt to indicate the dependence on x when needed. Since Mt∨x = (x−Mt)++Mt

we see that V admits the following representation:


E

(e−λτ (x−Mτ )++Mτ

Sτ

)(26.2.25)

for (t, x) ∈ [0, T ]× [1,∞) . It follows immediately from (26.2.25) that

x → V (t, x) is increasing and convex on [1,∞) (26.2.26)

for each t≥0 fixed. It is also obvious from (26.2.25) that t → V (t, x) is decreasingon [0, T ] with V (T, x) = x for each x ≥ 1 fixed.

1. We show that V : [0, T ] × [1,∞) → R is continuous. For this, usingsup(f)− sup(g) ≤ sup(f − g) and (y− z)+− (x− z)+ ≤ (y−x)+ for x, y, z ∈ R ,it follows that

V (t, y) − V (t, x) ≤ (y−x) sup0≤τ≤T−t

E

(e−λτ

Sτ

)≤ y−x (26.2.27)

for 1 ≤ x < y and all t ≥ 0 , where in the second inequality we used (26.2.9) todeduce that 1/St = exp(σBt − (r+σ2/2)t) ≤ exp(σBt − (σ2/2)t) and the latteris a martingale under P . From (26.2.27) with (26.2.26) we see that x → V (t, x)is continuous uniformly over t ∈ [0, T ] . Thus to prove that V is continuous on[0, T ] × [1,∞) it is enough to show that t → V (t, x) is continuous on [0, T ] foreach x ≥ 1 given and fixed. For this, take any t1 < t2 in [0, T ] and ε > 0 ,and let τε

1 be a stopping time such that E(e−λτε1 Xx

t1+τε1) ≥ V (t1, x)− ε . Setting

τε2 = τε

1 ∧ (T − t2) we see that V (t2, x) ≥ E (e−λτε2 Xx

t2+τε2) . Hence we get

0 ≤ V (t1, x) − V (t2, x) ≤ E(e−λτε

1 Xxt1+τε

1− e−λτε

2 Xxt2+τε

2

)+ ε. (26.2.28)

Letting first t2 − t1 → 0 using τε1 − τε

2 → 0 and then ε ↓ 0 we see thatV (t1, x) − V (t2, x) → 0 by dominated convergence. This shows that t → V (t, x)is continuous on [0, T ] , and the proof of the initial claim is complete.

Denote G(x) = x for x ≥ 1 and introduce the continuation set C = (t, x) ∈ [0, T ) × [1,∞) : V (t, x) > G(x) and the stopping set D = (t, x) ∈[0, T )× [1,∞) : V (t, x) = G(x) . Since V and G are continuous, we see that Cis open (and D is closed indeed) in [0, T ) × [1,∞) . Standard arguments basedon the strong Markov property [see Corollary 2.9 (Finite horizon) with Remark2.10] show that the first hitting time τD = inf 0≤s≤T − t : (t+s, Xt+s) ∈ D is optimal in (26.2.10).

2. We show that the continuation set C just defined is given by (26.2.17)for some decreasing function b : [0, T ) → (1,∞) . It follows in particular that


the stopping set coincides with the closure D in [0, T ) × [1,∞) of the set D in(26.2.18) as claimed. To verify the initial claim, note that by Ito’s formula (page67) and (26.2.6) we have

e−λsXt+s = Xt − (r+λ)∫ s

0

e−λuXt+u du +∫ s

0

e−λu dMt+u

St+u+ Ns (26.2.29)

where Ns = σ∫ s

0 e−λuXt+udBt+u is a martingale for 0 ≤ s ≤ T−t . Let t ∈ [0, T ]and x > y ≥ 1 be given and fixed. We will first show that (t, x) ∈ C impliesthat (t, y) ∈ C . For this, let τ∗ = τ∗(t, x) denote the optimal stopping time forV (t, x) . Taking the expectation in (26.2.29) stopped at τ∗ , first under Pt,y andthen under Pt,x , and using the optional sampling theorem (page 60) to get rid ofthe martingale part, we find

V (t, y) − y ≥ Et,y

(e−λτ∗Xt+τ∗

)− y (26.2.30)

= −(r+λ) Et,y

(∫ τ∗

0

e−λuXt+u du

)+ Et,y

(∫ τ∗

0

e−λu dMt+u

St+u

)≥ −(r+λ) Et,x

(∫ τ∗

0

e−λuXt+u du

)+ Et,x

(∫ τ∗

0

e−λu dMt+u

St+u

)= Et,x

(e−λτ∗Xt+τ∗

)− x = V (t, x) − x > 0

proving the claim. To explain the second inequality in (26.2.30) note that theprocess X under Pt,z can be realized as the process Xt,z under P where weset Xt,z

t+u = (S∗u ∨ z)/Su with S∗

u = max 0≤v≤u Sv . Then note that Xt,yt+u ≤ Xt,x

t+u

and d(S∗u ∨ y) ≥ d(S∗

u ∨ x) whenever y ≤ x , and thus each of the two termson the left-hand side of the inequality is larger than the corresponding term onthe right-hand side, implying the inequality. The fact just proved establishes theexistence of a function b : [0, T ] → [1,∞] such that the continuation set C isgiven by (26.2.17) above.

Let us show that b is decreasing. For this, with x ≥ 1 and t1 < t2 in [0, T ]given and fixed, it is enough to show that (t2, x) ∈ C implies that (t1, x) ∈ C .To verify this implication, recall that t → V (t, x) is decreasing on [0, T ] , so thatwe have

V (t1, x) ≥ V (t2, x) > x (26.2.31)

proving the claim.

Let us show that b does not take the value ∞ . For this, assume that thereexists t0 ∈ (0, T ] such that b(t) = ∞ for all 0 ≤ t ≤ t0 . It implies that (0, x) ∈ Cfor any x ≥ 1 given and fixed, so that if τ∗ = τ∗(0, x) denote the optimal stoppingtime for V (0, x) , we have V (0, x) > x which by (26.2.29) is equivalent to

E0,x

(∫ τ∗

0

e−λu dMu

Su

)> (r+λ) E0,x

(∫ τ∗

0

e−λuXu du

). (26.2.32)


Recalling that Mu = S∗u ∨ x we see that

E0,x

(∫ τ∗

0

e−λu dMu

Su

)≤ E

((max

0≤u≤T(1/Su)

)((S∗

T ∨ x) − x))

(26.2.33)

≤ E

((max

0≤u≤T(1/Su)

)S∗

T I(S∗T >x)

)→ 0

as x → ∞ . Recalling that Xu = (S∗u ∨ x)/Su and noting that τ∗ > t0 we see

that

E0,x

(∫ τ∗

0

e−λuXu du

)≥ e−λt0 x E

(∫ t0

0

du

Su

)→ ∞ (26.2.34)

as x → ∞ . From (26.2.33) and (26.2.34) we see that the strict inequality in(26.2.32) is violated if x is taken large enough, thus proving that b does not takethe value ∞ on (0, T ] . To disprove the case b(0+) = ∞ , i.e. t0 = 0 above, wemay note that the gain function G(x) = x in (26.2.10) is independent of time,so that b(0+) = ∞ would also imply that b(t) = ∞ for all 0 ≤ t ≤ δ in theproblem (26.2.10) with the horizon T +δ instead of T where δ > 0 . Applyingthe same argument as above to the T +δ problem (26.2.10) we again arrive at acontradiction. We thus may conclude that b(0+) < ∞ as claimed. Yet anotherquick argument for b to be finite in the case λ > 0 can be given by noting thatb(t) < α for all t ∈ [0, T ] where α ∈ (1,∞) is the optimal stopping point in theinfinite horizon problem given explicitly by the right-hand side of (26.1.25) above.Clearly b(t) ↑ α as T → ∞ for each t ≥ 0 , where we set α = ∞ in the caseλ = 0 .

Let us show that b cannot take the value 1 on [0, T ) . This fact is equivalentto the fact that the process (St, Mt) in (26.2.1) (with r+λ instead of r ) cannotbe optimally stopped at the diagonal s = m in (0,∞) × (0,∞) . The latter factis well known for diffusions in the maximum process problems of optimal stoppingwith linear cost (see Proposition 13.1) and only minor modifications are neededto extend the argument to the present case. For this, set Zt = σBt + (r−σ2/2)tand note that the exponential case of (26.2.1) (with r+λ instead of r ) reducesto the linear case of Proposition 13.1 for the diffusion Z and c = r+λ by meansof Jensen’s inequality as follows:

E(e−(r+λ)τMτ

)= E

(exp

(max

0≤t≤τZt − cτ

))(26.2.35)

≥ exp(E(

max0≤t≤τ

Zt − cτ))

.

Denoting τn = inf t > 0 : Zt = (− 1/n, 1/n) it is easily verified (see the proofof Proposition 13.1) that

E(

max0≤t≤τn

Zt

)≥ δ

nand E (τn) ≤ κ

n2(26.2.36)


for all n ≥ 1 with some constants δ > 0 and κ > 0 . Choosing n large enough,upon recalling (26.2.35), we see that (26.2.36) shows that it is never optimal tostop at the diagonal in the case of infinite horizon. To derive the same conclusionin the finite horizon case, replace τn by σn = τn ∧ T and note by the Markovinequality and (26.2.36) that

E(

max0≤t≤τn

Zt − max0≤t≤σn

Zt

)≤ 1

nP(τn >T

)(26.2.37)

≤ E (τn)n T

≤ κ

n3 T= O(n−3)

which together with (26.2.35) and (26.2.36) shows that

E(e−(r+λ)σnMσn

)≥ exp

(E(

max0≤t≤σn

Zt − cσn

))> 1 (26.2.38)

for n large enough. From (26.2.38) we see that it is never optimal to stop at thediagonal in the case of finite horizon either, and thus b does not take the value 1on [0, T ) as claimed.

Since the stopping set equals D = (t, x) ∈ [0, T )× [1,∞) : x ≥ b(t) and bis decreasing, it is easily seen that b is right-continuous on [0, T ) . Before we passto the proof of its continuity we first turn to the key principle of optimal stoppingin problem (26.2.10).

3. We show that the smooth-fit condition (26.2.13) holds. For this, let t ∈[0, T ) be given and fixed and set x = b(t) . We know that x > 1 so that thereexists ε > 0 such that x− ε > 1 too. Since V (t, x) = G(x) and V (t, x− ε) >G(x− ε) , we have:

V (t, x) − V (t, x− ε)ε

≤ G(x) − G(x− ε)ε

= 1 (26.2.39)

so that by letting ε ↓ 0 in (26.2.39) and using that the left-hand derivativeV −

x (t, x) exists since y → V (t, y) is convex, we get V −x (t, x) ≤ 1 . To prove

the reverse inequality, let τε = τ∗ε (t, x− ε) denote the optimal stopping time for

V (t, x− ε) . We then have:

V (t, x) − V (t, x− ε)ε

(26.2.40)

≥ 1ε

E

(e−λτε

((x−Mτε)++Mτε

Sτε

− (x− ε−Mτε)++ Mτε

Sτε

))=

1ε

E

(e−λτε

Sτε

((x−Mτε)

+− (x− ε−Mτε)+))

≥ 1ε

E

(e−λτε

Sτε

((x−Mτε)

+− (x− ε−Mτε)+)I(Mτε ≤x− ε)

)= E

(e−λτε

Sτε

I(Mτε ≤x− ε))

−→ 1


as ε ↓ 0 by bounded convergence, since τε → 0 so that Mτε → 1 with 1 <x− ε and likewise Sτε → 1 . It thus follows from (26.2.40) that V −

x (t, x) ≥1 and therefore V −

x (t, x) = 1 . Since V (t, y) = G(y) for y > x , it is clearthat V +

x (t, x) = 1 . We may thus conclude that y → V (t, y) is C1 at b(t) andVx(t, b(t)) = 1 as stated in (26.2.13).

4. We show that b is continuous on [0, T ] and that b(T−) = 1 . For this,note first that since the supremum in (26.2.10) is attained at the first exit time τb

from the open set C , standard arguments based on the strong Markov property(cf. Chapter III) imply that V is C1,2 on C and satisfies (26.2.11). Suppose thatthere exists t ∈ (0, T ] such that b(t−) > b(t) and fix any x ∈ [b(t), b(t−)) . Notethat by (26.2.13) we have

V (s, x) − x =∫ b(s)

x

∫ b(s)

y

Vxx(s, z) dz dy (26.2.41)

for each s ∈ (t− δ, t) where δ > 0 with t− δ > 0 . Since Vt−rx Vx+(σ2/2)x2 Vxx

−λV = 0 in C we see that (σ2/2)x2 Vxx = −Vt + rx Vx + λV ≥ rVx in C sinceVt ≤ 0 and Vx ≥ 0 upon recalling also that x ≥ 1 and λV ≥ 0 . Hence we see thatthere exists c > 0 such that Vxx ≥ c Vx in C∩(t, x) ∈ [0, T )×[1,∞) : x≤b(0) ,so that this inequality applies in particular to the integrand in (26.2.41). In thisway we get

V (s, x) − x ≥ c

∫ b(s)

x

∫ b(s)

y

Vx(s, z) dz dy = c

∫ b(s)

x

(b(s) − V (s, y)

)dy (26.2.42)

for all s ∈ (t− δ, t) . Letting s ↑ t we find that

V (t, x) − x ≥ c

∫ b(t−)

x

(b(t−) − y

)dy =

c

2(b(t−) − x

)2> 0 (26.2.43)

which is a contradiction since (t, x) belongs to the stopping set D . This showsthat b is continuous on [0, T ] . Note also that the same argument with t = Tshows that b(T−) = 1 .

5. We show that the normal reflection condition (26.2.14) holds. For this,note first that since x → V (t, x) is increasing (and convex) on [1,∞) it followsthat Vx(t, 1+) ≥ 0 for all t ∈ [0, T ) . Suppose that there exists t ∈ [0, T ) suchthat Vx(t, 1+) > 0 . Recalling that V is C1,2 on C so that t → Vx(t, 1+) iscontinuous on [0, T ) , we see that there exists δ > 0 such that Vx(s, 1+) ≥ ε > 0for all s ∈ [t, t + δ] with t + δ < T . Setting τδ = τb ∧ (t + δ) it follows by Ito’sformula (page 67) that

Et,1

(e−λτδ V (t+τδ, Xt+τδ

))

= V (t, 1) (26.2.44)

+ Et,1

(∫ τδ

0

e−λu Vx(t+u, Xt+u) dRt+u

)


using (26.2.11) and the optional sampling theorem (page 60) since Vx is bounded.Since (e−λ(s∧τb)V (t+(s∧τb), Xt+(s∧τb)))0≤s≤T−t is a martingale under Pt,1 ,

we find that the expression on the left-hand side in (26.2.44) equals the first termon the right-hand side, and thus

Et,1

(∫ τδ

0

e−λu Vx(t+u, Xt+u) dRt+u

)= 0. (26.2.45)

On the other hand, since Vx(t+u, Xt+u)dRt+u = Vx(t+u, 1+)dRt+u by (26.2.7),and Vx(t + u, 1+) ≥ ε > 0 for all u ∈ [0, τδ] , we see that (26.2.45) implies that

Et,1

(∫ τδ

0

dRt+u

)= 0. (26.2.46)

By (26.2.6) and the optional sampling theorem (page 60) we see that (26.2.46) isequivalent to

Et,1

(Xt+τδ

)− 1 + rEt,1

(∫ τδ

0

Xt+u du

)= 0. (26.2.47)

Since Xs ≥ 1 for all s ∈ [0, T ] we see that (26.2.47) implies that τδ = 0 Pt,1 -a.s. As clearly this is impossible, we see that Vx(t, 1+) = 0 for all t ∈ [0, T ) asclaimed in (26.2.14).

6. We show that b solves the equation (26.2.23) on [0, T ] . For this, setF (t, x) = e−λtV (t, x) and note that F : [0, T ) × [1,∞) → R is a continuousfunction satisfying the following conditions:

F is C1,2 on C ∪ D, (26.2.48)Ft + LXF is locally bounded, (26.2.49)x → F (t, x) is convex, (26.2.50)t → Fx(t, b(t)±) is continuous. (26.2.51)

To verify these claims, note first that F (t, x) = e−λtG(x) = e−λtx for(t, x) ∈ D so that the second part of (26.2.48) is obvious. Similarly, since F (t, x) =e−λt V (t, x) and V is C1,2 on C , we see that the same is true for F , implyingthe first part of (26.2.48). For (26.2.49), note that (Ft + LXF )(t, x) = e−λt(Vt +LXV −λV )(t, x) = 0 for (t, x) ∈ C by means of (26.2.11), and (Ft+LXF )(t, x) =e−λt(Gt + LXG−λG)(t, x) = −(r + λ) x e−λt for (t, x) ∈ D , implying the claim.[When we say in (26.2.49) that Ft + LXF is locally bounded, we mean thatFt +LXF is bounded on K∩(C∪D) for each compact set K in [0, T )× [1,∞) .]The condition (26.2.50) follows by (26.2.26) above. Finally, recall by (26.2.13) thatx → V (t, x) is C1 at b(t) with Vx(t, b(t)) = 1 so that Fx(t, b(t)±) = e−λt imply-ing (26.2.51). Let us also note that the condition (26.2.50) can further be relaxedto the form where Fxx = F1 +F2 on C ∪D where F1 is non-negative and F2 is


continuous on [0, T ) × [1,∞) . This will be referred to below as the relaxed formof (26.2.48)–(26.2.51).

Having a continuous function F : [0, T ) × [1,∞) → R satisfying (26.2.48)–(26.2.51) one finds (cf. Subsection 3.5) that for t ∈ [0, T ) the following change-of-variable formula holds:

F (t, Xt) = F (0, X0) +∫ t

0

(Ft+LXF )(s, Xs) I(Xs = b(s)) ds (26.2.52)

+∫ t

0

Fx(s, Xs) σXsI(Xs = b(s)) dBs +∫ t

0

Fx(s, Xs) I(Xs = b(s)) dRs

+12

∫ t

0


)I(Xs = b(s)) db

s(X)

where bs(X) is the local time of X at the curve b given by

bs(X) = P- lim

ε↓012ε

∫ s

0

I(b(r)− ε < Xr < b(r)+ε) σ2X2r dr (26.2.53)

and dbs(X) refers to the integration with respect to the continuous increasing

function s → bs(X) . Note also that formula (26.2.52) remains valid if b is replaced

by any other continuous function of bounded variation c : [0, T ] → R for which(26.2.48)–(26.2.51) hold with C and D defined in the same way.

Applying (26.2.52) to e−λsV (t+s, Xt+s) under Pt,x with (t, x) ∈ [0, T ) ×[1,∞) yields

e−λsV (t+s, Xt+s) = V (t, x) (26.2.54)

+∫ s

0

e−λu(Vt+LXV −λV

)(t+u, Xt+u) du + Ms

= V (t, x) +∫ s

0

e−λu(Gt+LXG−λG

)(t+u, Xt+u)

× I(Xt+u ≥ b(t+u)) du + Ms

= V (t, x) − (r+λ)∫ s

0

e−λuXt+u I(Xt+u ≥ b(t+u)) du + Ms

upon using (26.2.11), (26.2.12)+(26.2.16), (26.2.14), (26.2.13) and Gt+LXG−λG

= −(r+λ)G , where we set Ms =∫ s

0e−λuVx(t+u, Xt+u)σXt+u dBt+u for 0 ≤ s ≤

T − t . Since 0 ≤ Vx ≤ 1 on [0, T ]× [1,∞) , it is easily verified that (Ms)0≤s≤T−t

is a martingale, so that Et,xMs = 0 for all 0 ≤ s ≤ T − t . Inserting s = T − tin (26.2.54), using that V (T, x) = G(x) = x for all x ∈ [1,∞) , and taking thePt,x -expectation in the resulting identity, we get

e−λ(T−t)Et,xXT = V (t, x) (26.2.55)

− (r+λ)∫ T−t

0

e−λuEt,x

(Xt+u I(Xt+u ≥ b(t+u))

)du


for all (t, x) ∈ [0, T ) × [1,∞) . By (26.2.20) and (26.2.21) we see that (26.2.55)is the early exercise premium representation (26.2.24). Recalling that V (t, x) =G(x) = x for x ≥ b(t) , and setting x = b(t) in (26.2.55), we see that b satisfiesthe equation (26.2.23) as claimed.

7. We show that b is the unique solution of the equation (26.2.23) in theclass of continuous decreasing functions c : [0, T ] → R satisfying c(t) > 1 forall 0 ≤ t < T . The proof of this fact will be carried out in several remainingparagraphs to the end of the main proof. Let us thus assume that a function cbelonging to the class described above solves (26.2.23), and let us show that thisc must then coincide with the optimal stopping boundary b .

For this, in view of (26.2.55), let us introduce the function

U c(t, x) = e−λ(T−t)Et,xXT (26.2.56)

+ (r+λ)∫ T−t

0

e−λuEt,x

(Xt+u I(Xt+u≥c(t+u))

)du

for (t, x) ∈ [0, T ) × [1,∞) . Using (26.2.20) and (26.2.21) as in (26.2.24) we seethat (26.2.56) reads

U c(t, x) = e−λ(T−t)F (T − t, x) + (r+λ)∫ T−t

0

e−λu G(u, x, c(t+u)) du (26.2.57)

for (t, x) ∈ [0, T )× [1,∞) . A direct inspection of the expressions in (26.2.57) using(26.2.20)–(26.2.22) shows that U c

x is continuous on [0, T )× [1,∞) .

8. In accordance with (26.2.24) define a function V c : [0, T ) × [1,∞) → R

by setting V c(t, x) = U c(t, x) for x < c(t) and V c(t, x) = G(x) for x ≥ c(t)when 0 ≤ t < T . Note that since c solves (26.2.23) we have that V c is continuouson [0, T )× [1,∞) , i.e. V c(t, x) = U c(t, x) = G(x) for x = c(t) when 0 ≤ t < T .Let C and D be defined by means of c as in (26.2.17) and (26.2.18) respectively.

Standard arguments based on the Markov property (or a direct verification)show that V c i.e. U c is C1,2 on C and that

V ct + LXV c = λV c in C, (26.2.58)

V cx (t, 1+) = 0 (26.2.59)

for all t ∈ [0, T ) . Moreover, since U cx is continuous on [0, T )× [1,∞) we see that

V cx is continuous on C . Finally, it is obvious that V c i.e. G is C1,2 on D .

9. Summarizing the preceding conclusions one can easily verify that thefunction F : [0, T ) × [1,∞) → R defined by F (t, x) = e−λtV c(t, x) satisfies(26.2.48)–(26.2.51) (in the relaxed form) so that (26.2.52) can be applied. In thisway, under Pt,x with (t, x) ∈ [0, T ) × [1,∞) given and fixed, using (26.2.59) we


get

e−λsV c(t+s, Xt+s) = V c(t, x) (26.2.60)

+∫ s

0

e−λu(V c

t +LXV c −λV c)(t+u, Xt+u)I(Xt+u = c(t+u)) du

+ M cs +

12

∫ s

0

e−λu∆xV cx (t+u, c(t+u)) dc

u(X)

where M cs =

∫ s

0 e−λuV cx (t + u, Xt+u) σXt+u I(Xt+u = c(t + u)) dBt+u and we

set ∆xV cx (v, c(v)) = V c

x (v, c(v)+) − V cx (v, c(v)−) for t ≤ v ≤ T . Moreover, it is

readily seen from the explicit expression for V cx obtained using (26.2.57) above

that (M cs )0≤s≤T−t is a martingale under Pt,x so that Et,x(M c

s ) = 0 for each0≤s≤T − t .

10. Setting s = T − t in (26.2.60) and then taking the Pt,x -expectation,using that V c(T, x) = G(x) for all x ≥ 1 and that V c satisfies (26.2.58) in C ,we get

e−λ(T−t)Et,xXT = V c(t, x) (26.2.61)

− (r+λ)∫ T−t

0

e−λuEt,x

(Xt+u I(Xt+u≥c(t+u))

)du

+12

∫ T−t

0

e−λu∆xV cx (t+u, c(t+u)) duEt,x(c

u(X))

for all (t, x) ∈ [0, T ) × [1,∞) . Comparing (26.2.61) with (26.2.56), and recallingthe definition of V c in terms of U c and G , we get∫ T−t

0

e−λu∆xV cx (t+u, c(t+u)) duEt,x(c

u(X)) (26.2.62)

= 2(U c(t, x)−G(x)

)I(x≥c(t))

for all 0 ≤ t < T and x ≥ 1 , where I(x≥ c(t)) equals 1 if x ≥ c(t) and 0 ifx < c(t) .


x → V c(t, x) is C1 at c(t) for each 0 ≤ t < T (26.2.63)

given and fixed, then it will follow that

U c(t, x) = G(x) for all x ≥ c(t) . (26.2.64)

On the other hand, if we know that (26.2.64) holds, then using the general fact

∂

∂x

(U c(t, x) − G(x)

)∣∣∣x=c(t)

= V cx (t, c(t)−) − V c

x (t, c(t)+) (26.2.65)

= −∆xV cx (t, c(t))


for all 0 ≤ t < T , we see that (26.2.63) holds too (since U cx is continuous). The

equivalence of (26.2.63) and (26.2.64) suggests that instead of dealing with theequation (26.2.62) in order to derive (26.2.62) above we may rather concentrateon establishing (26.2.63) directly.

12. To derive (26.2.64) first note that standard arguments based on theMarkov property (or a direct verification) show that U c is C1,2 on D and that

U ct + LXU c − λU c = −(r+λ)G in D . (26.2.66)

Since the function F : [0, T ) × [1,∞) → R defined by F (t, x) = e−λtU c(t, x)is continuous and satisfies (26.2.48)–(26.2.51) (in the relaxed form), we see that(26.2.52) can be applied just like in (26.2.60) with U c instead of V c , and thisyields

e−λsU c(t+s, Xt+s) = U c(t, x) (26.2.67)

− (r+λ)∫ s

0

e−λuXt+u I(Xt+u ≥ c(t+u)) du + M cs

upon using (26.2.58)–(26.2.59) and (26.2.66) as well as that ∆xU cx(t + u, c(t +

u)) = 0 for 0 ≤ u ≤ s since U cx is continuous. In (26.2.67) we have M c

s =∫ s

0 e−λu U cx(t + u, Xt+u) σXt+u I(Xt+u = c(t + u)) dBt+u and (M c

s )0≤s≤T−t is amartingale under Pt,x .

Next note that Ito’s formula (page 67) implies

e−λsG(Xt+s) = G(x) − (r+λ)∫ s

0

e−λuXt+u du + Ms (26.2.68)

+∫ s

0

e−λu dRt+u

upon using that Gt +LXG−rG = −(r+λ)G as well as that Gx(t+u, Xt+u) = 1for 0 ≤ u ≤ s . In (26.2.68) we have Ms =

∫ s

0 e−λuσXt+udBt+u and (Ms)0≤s≤T−t

is a martingale under Pt,x .

For x ≥ c(t) consider the stopping time

σc = inf 0 ≤ s ≤ T − t : Xt+s ≤ c(t+s). (26.2.69)

Then using that U c(t, c(t)) = G(c(t)) for all 0 ≤ t < T since c solves (26.2.23),and that U c(T, x) = G(x) for all x ≥ 1 by (26.2.56), we see that U c(t +σc, Xt+σc) = G(Xt+σc) . Hence from (26.2.67) and (26.2.68) using the optional


sampling theorem (page 60) we find

U c(t, x) = Et,x

(e−λσcU c(t+σc, Xt+σc)

)(26.2.70)

+ (r+λ) Et,x

(∫ σc

0

e−λuXt+u I(Xt+u≥c(t+u)) du

)= Et,x

(e−rσcG(Xt+σc)

)+ (r + λ) Et,x

(∫ σc

0


)= G(x) − (r+λ) Et,x

(∫ σc

0

e−λuXt+u du

)+ (r + λ)Et,x

(∫ σc

0


)= G(x)

since Xt+u ≥ c(t + u) > 1 for all 0 ≤ u ≤ σc . This establishes (26.2.64) andthus (26.2.63) holds too.


e−λsU c(t+s, Xt+s) + (r+λ)∫ s

0

e−λuXt+u I(Xt+u ≥ c(t+u)) du (26.2.71)

is a martingale under Pt,x for 0 ≤ s ≤ T − t . This verification moreover showsthat the martingale property of (26.2.71) does not require that c is increasing butonly measurable. Taken together with the rest of the proof below this shows thatthe claim of uniqueness for the equation (26.2.23) holds in the class of continuousfunctions c : [0, T ] → R such that c(t)>1 for all 0 < t < T .

13. Consider the stopping time

τc = inf 0 ≤ s ≤ T − t : Xt+s ≥ c(t+s). (26.2.72)

Note that (26.2.60) using (26.2.58) and (26.2.63) reads

e−λsV c(t+s, Xt+s) = V c(t, x) (26.2.73)

− (r+λ)∫ s

0

e−λuXt+u I(Xt+u ≥ c(t+u)) du + M cs

where (M cs )0≤s≤T−t is a martingale under Pt,x . Thus Et,xM c

τc= 0 , so that after

inserting τc in place of s in (26.2.73), it follows upon taking the Pt,x -expectationthat

V c(t, x) = Et,x

(e−λτcXt+τc

)(26.2.74)


for all (t, x) ∈ [0, T )× [1,∞) where we use that V c(t, x) = G(x) = x for x ≥ c(t)or t = T . Comparing (26.2.74) with (26.2.10) we see that

V c(t, x) ≤ V (t, x) (26.2.75)

for all (t, x) ∈ [0, T )× [1,∞) .

14. Let us now show that b ≥ c on [0, T ] . For this, recall that by the samearguments as for V c we also have

e−λsV (t+s, Xt+s) = V (t, x) (26.2.76)

− (r+λ)∫ s

0

e−λuXt+u I(Xt+u ≥ b(t+u)) du + M bs

where (M bs )0≤s≤T−t is a martingale under Pt,x . Fix (t, x) ∈ [0, T )× [1,∞) such

that x > b(t) ∨ c(t) and consider the stopping time

σb = inf 0 ≤ s ≤ T − t : Xt+s ≤ b(t+s). (26.2.77)

Inserting σb in place of s in (26.2.73) and (26.2.76) and taking the Pt,x -expec-tation, we get

Et,x

(e−λσbV c(t+σb, Xt+σb

))

(26.2.78)

= x − (r + λ) Et,x

(∫ σb

0

e−λuXt+u I(Xt+u≥c(t + u)) du

),

Et,x

(e−λσbV (t+σb, Xt+σb

))

= x − (r+λ) Et,x

(∫ σb

0

e−λuXt+u du

). (26.2.79)


Et,x

(∫ σb

0

e−λuXt+u I(Xt+u ≥ c(t+u)) du

)(26.2.80)

≥ Et,x

(∫ σb

0

e−λuXt+u du

)from where it follows by the continuity of c and b , using Xt+u > 0 , that b(t) ≥c(t) for all t ∈ [0, T ] .

15. Finally, let us show that c must be equal to b . For this, assume thatthere is t ∈ (0, T ) such that b(t) > c(t) , and pick x ∈ (c(t), b(t)) . Under Pt,x con-sider the stopping time τb from (26.2.19). Inserting τb in place of s in (26.2.73)and (26.2.76) and taking the Pt,x -expectation, we get

Et,x

(e−λτbXt+τb

)= V c(t, x) (26.2.81)

− (r+λ) Et,x

(∫ τb

0


),

Et,x

(e−λτbXt+τb

)= V (t, x). (26.2.82)



Et,x

(∫ τb

0


)≤ 0 (26.2.83)

from where it follows by the continuity of c and b using Xt+u > 0 that such apoint x cannot exist. Thus c must be equal to b , and the proof is complete.

Notes. According to theory of modern finance (see e.g. [197]) the arbitrage-free price of the Russian option (first introduced and studied in [185] and [186])is given by (26.2.1) above where M denotes the maximum of the stock price S .This option is characterized by “reduced regret” because its owner is paid themaximum stock price up to the time of exercise and hence feels less remorse fornot having exercised the option earlier.

In the case of infinite horizon T , and when Mτ in (26.2.1) is replaced bye−λτMτ , the problem was solved in [185] and [186]. The original derivation [185]was two-dimensional (see Section 13 for a general principle in this context) and thesubsequent derivation [186] reduced the problem to one dimension using a changeof measure. The latter methodology was also adopted in the present section.

Note that the infinite horizon formulation requires the discounting rate λ > 0to be present (i.e. non-zero), since otherwise the option price would be infinite.Clearly, such a discounting rate is not needed (i.e. can be taken zero) when thehorizon T is finite, so that the most attractive feature of the option — no regret— remains fully preserved.

The fact that the Russian option problem becomes one-dimensional (after achange of measure is applied) sets the mathematical problem on an equal footingwith the American option problem (put or call) with finite horizon. The latterproblem, on the other hand, has been studied since the 1960’s, and for moredetails and references we refer to Section 25 above. The main aim of the presentsection is to extend these results to the Russian option with finite horizon.

We showed above (following [165]) that the optimal stopping boundary for theRussian option with finite horizon can be characterized as the unique solution of anonlinear integral equation arising from the early exercise premium representation(an explicit formula for the arbitrage-free price in terms of the optimal stoppingboundary having a clear economic interpretation). The results obtained stand ina complete parallel with the best known results on the American put option withfinite horizon (cf. Subsection 25.2 above). The key argument in the proof reliesupon a local time-space formula (cf. Subsection 3.5). Papers [57] and [47] provideuseful additions to the main results of the present section.

27. The Asian option

Unlike in the case of the American option (Section 25) and the Russian option(Section 26) it turns out that the infinite horizon formulation of the Asian option

Section 27. The Asian option 417

problem considered below leads to a trivial solution: the value function is constantand it is never optimal to stop (see the text following (27.1.31) below). This ishardly a rule for Asian options as their infinite horizon formulations, contrary towhat one could expect generally, are more difficult than finite horizon ones. Thereason for this unexpected twist is twofold. Firstly, the integral functional is morecomplicated than the maximum functional after the state variable is added to makeit Markovian (recall our discussions in Chapter III). Secondly, the existence of afinite horizon (i.e. the end of time) enables one to use backward induction upontaking the horizon as an initial point. Nonlinear integral equations (derived in thepresent chapter) may be viewed as a continuous-time analogue of the method ofbackward induction considered in Chapter I above. The fact that these equationshave unique solutions constitutes the key element which makes finite horizonsmore amenable.


1. According to financial theory (see e.g. [197]) the arbitrage-free price of the earlyexercise Asian call option with floating strike is given by

V = sup0<τ≤T

E(e−rτ

(Sτ − 1

τ Iτ

)+) (27.1.1)

where τ is a stopping time of the geometric Brownian motion S = (St)0≤t≤T

solvingdSt = rSt dt + σSt dBt (S0 = s) (27.1.2)

and I = (It)0≤t≤T is the integral process given by

It = a +∫ t

0

Ss ds (27.1.3)

where s > 0 and a ≥ 0 are given and fixed. We recall that B = (Bt)t≥0

is a standard Brownian motion started at zero, T > 0 is the expiration date(maturity), r > 0 is the interest rate, and σ > 0 is the volatility coefficient.

By change of measure (cf. Subsection 5.3 above) we may write

V = sup0<τ≤T

E

(e−rτSτ

(1 − 1

τXτ

)+)

= s sup0<τ≤T

E

((1 − 1

τXτ

)+)

(27.1.4)

where we setXt =

It

St(27.1.5)

and P is a probability measure defined by dP = exp(σBT − (σ2/2)T ) dP so thatBt = Bt − σt is a standard Brownian motion under P for 0 ≤ t ≤ T . By Ito’sformula (page 67) one finds that

dXt = (1 − rXt) dt + σXt dBt (X0 = x) (27.1.6)


under P where B = −B is a standard Brownian motion and x = a/s . Theinfinitesimal generator of X is therefore given by

LX = (1 − rx)∂

∂x+

σ2

2x2 ∂2

∂x2. (27.1.7)

For further reference recall that the strong solution of (27.1.2) is given by

St = s exp(

σBt +(r − σ2

2

)t

)= s exp

(σBt +

(r +

σ2

2

)t

)(27.1.8)

for 0 ≤ t ≤ T where B and B are standard Brownian motions under P andP respectively. When dealing with the process X on its own, however, note thatthere is no restriction to assume that s = 1 and a = x with x ≥ 0 .

Summarizing the preceding facts we see that the early exercise Asian callproblem reduces to solving the following optimal stopping problem:

V (t, x) = sup0<τ≤T−t

Et,x

((1 − 1

t + τXt+τ

)+)(27.1.9)

where τ is a stopping time of the diffusion process X solving (27.1.6) above andXt = x under Pt,x with (t, x) ∈ [0, T ]× [0,∞) given and fixed.

Standard Markovian arguments (cf. Chapter III) indicate that V from(27.1.9) solves the following free-boundary problem:

Vt + LXV = 0 in C, (27.1.10)

V (t, x) =(1 − x

t

)+for x = b(t) or t = T, (27.1.11)

Vx(t, x) = −1t

for x = b(t) (smooth fit), (27.1.12)

V (t, x) >(1 − x

t

)+in C, (27.1.13)

V (t, x) =(1 − x

t

)+in D (27.1.14)

where the continuation set C and the stopping set D (as the closure of the setD below) are defined by

C = (t, x) ∈ [0, T )×[0,∞) : x > b(t) , (27.1.15)

D = (t, x) ∈ [0, T )×[0,∞) : x < b(t) , (27.1.16)

and b : [0, T ] → R is the (unknown) optimal stopping boundary, i.e. the stoppingtime

τb = inf 0 ≤ s ≤ T − t : Xt+s ≤ b(t+s) (27.1.17)


is optimal in (27.1.9) (i.e. the supremum is attained at this stopping time). Itfollows from the result of Theorem 27.1 below that the free-boundary problem(27.1.10)–(27.1.14) characterizes the value function V and the optimal stoppingboundary b in a unique manner (proving also the existence of the latter).

2. In the sequel we make use of the following functions:

F (t, x) = E0,x

(1 − Xt

T

)+

=∫ ∞

0

∫ ∞

0

(1 − x + a

Ts

)+f(t, s, a) ds da, (27.1.18)

G(t, x, y) = E0,x

(Xt I(Xt ≤ y)

)(27.1.19)

=∫ ∞

0

∫ ∞

0

(x + a

s

)I(x + a

s≤ y

)f(t, s, a) ds da,

H(t, x, y) = P0,x(Xt ≤ y) =∫ ∞

0

∫ ∞

0

I(x + a

s≤ y

)f(t, s, a) ds da, (27.1.20)

for t > 0 and x, y ≥ 0 , where (s, a) → f(t, s, a) is the probability densityfunction of (St, It) under P with S0 = 1 and I0 = 0 given by

f(t, s, a) =2√

2π3/2σ3

sr/σ2

a2√

texp

(2π2

σ2t− (r + σ2/2)2

2σ2t − 2

σ2a(1 + s)

)(27.1.21)

×∫ ∞

0

exp(− 2z2

σ2t− 4

√s

σ2acosh(z)

)sinh(z) sin

(4πz

σ2t

)dz

for s > 0 and a > 0 . (For a derivation of the right-hand side in (27.1.21) see theAppendix below.)

The main result of the present section may now be stated as follows.

Theorem 27.1. The optimal stopping boundary in the Asian call problem (27.1.9)can be characterized as the unique continuous increasing solution b : [0, T ] → R

of the nonlinear integral equation

1 − b(t)t

= F (T − t, b(t)) (27.1.22)

−∫ T−t

0

1t+u

((1

t+u+ r

)cG(u, b(t), b(t+u))

− H(u, b(t), b(t+u)))

du

satisfying 0 < b(t) < t/(1+ rt) for all 0 < t < T . The solution b satisfiesb(0+) = 0 and b(T−) = T/(1+rT ) , and the stopping time τb from (27.1.17) isoptimal in (27.1.9).


The arbitrage-free price of the Asian call option (27.1.9) admits the following“early exercise premium” representation:

V (t, x) = F (T − t, x) −∫ T−t

0

1t + u

((1

t+u+ r

)G(u, x, b(t+u)) (27.1.23)

− H(u, x, b(t+u)))

du

for all (t, x) ∈ [0, T ] × [0,∞) . [Further properties of V and b are exhibited inthe proof below.]

Proof. The proof will be carried out in several steps. We begin by stating somegeneral remarks which will be freely used below without further mention.

1. The reason that we take the supremum in (27.1.1) and (27.1.9) overτ > 0 is that the ratio 1/(t+ τ) is not well defined for τ = 0 when t = 0 .Note however in (27.1.1) that Iτ/τ → ∞ as τ ↓ 0 when I0 = a > 0 and thatIτ/τ → s as τ ↓ 0 when I0 = a = 0 . Similarly, note in (27.1.9) that Xτ/τ → ∞as τ ↓ 0 when X0 = x > 0 and Xτ/τ → 1 as τ ↓ 0 when X0 = x = 0 . Thus inboth cases the gain process (the integrand in (27.1.1) and (27.1.9)) tends to 0 asτ ↓ 0 . This shows that in either (27.1.1) or (27.1.9) it is never optimal to stop att = 0 . To avoid similar (purely technical) complications in the proof to follow wewill equivalently consider V (t, x) only for t > 0 with the supremum taken overτ ≥ 0 . The case of t = 0 will become evident (by continuity) at the end of theproof.

2. Recall that it is no restriction to assume that s = 1 and a = x sothat Xt = (x + It)/St with I0 = 0 and S0 = 1 . We will write Xx

t instead ofXt to indicate the dependence on x when needed. It follows that V admits thefollowing representation:


E

(1 − x + Iτ

(t + τ)Sτ

)+

(27.1.24)

for (t, x) ∈ (0, T ]× [0,∞) . From (27.1.24) we immediately see that

x → V (t, x) is decreasing and convex on [0,∞) (27.1.25)

for each t > 0 fixed.

3. We show that V : (0, T ] × [0,∞) → R is continuous. For this, usingsup f − sup g ≤ sup(f − g) and (z − x)+ − (z − y)+ ≤ (y − x)+ for x, y, z ∈ R ,


we get

V (t, x) − V (t, y) (27.1.26)

≤ sup0≤τ≤T−t

(E

(1 − x+Iτ

(t+τ)Sτ

)+

− E

(1 − y+Iτ

(t+τ)Sτ

)+)≤ (y − x) sup

0≤τ≤T−tE

(1

(t + τ)Sτ

)≤ 1

t(y − x)

for 0 ≤ x ≤ y and t > 0 , where in the last inequality we used (27.1.8) to deducethat 1/St = exp(σBt − (r + σ2/2)t) ≤ exp(σBt − (σ2/2)t) and the latter is amartingale under P . From (27.1.26) with (27.1.25) we see that x → V (t, x) iscontinuous at x0 uniformly over t ∈ [t0−δ, t0+δ] for some δ > 0 (small enough)whenever (t0, x0) ∈ (0, T ] × [0,∞) is given and fixed. Thus to prove that V iscontinuous on (0, T ]× [0,∞) it is enough to show that t → V (t, x) is continuouson (0, T ] for each x ≥ 0 given and fixed. For this, take any t1 < t2 in (0, T ] andε > 0 , and let τε

1 be a stopping time such that E ((1 − (Xxt1+τε

1)/(t1 + τε

1 ))+) ≥V (t1, x)−ε . Setting τε

2 = τε1∧(T−t2) we see that V (t2, x) ≥ E((1−(Xt2+τε

2)/(t2+

τε2 ))+) . Hence we get

V (t1, x) − V (t2, x) (27.1.27)

≤ E

((1 − Xx

t1+τε1

t1+τε1

)+)− E

((1 − Xx

t2+τε2

t2+τε2

)+)+ ε

≤ E

((Xx

t2+τε2

t2+τε2

− Xxt1+τε

1

t1+τε1

)+)+ ε.

Letting first t2 − t1 → 0 using τε1 − τε

2 → 0 and then ε ↓ 0 we see thatlim sup t2−t1→0(V (t1, x) − V (t2, x)) ≤ 0 by dominated convergence. On the otherhand, let τε

2 be a stopping time such that E((1 − (Xxt2+τε

2)/(t2 + τε

2 ))+) ≥V (t2, x) − ε . Then we have

V (t1, x) − V (t2, x) (27.1.28)

≥ E

((1 − Xx

t1+τε2

t1+τε2

)+)− E

((1 − Xx

t2+τε2

t2+τε2

)+)− ε.

Letting first t2 − t1 → 0 and then ε ↓ 0 we see that lim inf t2−t1→0(V (t1, x) −V (t2, x)) ≥ 0 . Combining the two inequalities we find that t → V (t, x) is contin-uous on (0, T ] . This completes the proof of the initial claim.

4. Denote the gain function by G(t, x) = (1 − x/t)+ for (t, x) ∈ (0, T ] ×[0,∞) and introduce the continuation set C = (t, x) ∈ (0, T )× [0,∞) : V (t, x) >G(t, x) and the stopping set D = (t, x) ∈ (0, T ) × [0,∞) : V (t, x) = G(t, x) .Since V and G are continuous, we see that C is open (and D is closed indeed)


in (0, T )× [0,∞) . Standard arguments based on the strong Markov property [seeCorollary 2.9 (Finite horizon) with Remark 2.10] show that the first hitting timeτD = inf 0 ≤ s ≤ T − t : (t + s, Xt+s) ∈ D is optimal in (27.1.9) as well as thatV is C1,2 on C and satisfies (27.1.10). In order to determine the structure ofthe optimal stopping time τD (i.e. the shape of the sets C and D ) we will firstexamine basic properties of the diffusion process X solving (27.1.6) under P .

5. The state space of X equals [0,∞) and it is clear from the representa-tion (27.1.5) with (27.1.8) that 0 is an entrance boundary point. The drift of Xis given by b(x) = 1−rx and the diffusion coefficient of X is given by σ(x) = σxfor x ≥ 0 . Hence we see that b(x) is greater/less than 0 if and only if x isless/greater than 1/r . This shows that there is a permanent push (drift) of Xtowards the constant level 1/r (when X is above 1/r the push of X is down-wards and when X is below 1/r the push of X is upwards). The scale functionof X is given by

S(x) =∫ x

1

y2r/σ2e2/σ2y dy (27.1.29)

for x > 0 , and the speed measure of X is given by

m(dx) = (2/σ2)x−2(1+r/σ2) e−2/σ2x dx (27.1.30)

on the Borel σ -algebra of (0,∞) . Since S(0) = −∞ and S(∞) = +∞ we seethat X is recurrent. Moreover, since

∫∞0 m(dx)(2/σ2)−2r/σ2

Γ(1+2r/σ2) is finitewe find that X has an invariant probability density function given by

f(x) =(2/σ2)1+2r/σ2

Γ(1+2r/σ2)1

x2(1+r/σ2)e−2/σ2x (27.1.31)

for x > 0 . In particular, it follows that Xt/t → 0 P -a.s. as t → ∞ . This fact hasan important consequence for the optimal stopping problem (27.1.9): If the horizonT is infinite, then it is never optimal to stop. Indeed, in this case letting τ ≡ t andpassing to the limit for t → ∞ we see that V ≡ 1 on (0,∞)× [0,∞) . This showsthat the infinite horizon formulation of the problem (27.1.9) provides no usefulinformation to the finite horizon formulation (unlike in the cases of American andRussian options above). To examine the latter beyond the trivial fact that allpoints (t, x) with x ≥ t belong to C (which is easily seen by considering thehitting times τε = inf 0 ≤ s ≤ T − t : Xt+s ≤ (t + s) − ε and noting thatPt,x(0 < τε < T − t) > 0 if x ≥ t with 0 < t < T ) we will examine the gainprocess in the problem (27.1.9) using stochastic calculus as follows.

6. Setting α(t) = t for 0 ≤ t ≤ T to denote the diagonal in the state spaceand applying the local time-space formula (cf. Subsection 3.5) under Pt,x when


(t, x) ∈ (0, T )× [0,∞) is given and fixed, we get

G(t+s, Xt+s) = G(t, x) +∫ s

0

Gt(t + u, Xt+u) du (27.1.32)

+∫ s

0

Gx(t+u, Xt+u) dXt+u +12

∫ s

0

Gxx(t+u, Xt+u) d〈X, X〉t+u

+12

∫ s

0

(Gx(t + u, α(t+u)+)− Gx(t + u, α(t+u)−)

)dα

t+u(X)

= G(t, x) +∫ s

0

(Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < α(t+u)

)du

− σ

∫ s

0

Xt+u

t + uI(Xt+u < α(t+u)

)dBu +

12

∫ s

0

dαt+u(X)t + u

where αt+u(X) is the local time of X on the curve α given by

αt+u(X) (27.1.33)

= P- limε↓0

12ε

∫ u

0

I(α(t+v)− ε < Xt+v < α(t+v)+ε

)d〈X, X〉t+v

= P- limε↓0

12ε

∫ u

0

I(α(t+v)− ε < Xt+v < α(t+v)+ε

) σ2

2X2

t+v dv

and dαt+u(X) refers to the integration with respect to the continuous increasing

function u → αt+u(X) . From (27.1.32) we respectively read

G(t + s, Xt+s) = G(t, x) + As + Ms + Ls (27.1.34)

where A and L are processes of bounded variation ( L is increasing ) and M is acontinuous (local) martingale. We note moreover that s → Ls is strictly increasingonly when Xs = α(s) for 0 ≤ s ≤ T − t i.e. when X visits α . On the otherhand, when X is below α then the integrand a(t+u, Xt+u) of As may be eitherpositive or negative. To determine both sets exactly we need to examine the signof the expression a(t, x) = x/t2 − (1− rx)/t . It follows that a(t, x) is larger/lessthan 0 if and only if x is larger/less than γ(t) where γ(t) = t/(1+ rt) for0 ≤ t ≤ T . By considering the exit times from small balls in (0, T )× [0,∞) withcentre at (t, x) and making use of (27.1.32) with the optional sampling theorem(page 60) to get rid of the martingale part, upon observing that γ(t) < α(t) forall 0 < t ≤ T so that the local time part is zero, we see that all points (t, x) lyingabove the curve γ (i.e. x > γ(t) for 0 < t < T ) belong to the continuation setC . Exactly the same arguments (based on the fact that the favourable sets aboveγ and on α are far away from X ) show that for each x < γ(T ) = T/(1+rT )given and fixed, all points (t, x) belong to the stopping set D when t is closeto T . Moreover, recalling (27.1.25) and the fact that V (t, x) ≥ G(t, x) for allx ≥ 0 with t ∈ (0, T ) fixed, we see that for each t ∈ (0, T ) there is a point


b(t) ∈ [0, γ(t)] such that V (t, x) > G(t, x) for x > b(t) and V (t, x) = G(t, x)for x ∈ [0, b(t)] . Combining it with the previous conclusion on D we find thatb(T−) = γ(T ) = T/(1+ rT ) . (Yet another argument for this identity will begiven below. Note that this identity is different from the identity b(T−) = Tused in [89, p. 1126].) This establishes the existence of the nontrivial (nonzero)optimal stopping boundary b on a left-neighbourhood of T . We will now showthat b extends (continuously and decreasingly) from the initial neighbourhood ofT backward in time as long as it visits 0 at some time t0 ∈ [0, T ) , and later inthe second part of the proof below we will deduce that this t0 is equal to 0 . Thekey argument in the proof is provided by the following inequality. Notice that thisinequality is not obvious a priori (unlike in the cases of American and Russianoptions above) since t → G(t, x) is increasing and the supremum in (27.1.9) istaken over a smaller class of stopping times τ ∈ [0, T − t] when t is larger.

7. We show that the inequality

Vt(t, x) ≤ Gt(t, x) (27.1.35)

is satisfied for all (t, x) ∈ C . (It may be noted from (27.1.10) that Vt = −(1 −rx)Vx − (σ2/2)x2 Vxx ≤ (1 − rx)/t since Vx ≥ −1/t and Vxx ≥ 0 by (27.1.25),so that Vt ≤ Gt holds above γ because (1 − rx)/t ≤ x/t2 if and only if x ≥t/(1+rt) . Hence the main issue is to show that (27.1.35) holds below γ and aboveb . Any analytic proof of this fact seems difficult and we resort to probabilisticarguments.)

To prove (27.1.35) fix 0 < t < t + h < T and x ≥ 0 so that x ≤ γ(t) .Let τ = τS(t + h, x) be the optimal stopping time for V (t + h, x) . Since τ ∈[0, T − t−h] ⊆ [0, T − t] we see that V (t, x) ≥ Et,x((1 − Xt+τ/(t + τ))+) so thatusing the inequality stated prior to (27.1.26) above (and the convenient refinementby an indicator function), we get

V (t + h, x) − V (t, x) −(G(t + h, x) − G(t, x)

)(27.1.36)

≤ E

((1 − x + Iτ

(t+h+τ)Sτ

)+)− E

((1 − x + Iτ

(t+τ)Sτ

)+)−(

x

t− x

t+h

)

≤ E

((x + Iτ

(t+τ)Sτ− x + Iτ

(t+h+τ)Sτ

)I

(x + Iτ

(t+h+τ)Sτ≤ 1

))− xh

t (t + h)

= E

(x + Iτ

Sτ

(1

t + τ− 1

t + h + τ

)I

(x + Iτ

(t + h + τ)Sτ≤ 1

))− xh

t (t + h)

= E

(x + Iτ

(t + h + τ)Sτ

h

t + τI

(x + Iτ

(t + h + τ)Sτ≤ 1

))− xh

t (t + h)

≤ h

tE

(x + Iτ

(t + h + τ)SτI

(x + Iτ

(t + h + τ)Sτ≤ 1

))− xh

t (t + h)≤ 0


where the final inequality follows from the fact that with Z := (x + Iτ )/((t +h + τ)Sτ ) we have V (t + h, x) = E((1−Z)+) = E((1−Z) I(Z ≤ 1)) = P(Z ≤1) − E (Z I(Z ≤ 1)) ≥ G(t + h, x) = 1 − x/(t + h) so that E (Z I(Z ≤ 1)) ≤P(Z ≤ 1) − 1 + x/(t + h) ≤ x/(t + h) as claimed. Dividing the initial expressionin (27.1.36) by h and letting h ↓ 0 we obtain (27.1.35) for all (t, x) ∈ C suchthat x ≤ γ(t) . Since Vt ≤ Gt above γ (as stated following (27.1.35) above) thiscompletes the proof of (27.1.35).

8. We show that t → b(t) is increasing on (0, T ) . This is an immediateconsequence of (27.1.36). Indeed, if (t1, x) belongs to C and t0 from (0, T )satisfies t0 < t1 , then by (27.1.36) we have that V (t0, x) − G(t0, x) ≥ V (t1, x) −G(t1, x) > 0 so that (t0, x) must belong to C . It follows that b cannot be strictlydecreasing thus proving the claim.

9. We show that the smooth-fit condition (27.1.12) holds, i.e. that x →V (t, x) is C1 at b(t) . For this, fix a point (t, x) ∈ (0, T ) × (0,∞) lying at theboundary so that x = b(t) . Then x ≤ γ(t) < α(t) and for all ε > 0 such thatx + ε < α(t) we have

V (t, x + ε) − V (t, x)ε

≥ G(t, x + ε) − G(t, x)ε

= −1t. (27.1.37)

Letting ε ↓ 0 and using that the limit on the left-hand side exists (since x →V (t, x) is convex), we get the inequality

∂+V

∂x(t, x) ≥ ∂G

∂x(t, x) = −1

t. (27.1.38)

To prove the converse inequality, fix ε > 0 such that x + ε < α(t) , and considerthe stopping times τε = τS(t, x + ε) being optimal for V (t, x + ε) . Then we have

V (t, x+ε) − V (t, x)ε

(27.1.39)

≤ 1ε

(E

[(1 − x+ε+Iτε

(t+τε)Sτε

)+−(

1 − x+Iτε

(t+τε)Sτε

)+ ])

≤ 1ε

E

(x + Iτε

(t + τε)Sτε

− x + ε + Iτε

(t + τε)Sτε

)= − E

(1

(t + τε)Sτε

).

Since each point x in (0,∞) is regular for X , and the boundary b is increasing,it follows that τε ↓ 0 P -a.s. as ε ↓ 0 . Letting ε ↓ 0 in (27.1.39) we get

∂+V

∂x(t, x) ≤ −1

t(27.1.40)

by dominated convergence. It follows from (27.1.38) and (27.1.40) that(∂+V/∂x)(t, x) = −1/t implying the claim.


10. We show that b is continuous. Note that the same proof also showsthat b(T−) = T/(1 + rT ) as already established above by a different method.

Let us first show that b is right-continuous. For this, fix t ∈ (0, T ) andconsider a sequence tn ↓ t as n → ∞ . Since b is increasing, the right-hand limitb(t+) exists. Because (tn, b(tn)) ∈ D for all n ≥ 1 , and D is closed, it followsthat (t, b(t+)) ∈ D . Hence by (27.1.16) we see b(t+) ≤ b(t) . Since the reverseinequality follows obviously from the fact that b is increasing, this completes theproof of the first claim.

Let us next show that b is left-continuous. Suppose that there exists t ∈(0, T ) such that b(t−) < b(t) . Fix a point x in (b(t−), b(t)] and note by (27.1.12)that for s < t we have

V (s, x) − G(s, x) =∫ x

b(s)

∫ y

b(s)

(Vxx(s, z) − Gxx(s, z)

)dz dy (27.1.41)

upon recalling that V is C1,2 on C . Note that Gxx = 0 below α so that ifVxx ≥ c on R = (u, y) ∈ C : s ≤ u < t and b(u) < y ≤ x for some c > 0 (forall s < t close enough to t and some x > b(t−) close enough to b(t−) ) then byletting s ↑ t in (27.1.41) we get

V (t, x) − G(t, x) ≥ c(x − b(t))2

2> 0 (27.1.42)

contradicting the fact that (t, x) belongs to D and thus is an optimal stoppingpoint. Hence the proof reduces to showing that Vxx ≥ c on small enough R forsome c > 0 .

To derive the latter fact we may first note from (27.1.10) upon using (27.1.35)that Vxx = (2/(σ2x2))(−Vt − (1 − rx)Vx) ≥ (2/(σ2x2))(−x/t2 − (1 − rx)Vx) .Suppose now that for each δ > 0 there is s < t close enough to t and there isx > b(t−) close enough to b(t−) such that Vx(u, y) ≤ −1/u+δ for all (u, y) ∈ R(where we recall that −1/u = Gx(u, y) for all (u, y) ∈ R ). Then from the previousinequality we find that Vxx(u, y) ≥ (2/(σ2y2))(−y/u2 + (1 − ry)(1/u − δ)) =(2/(σ2y2))((u − y(1 + ru))/u2 − δ(1 − ru)) ≥ c > 0 for δ > 0 small enough sincey < u/(1 + ru) = γ(u) and y < 1/r for all (u, y) ∈ R . Hence the proof reducesto showing that Vx(u, y) ≤ −1/u + δ for all (u, y) ∈ R with R small enoughwhen δ > 0 is given and fixed.

To derive the latter inequality we can make use of the estimate (27.1.39) toconclude that

V (u, y + ε) − V (u, y)ε

≤ − E

(1

(u + σε)Mσε

)(27.1.43)

where σε = inf 0 ≤ v ≤ T − u : Xy+εu+v = b(u) and Mt = sup0≤s≤t Ss . A

simple comparison argument (based on the fact that b is increasing) shows that


the supremum over all (u, y) ∈ R on the right-hand side of (27.1.43) is attainedat (s, x + ε) . Letting ε ↓ 0 in (27.1.43) we thus get

Vx(u, y) ≤ − E

(1

(u + σ)Mσ

)(27.1.44)

for all (u, y) ∈ R where σ = inf 0 ≤ v ≤ T − s : Xxs+v = b(s) . Since by

regularity of X we find that σ ↓ 0 P -a.s. as s ↑ t and x ↓ b(t−) , it follows from(27.1.44) that

Vx(u, y) ≤ − 1u

+ E

((u + σ)Mσ − u

u (u + σ)Mσ

)≤ − 1

u+ δ (27.1.45)

for all s < t close enough to t and some x > b(t−) close enough to b(t−) . Thiscompletes the proof of the second claim, and thus the initial claim is proved aswell.

11. We show that V is given by the formula (27.1.23) and that b solvesequation (27.1.22). For this, note that V satisfies the following conditions:

V is C1,2 on C ∪ D, (27.1.46)Vt + LXV is locally bounded, (27.1.47)x → V (t, x) is convex, (27.1.48)t → Vx(t, b(t)±) is continuous. (27.1.49)

Indeed, the conditions (27.1.46) and (27.1.47) follow from the facts that V isC1,2 on C and V = G on D upon recalling that D lies below γ so thatG(t, x) = 1− x/t for all (t, x) ∈ D and thus G is C1,2 on D . [When we say in(27.1.47) that Vt + LXV is locally bounded, we mean that Vt + LXV is boundedon K ∩ (C ∪D) for each compact set K in [0, T ]×R+. ] The condition (27.1.48)was established in (27.1.25) above. The condition (27.1.49) follows from (27.1.12)since according to the latter we have Vx(t, b(t)±) = −1/t for t > 0 .

Since (27.1.46)–(27.1.49) are satisfied we know that the local time-space for-mula (cf. Subsection 3.5) can be applied. This gives

V (t+s, Xt+s) = V (t, x) (27.1.50)

+∫ s

0

(Vt + LXV

)(t+u, Xt+u) I

(Xt+u = b(t+u)

)du

+∫ s

0

σ Xt+u Vx(t + u, Xt+u) I(Xt+u = b(t+u)

)dBu

+12

∫ s

0

(Vx(t+u, Xt+u+) − Vx(t+u, Xt+u−)

)× I

(Xt+u = b(t+u)

)d b

t+u(X)

=∫ s

0

(Gt + LXG

)(t + u, Xt+u) I

(Xt+u < b(t+u)

)du + Ms


where the final equality follows by the smooth-fit condition (27.1.12) and Ms =∫ s

0σXt+uVx(t + u, Xt+u) I

(Xt+u = b(t+u)

)dBu is a continuous martingale for

0 ≤ s ≤ T − t with t > 0 . Noting that (Gt + LXG)(t, x) = x/t2 − (1 − rx)/t forx < t we see that (27.1.50) yields

V (t + s, Xt+s) = V (t, x) (27.1.51)

+∫ s

0

(Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < b(t+u)

)du + Ms.

Setting s = T − t , using that V (T, x) = G(T, x) for all x ≥ 0 , and taking thePt,x -expectation in (27.1.51), we find by the optional sampling theorem (page 60)that

Et,x

(1 − XT

T

)+= V (t, x) (27.1.52)

+∫ T−t

0

Et,x

((Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < b(t+u)

))du.

Making use of (27.1.18)–(27.1.20) we see that (27.1.52) is the formula (27.1.23).Moreover, inserting x = b(t) in (27.1.52) and using that V (t, b(t)) = G(t, b(t)) =1 − b(t)/t , we see that b satisfies the equation (27.1.22) as claimed.

12. We show that b(t) > 0 for all 0 < t ≤ T and that b(0+) = 0 . For this,suppose that b(t0) = 0 for some t0 ∈ (0, T ) and fix t ∈ (0, t0) . Then (t, x) ∈ Cfor all x > 0 as small as desired. Taking any such (t, x) ∈ C and denoting byτD = τD(t, x) the first hitting time to D under Pt,x , we find by (27.1.51) that

V (t + τD, Xt+τD) = G(t + τD, Xt+τD

) =(

1 − Xt+τD

t + τD

)+(27.1.53)

= V (t, x) + Mt+τD= 1 − x

t+ Mt+τD

.

Taking the Pt,x -expectation and letting x ↓ 0 we get

Et,0

(1 − Xt+τD

t + τD

)+= 1 (27.1.54)

where τD = τD(t, 0) . As clearly Pt,0(Xt+τD≥ T ) > 0 we see that the left-hand

side of (27.1.54) is strictly smaller than 1 thus contradicting the identity. Thisshows that b(t) must be strictly positive for all 0 < t ≤ T . Combining thisconclusion with the known inequality b(t) ≤ γ(t) which is valid for all 0 < t ≤ Twe see that b(0+) = 0 as claimed.

13. We show that b is the unique solution of the nonlinear integral equation(27.1.22) in the class of continuous functions c : (0, T ) → R satisfying 0 < c(t) <t/(1 + rt) for all 0 < t < T . (Note that this class is larger than the class of


functions having the established properties of b which is moreover known to beincreasing.) The proof of the uniqueness will be presented in the final three stepsof the main proof as follows.

14. Let c : (0, T ] → R be a continuous solution of the equation (27.1.22)satisfying 0 < c(t) < t for all 0 < t < T . We want to show that this c must thenbe equal to the optimal stopping boundary b .

Motivated by the derivation (27.1.50)–(27.1.52) which leads to the formula(27.1.55), let us consider the function U c : (0, T ]× [0,∞) → R defined as follows:

U c(t, x) = E t,x

((1 − XT

T

)+)(27.1.55)

−∫ T−t

0

Et,x

((Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < c(t+u)

))du

for (t, x) ∈ (0, T ]× [0,∞) . In terms of (27.1.18)–(27.1.20) note that U c is explic-itly given by

U c(t, x) = F (T − t, x) (27.1.56)

−∫ T−t

0

1t+u

((1

t+u + r)G(u, x, c(t+u)

)− H(u, x, c(t+u)

))du

for (t, x) ∈ (0, T ]× [0,∞) . Observe that the fact that c solves (27.1.22) on (0, T )means exactly that U c(t, c(t)) = G(t, c(t)) for all 0 < t < T . We will nowmoreover show that U c(t, x) = G(t, x) for all x ∈ [0, c(t)] with t ∈ (0, T ) . Thisis the key point in the proof (cf. Subsections 25.2 and 26.2 above) that can bederived using a martingale argument as follows.

If X = (Xt)t≥0 is a Markov process (with values in a general state space)and we set F (t, x) = ExG(XT−t) for a (bounded) measurable function G withPx(X0 = x) = 1 , then the Markov property of X implies that F (t, Xt) is a mar-tingale under Px for 0 ≤ t ≤ T . Similarly, if we set F (t, x) = Ex(

∫ T−t

0H(Xu) du)

for a (bounded) measurable function H with Px(X0 = x) = 1 , then the Markovproperty of X implies that F (t, Xt) +

∫ t

0 H(Xu) du is a martingale under Px

for 0 ≤ t ≤ T . Combining these two martingale facts applied to the time-spaceMarkov process (t + s, Xt+s) instead of Xs , we find that

U c(t + s, Xt+s) −∫ s

0

(Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < c(t+u)

)du (27.1.57)

is a martingale under Pt,x for 0 ≤ s ≤ T − t . We may thus write

U c(t + s, Xt+s) (27.1.58)

−∫ s

0

(Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < c(t+u)

)du = U c(t, x) + Ns


where (Ns)0≤s≤T−t is a martingale with N0 = 0 under Pt,x .

On the other hand, we know from (27.1.32) that

G(t + s, Xt+s) = G(t, x) (27.1.59)

+∫ s

0

(Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < α(t+u)

)du + Ms + Ls

where Ms = −σ∫ s

0 (Xt+u/(t + u)) I(Xt+u < α(t+u)) dBu is a continuous martin-gale under Pt,x and Ls = (1/2)

∫ s

0dα

t+u(X)/(t + u) is an increasing process for0 ≤ s ≤ T − t .

For 0 ≤ x ≤ c(t) with t ∈ (0, T ) given and fixed, consider the stopping time

σc = inf 0 ≤ s ≤ T − t : Xt+s ≥ c(t+s) . (27.1.60)

Using that U c(t, c(t)) = G(t, c(t)) for all 0 < t < T (since c solves (27.1.22)as pointed out above) and that U c(T, x) = G(T, x) for all x ≥ 0 , we see thatU c(t + σc, Xt+σc) = G(t + σc, Xt+σc) . Hence from (27.1.58) and (27.1.59) usingthe optional sampling theorem (page 60) we find

U c(t, x) = Et,x

(U c(t + σc, Xt+σc)

)(27.1.61)

− Et,x

(∫ σc

0

(Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < c(t+u)

)du

)= Et,x

(G(t + σc, Xt+σc)

)− Et,x

(∫ σc

0

(Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < c(t+u)

)du

)= G(t, x) + Et,x

(∫ σc

0

(Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < α(t+u)

)du

)− Et,x

(∫ σc

0

(Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < c(t+u)

)du

)= G(t, x)

since Xt+u < α(t+u) and Xt+u < c(t+u) for all 0 ≤ u < σc . This proves thatU c(t, x) = G(t, x) for all x ∈ [0, c(t)] with t ∈ (0, T ) as claimed.

15. We show that U c(t, x) ≤ V (t, x) for all (t, x) ∈ (0, T ] × [0,∞) . Forthis, consider the stopping time

τc = inf 0 ≤ s ≤ T − t : Xt+s ≤ c(t+s) (27.1.62)


under Pt,x with (t, x) ∈ (0, T ] × [0,∞) given and fixed. The same argumentsas those given following (27.1.60) above show that U c(t + τc, Xt+τc) = G(t +τc, Xt+τc) . Inserting τc instead of s in (27.1.58) and using the optional samplingtheorem (page 60) we get

U c(t, x) = Et,x U c(t + τc, Xt+τc) = Et,x G(t + τc, Xt+τc) ≤ V (t, x) (27.1.63)

where the final inequality follows from the definition of V proving the claim.

16. We show that c ≥ b on [0, T ] . For this, consider the stopping time

σb = inf 0 ≤ s ≤ T − t : Xt+s ≥ b(t+s) (27.1.64)

under Pt,x where (t, x) ∈ (0, T ) × [0,∞) such that x < b(t) ∧ c(t) . Inserting σb

in place of s in (27.1.51) and (27.1.58) and using the optional sampling theorem(page 60) we get

Et,x V (t+σb, Xt+σb) = G(t, x) (27.1.65)

+ Et,x

(∫ σb

0

(Xt+u

(t+u)2− 1− rXt+u

(t+u)

)du

),

Et,x U c(t + σb, Xt+σb) = G(t, x) (27.1.66)

+ Et,x

(∫ σb

0

(Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < c(t+u)

)du

)where we also use that V (t, x) = U c(t, x) = G(t, x) for x < b(t) ∧ c(t) . SinceU c ≤ V it follows from (27.1.65) and (27.1.66) that

Et,x

(∫ σb

0

(Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u ≥ c(t+u)

)du

)≥ 0. (27.1.67)

Due to the fact that b(t) < t/(1+rt) for all 0 < t < T , we see that Xt+u/(t+u)2

−(1 − rXt+u)/(t + u) < 0 in (27.1.67) so that by the continuity of b and c itfollows that c ≥ b on [0, T ] as claimed.

17. We show that c must be equal to b . For this, let us assume that thereis t ∈ (0, T ) such that c(t) > b(t) . Pick x ∈ (b(t), c(t)) and consider the stoppingtime τb from (27.1.17). Inserting τb instead of s in (27.1.51) and (27.1.58) andusing the optional sampling theorem (page 60) we get

Et,x G(t + τb, Xt+τb) = V (t, x), (27.1.68)

Et,x (G(t + τb, Xt+τb) = U c(t, x) (27.1.69)

+ Et,x

(∫ τb

0

(Xt+u

(t+u)2− 1− rXt+u

(t+u)

)I(Xt+u < c(t+u)

)du

)


where we also use that V (t + τb, Xt+τb) = U c(t + τb, Xt+τb

)G(t + τb, Xt+τb) upon

recalling that c ≥ b and U c = G either below c or at T . Since U c ≤ V we seefrom (27.1.68) and (27.1.69) that

Et,x

(∫ τb

0

(Xt+u

(t + u)2− 1 − rXt+u

(t + u)

)I(Xt+u < c(t+u)

)du

)≥ 0. (27.1.70)

Due to the fact that c(t) < t/(1+rt) for all 0 < t < T by assumption, we see thatXt+u/(t + u)2 − (1− rXt+u)/(t + u) < 0 in (27.1.70) so that by the continuity ofb and c it follows that such a point (t, x) cannot exist. Thus c must be equalto b , and the proof is complete.

3. Remarks on numerics. 1. The following method can be used to calculatethe optimal stopping boundary b numerically by means of the integral equation(27.1.22). Note that the formula (27.1.23) can be used to calculate the arbitrage-free price V when b is known.

Set ti = ih for i = 0, 1, . . . , n where h = T/n and denote

J(t, b(t)) = 1 − b(t)t − F (T − t, b(t)), (27.1.71)

K(t, b(t); t+u, b(t+u)) (27.1.72)

= 1t+u

((1

t+u + r)G(u, b(t), b(t+u))− H(u, b(t), b(t+u))

).

Then the following discrete approximation of the integral equation (27.1.22) isvalid:

J(ti, b(ti)) =n∑

j=i+1

K(ti, b(ti); tj , b(tj))h (27.1.73)

for i = 0, 1, . . . , n − 1 . Letting i = n− 1 and b(tn) = T/(1+rT ) we can solveequation (27.1.73) numerically and get a number b(tn−1) . Letting i = n− 2and using the values of b(tn−1) and b(tn) we can solve equation (27.1.73) nu-merically and get a number b(tn−2) . Continuing the recursion we obtain b(tn),b(tn−1), . . . , b(t1), b(t0) as an approximation of the optimal stopping boundary bat points 0, h, . . . , T −h, T .

It is an interesting numerical problem to show that the approximation con-verges to the true function b on [0, T ] as h ↓ 0 . Another interesting problem isto derive the rate of convergence.

2. To perform the previous recursion we need to compute the functionsF , G , H from (27.1.18)–(27.1.20) as efficiently as possible. Simply by observ-ing the expressions (27.1.18)–(27.1.21) it is apparent that finding these functionsnumerically is not trivial. Moreover, the nature of the probability density func-tion f in (27.1.21) presents a further numerical challenge. Part of this probabilitydensity function is the Hartman–Watson density discussed in [8]. As t tends to


zero, the numerical estimate of the Hartman–Watson density oscillates, with theoscillations increasing rapidly in both amplitude and frequency as t gets closer tozero. The authors of [8] mention that this may be a consequence of the fact thatt → exp(2π2/σ2t) rapidly increases to infinity while z → sin(4πz/σ2t) oscillatesmore and more frequently. This rapid oscillation makes accurate estimation off(t, s, a) with t close to zero very difficult.

The problems when dealing with t close to zero are relevant to pricing theearly exercise Asian call option. To find the optimal stopping boundary b as thesolution to the implicit equation (27.1.73) it is necessary to work backward fromT to 0 . Thus to get an accurate estimate for b when b(T ) is given, the nextestimate of b(u) must be found for some value of u close to T so that t = T −uwill be close to zero.

Even if we get an accurate estimate for f , to solve (27.1.18)–(27.1.20) weneed to evaluate two nested integrals. This is slow computationally. A crude at-tempt has been made at storing values for f and using these to estimate F , G , Hin (27.1.18)–(27.1.20) but this method has not produced reliable results.

3. Another approach to finding the functions F , G , H from (27.1.18)–(27.1.20) can be based on numerical solutions of partial differential equations.Two distinct methods are available.

Consider the transition probability density of the process X given by

p(s, x; t, y) =d

dyP(Xt ≤ y | Xs = x) (27.1.74)

where 0 ≤ s < t and x, y ≥ 0 . Since p(s, x; t, y) = p(0, x; t− s, y) we see thatthere is no restriction to assume that s = 0 in the sequel.

The forward equation approach leads to the initial-value problem

pt = −((1− ry)p)y + (Dyp)yy ( t > 0 , y > 0 ), (27.1.75)

p(0, x; 0+, y) = δ(y−x) ( y ≥ 0 ) (27.1.76)

where D = σ2/2 and x ≥ 0 is given and fixed (recall that δ denotes the Diracdelta function). Standard results (cf. [64]) imply that there is a unique non-negativesolution (t, y) → p(0, x; t, y) of (27.1.75)–(27.1.76). The solution p satisfies thefollowing boundary conditions:

p(0, x; t, 0+) = 0 (0 is entrance ), (27.1.77)

p(0, x; t,∞−) = 0 (∞ is normal ). (27.1.78)

The solution p satisfies the following integrability condition:∫ ∞

0

p(0, x; t, y) dy = 1 (27.1.79)


for all x ≥ 0 and all t ≥ 0 . Once the solution (t, y) → p(0, x; t, y) of (27.1.75)–(27.1.76) has been found, the functions F , G , H from (27.1.18)–(27.1.20) canbe computed using the general formula

E0,x

(g(Xt)

)=∫ ∞

0

g(y) p(0, x; t, y) dy (27.1.80)

upon choosing the appropriate function g : R+ → R+ .The backward equation approach leads to the terminal-value problem

qt = (1− rx) qx + D x2 qxx ( t > 0, x > 0 ), (27.1.81)

q(T, x) = h(x) (x ≥ 0 ) (27.1.82)

where h : R+ → R+ is a given function. Standard results (cf. [64]) imply thatthere is a unique non-negative solution (t, x) → q(t, x) of (27.1.81)–(27.1.82).Taking x → h(x) to be x → (1−x/T )+ (with T fixed ), x → x I(x≤y) (withy fixed ), x → I(x≤y) (with y fixed ) it follows that the unique non-negative so-lution q of (27.1.81)–(27.1.82) coincides with F , G , H from (27.1.18)–(27.1.20)respectively. (For numerical results of a similar approach see [177].)

4. It is an interesting numerical problem to carry out either of the twomethods described above and produce approximations to the optimal stoppingboundary b using (27.1.73). Another interesting problem is to derive the rate ofconvergence.

4. Appendix. In this appendix we exhibit an explicit expression for the prob-ability density function f of (St, It) under P with S0 = 1 and I0 = 0 given in(27.1.21) above.

Let B = (Bt)t≥0 be a standard Brownian motion defined on a probabilityspace (Ω,F , P) . With t > 0 and ν ∈ R given and fixed recall from [224, p. 527]that the random variable A

(ν)t =

∫ t

0e2(Bs+νs)ds has the conditional distribution

P(A

(ν)t ∈ dy

∣∣Bt + νt = x)

= a(t, x, y) dy (27.1.83)

where the density function a for y > 0 is given by

a(t, x, y) =1

πy2exp

(x2 + π2

2t+ x − 1

2y

(1 + e2x

))(27.1.84)

×∫ ∞

0

exp(−z2

2t− ex

ycosh(z)

)sinh(z) sin

(πz

t

)dz.

This implies that the random vector(2(Bt + νt), A(ν)

t

)has the distribution

P(2(Bt + νt) ∈ dx, A

(ν)t ∈ dy

)= b(t, x, y) dx dy (27.1.85)


where the density function b for y > 0 is given by

b(t, x, y) = a(t,

x

2, y) 1

2√

tϕ

(x − 2νt

2√

t

)(27.1.86)

=1

(2π)3/2y2√

texp

(π2

2t+(ν + 1

2

)x − ν2

2t − 1

2y

(1 + ex

))×∫ ∞

0

exp(−z2

2t− ex/2

ycosh(z)

)sinh(z) sin

(πz

t

)dz

and we set ϕ(z) = (1/√

2π)e−z2/2 for z ∈ R (for related expressions in terms ofHermite functions see [46] and [181]).

Denoting Kt = αBt + βt and Lt =∫ t

0 eαBs+βsds with α = 0 and β ∈ R

given and fixed, and using that the scaling property of B implies

P

(αBt+βt ≤ x,

∫ t

0

eαBs+βs ds ≤ y

)(27.1.87)

= P

(2(Bt′ + νt′) ≤ x,

∫ t′

0

e2(Bs+νs) ds ≤ α2

4y

)with t′ = α2t/4 and ν = 2β/α2 , it follows by applying (27.1.85) and (27.1.86)that the random vector (Kt, Lt) has the distribution

P(Kt ∈ dx, Lt ∈ dy

)= c(t, x, y) dx dy (27.1.88)

where the density function c for y > 0 is given by

c(t, x, y) =α2

4b

(α2

4t, x,

α2

4y

)(27.1.89)

=2√

2π3/2α3

1y2√

texp

(2π2

α2t+( β

α2+

12

)x − β2

2α2t − 2

α2y

(1+ex

))×∫ ∞

0

exp(− 2z2

α2t− 4ex/2

α2ycosh(z)

)sinh(z) sin

(4πz

α2t

)dz.

From (27.1.8) and (27.1.3) we see that

f(t, s, a) =1s

c(t, log s, a) =1s

α2

4b

(α2

4t, log s,

α2

4a

)(27.1.90)

with α = σ and β = r + σ2/2 . Hence (27.1.21) follows by the final expressionin (27.1.86).

Notes. According to financial theory (see e.g. [197]) the arbitrage-free price ofthe early exercise Asian call option with floating strike is given as V in (27.1.1)


above where Iτ/τ denotes the arithmetic average of the stock price S up totime τ . The problem was first studied in [89] where approximations to the valuefunction V and the optimal boundary b were derived. The main aim of thepresent section (following [170]) is to derive exact expressions for V and b .

The optimal stopping problem (27.1.1) is three-dimensional. When a changeof measure is applied (as in [186] and [115]) the problem reduces to (27.1.9) andbecomes two-dimensional. The problem (27.1.9) is more complicated than the well-known problems of American and Russian options (cf. Sections 25 and 26 above)since the gain function depends on time in a nonlinear way. From the result ofTheorem 27.1 above it follows that the free-boundary problem (27.1.10)–(27.1.14)characterizes the value function V and the optimal stopping boundary b in aunique manner. Our main aim, however, is to follow the train of thought initiatedby Kolodner [114] where V is initially expressed in terms of b , and b itself is thenshown to satisfy a nonlinear integral equation. A particularly simple approach forachieving this goal in the case of the American put option has been suggested in[110], [102], [27] and we take it up in the present section. We moreover see (asin [164] and [165]) that the nonlinear equation derived for b cannot have othersolutions. The key argument in the proof relies upon a local time-space formula(see Subsection 3.5).

The latter fact of uniqueness may be seen as the principal result of thesection. The same method of proof can also be used to show the uniqueness ofthe optimal stopping boundary solving nonlinear integral equations derived in[89] and [223] where this question was not explicitly addressed. These equationsarise from the early exercise Asian options (call or put) with floating strike basedon geometric averaging. The early exercise Asian put option with floating strikecan be dealt with analogously to the Asian call option treated here. For financialinterpretations of the early exercise Asian options and other references on the topicsee [89] and [223].

Chapter VIII.

Optimal stopping in financial engineering

28. Ultimate position

The problem to be discussed in this section is motivated by the optimal stoppingproblem studied in Section 30 below and our wish to cover the Mayer formulationof the same problem (cf. Section 6). Since the gain process in the optimal stoppingproblem depends on the future, we refer to it as an optimal prediction problem.These problems appear to be of particular interest in financial engineering.

1. Let B = (Bt)0≤t≤1 be a standard Brownian motion defined on a proba-bility space (Ω,F , P) , and let M : R → R be a measurable (continuous) functionsuch that EM(B1)2 < ∞ .

Consider the optimal prediction problem

V = inf0≤τ≤1

E(M(B1) − Bτ

)2 (28.0.1)

where the infimum is taken over all stopping times τ of B (satisfying 0 ≤ τ ≤ 1 ).Note that M(B1) is not adapted to the natural filtration FB

t = σ(Bs : 0 ≤ s ≤ t)of B for t ∈ [0, 1〉 so that the problem (28.0.1) falls outside the scope of generaltheory of optimal stopping from Chapter I.

The following simple arguments reduce the optimal prediction problem(28.0.1) to an optimal stopping problem (in terms of the general optimal stop-ping theory). For this, note that

E((

M(B1) − Bt

)2 ∣∣FBt

)= E

((M(B1 − Bt + Bt) − Bt

)2 ∣∣FBt

)(28.0.2)

= E(M(B1−t + x) − x

)2∣∣x=Bt

upon using that B1−Bt is independent from FBt and equally distributed as

B1−t .

438 Chapter VIII. Optimal stopping in financial engineering

Let

G(t, x) = E(M(B1−t + x) − x

)2 = E(M(

√1− t B1 + x) − x

)2 (28.0.3)

=∫

R

(M(

√1− t y + x) − x

)2ϕ(y) dy

where we use that B1−t =law√

1− t B1 and set

ϕ(y) =1√2π

e−y2/2 (28.0.4)

for y ∈ R to denote the standard normal density function. We get from (28.0.2)and (28.0.3) that

E((

M(B1) − Bt

)2 ∣∣FBt

)= G(t, Bt) (28.0.5)

for 0 ≤ t ≤ 1 .

2. Standard arguments based on the fact that each stopping time is the limitof a decreasing sequence of discrete stopping times imply that (28.0.5) extends asfollows:

E((

M(B1) − Bτ

)2 ∣∣FBτ

)= G(τ, Bτ ) (28.0.6)

for all stopping times τ of B . Taking E in (28.0.6) we find that the optimalprediction problem (28.0.1) is equivalent to the optimal stopping problem

V = inf0≤τ≤1

EG(τ, Bτ ) (28.0.7)

where the infimum is taken over all stopping times τ of B (satisfying 0 ≤ τ ≤ 1 ).

This problem can be treated by the methods of Chapters VI and VII. Wewill omit further details. (Note that when M(x) = x for all x ∈ R , then theoptimal stopping time τ∗ is trivial as it equals 1 identically.)

29. Ultimate integral

The problem to be discussed in this section (similarly to the previous section) ismotivated by the optimal prediction problem studied in Section 30 below and ourwish to cover the Lagrange formulation of the same problem (cf. Section 6).

1. Let B = (Bt)0≤t≤1 be a standard Brownian motion defined on a proba-bility space (Ω,F , P) , and let L : R → R be a measurable (continuous) functionsuch that E

( ∫ 1

0 L(Bt) dt)2

< ∞ .

Consider the optimal prediction problem

V = inf0≤τ≤1

E

(∫ 1

0

L(Bt) dt − Bτ

)2

(29.0.8)

Section 29. Ultimate integral 439

where the infimum is taken over all stopping times τ of B (satisfying 0 ≤ τ ≤ 1 ).Note that

∫ 1

0 L(Bt) dt is not adapted to the natural filtration FBt = σ(Bs : 0 ≤

s ≤ t) of B for t ∈ [0, 1) so that the problem (29.0.8) falls outside the scope ofgeneral theory of optimal stopping from Chapter I.

The following simple arguments reduce the optimal prediction problem(29.0.8) to an optimal stopping problem (in terms of the general optimal stop-ping theory). In the sequel we will assume that L is continuous. Set M(x) =∫ x

0

∫ y

0L(z) dz dy for x ∈ R . Then M is C2 and Ito’s formula (page 67) yields

M(Bt) = M(0) +∫ t

0

M ′(Bs) dBs +12

∫ t

0

M ′′(Bs) ds. (29.0.9)

Hence we find∫ 1

0

L(Bs) ds =∫ 1

0

M ′′(Bs) ds (29.0.10)

= 2(

M(B1) − M(0) −∫ 1

0

M ′(Bs) dBs

).


E

[∫ 1

0

L(Bt) dt − Bτ

]2

(29.0.11)

= E

[2(

M(B1) − M(0) −∫ 1

0

M ′(Bt) dBt

)− Bτ

]2

= 4 E

[M(B1) − M(0) −

∫ 1

0

M ′(Bt) dBt

]2

− 4 E (M(B1)Bτ ) − 4 E

[(∫ 1

0

M ′(Bt) dBt

)Bτ

]+ EB2

τ .

By (29.0.10) we have

E

(M(B1) − M(0) −

∫ 1

0

M ′(Bt) dBt

)2

(29.0.12)

=14

E

(∫ 1

0

L(Bt) dt

)2

=: CL.

By stationary and independent increments of B (just as in (28.0.2)–(28.0.7) inSection 28 above) we get

E (M(B1)Bτ ) = E(M(B1 − Bτ + Bτ )Bτ

)(29.0.13)

= E(M(√

1− t B1 + x))∣∣∣

t=τ,x=Bτ

= G(τ, Bτ )


for all stopping times τ of B (satisfying 0 ≤ τ ≤ 1 ). Finally, by the martingaleproperty of

∫ t

0 M ′(Bs) dBs for t ∈ [0, 1] we obtain

E

[(∫ 1

0

M ′(Bt) dBt

)Bτ

]= E

[(∫ τ

0

M ′(Bt) dBt

)Bτ

](29.0.14)

= E

∫ τ

0

M ′(Bt) dt

as long as E∫ 1

0 (M ′(Bt))2 dt < ∞ for instance. Inserting (29.0.12)–(29.0.14) into(29.0.11) and using that EB2

τ = Eτ , we get

E

[∫ 1

0

L(Bt) dt − Bτ

]2

(29.0.15)

= 4(

CL − EG(τ, Bτ ) − E

∫ τ

0

M ′(Bt) dt

)+ Eτ.

Setting H = M ′ − 14 this shows that the optimal prediction problem (29.0.8) is

equivalent to the optimal stopping problem

V = sup0≤τ≤1

E

(G(τ, Bτ ) +

∫ τ

0

H(Bt) dt

)(29.0.16)

where the supremum is taken over all stopping times τ of B (satisfying 0 ≤ τ ≤1 ).

2. Consider the case when L(x) = x for all x ∈ R in the problem (29.0.8).Setting I1 =

∫ 1

0Bt dt we find by the integration by parts formula (or Ito’s formula

applied to tBt and letting t = 1 in the result) that the following analogue of theformula (30.1.7) below is valid:

I1 =∫ 1

0

(1 − t) dBt. (29.0.17)

Denoting Mt =∫ t

0(1 − s) dBs , it follows by the martingale property of the latter

for t ∈ [0, 1] that

E (I1 − Bτ )2 = E |I1|2 − 2 E (I1Bτ ) + E |Bτ |2 (29.0.18)

=13− 2 E

[∫ τ

0

(1− s) ds

]+ Eτ

=13

+ E (τ2 −2τ) + Eτ =13

+ E (τ2 − τ)

for all stopping time τ of B (satisfying 0 ≤ τ ≤ 1 ). Hence we see that

V = inf0≤τ≤1

E (I1 − Bτ )2 =112

= 0.08 . . . (29.0.19)

and that the infimum is attained at τ∗ ≡ 1/2 . This shows that the problem(29.0.8) with L(x) = x for x ∈ R has a trivial solution.

Section 30. Ultimate maximum 441

30. Ultimate maximum

Imagine the real-line movement of a Brownian particle started at 0 during thetime interval [0, 1] . Let S1 denote the maximal positive height that the particleever reaches during this time interval. As S1 is a random quantity whose valuesdepend on the entire Brownian path over the time interval, its ultimate value isat any given time t ∈ [0, 1) unknown. Following the Brownian particle from theinitial time 0 onward, the question arises naturally of how to determine a timewhen the movement should be terminated so that the position of the particle atthat time is as “close” as possible to the ultimate maximum S1 . In the next twosubsections we present the solution to this problem if “closeness” is measured bya mean-square distance.

30.1. Free Brownian motion

1. To formulate the problem above more precisely, let B = (Bt)0≤t≤1 be astandard Brownian motion defined on a probability space (Ω,F , P) , and let(FB

t )0≤t≤1 denote the natural filtration generated by B . Letting M denote thefamily of all stopping (Markov) times τ with respect to (FB

t )0≤t≤1 satisfying0 ≤ τ ≤ 1 , the problem is to compute

V∗ = infτ∈M

E(Bτ − max

0≤t≤1Bt

)2

(30.1.1)

and to find an optimal stopping time (the one at which the infimum in (30.1.1) isattained).

The solution of this problem is presented in Theorem 30.1 below. It turnsout that the maximum process S = (St)0≤t≤1 given by

St = sup0≤s≤t

Bs (30.1.2)

and the CUSUM-type (reflected) process S −B = (St −Bt)0≤t≤1 play a key rolein the solution.

The optimal stopping problem (30.1.1) is of interest, for example, in financialengineering where an optimal decision (i.e. optimal stopping time) should be basedon a prediction of the time when the observed process take its maximal value(over a given time interval). The argument also carries over to many other appliedproblems where such predictions play a role.

2. The main result of this subsection is contained in the following theorem.Below we let

ϕ(x) =1√2π

e−x2/2 & Φ(x) =∫ x

−∞ϕ(y) dy (x ∈ R) (30.1.3)

denote the density and distribution function of a standard normal variable.


Theorem 30.1. Consider the optimal stopping problem (30.1.1) where B =(Bt)0≤t≤1 is a standard Brownian motion. Then the value V∗ is given by theformula

V∗ = 2Φ(z∗) − 1 = 0.73 . . . (30.1.4)

where z∗ = 1.12 . . . is the unique root of the equation

4Φ(z∗) − 2z∗ϕ(z∗) − 3 = 0 (30.1.5)

and the following stopping time is optimal (see Figures VIII.2–VIII.5):

τ∗ = inf

0 ≤ t ≤ 1 : St −Bt ≥ z∗√

1− t

(30.1.6)

where St is given by (30.1.2) above.

Proof. Since S1 = sup0≤s≤1 Bs is a square-integrable functional of the Brownianpath on [0, 1] , by the Ito–Clark representation theorem (see e.g. [174, p. 199])there exists a unique (FB

t )0≤t≤1 -adapted process H = (Ht)0≤t≤1 satisfyingE(∫ 1

0 H2t dt) < ∞ such that

S1 = a +∫ 1

0

Ht dBt (30.1.7)

where a = ES1 . Moreover, the following explicit formula is known to be valid:

Ht = 2(

1 − Φ(

St −Bt√1− t

))(30.1.8)

for 0 ≤ t ≤ 1 (see e.g. [178, p. 93] and [107, p. 365] or paragraph 3 below for adirect argument).

1. Associate with H the square-integrable martingale M = (Mt)0≤t≤1

given by

Mt =∫ t

0

Hs dBs. (30.1.9)

By the martingale property of M and the optional sampling theorem (page 60),we obtain

E(Bτ −S1)2 = E|Bτ |2 − 2E(BτM1) + E|S1|2 (30.1.10)

= Eτ − 2E(BτMτ ) + 1 = E

(∫ τ

0

(1−2Ht

)dt

)+ 1

for all τ ∈ M (recall that S1 =law |B1| ). Inserting (30.1.8) into (30.1.10) we seethat (30.1.1) can be rewritten as

V∗ = infτ∈M

E

(∫ τ

0

F

(St −Bt√

1− t

)dt

)+ 1 (30.1.11)

where we denote F (x) = 4Φ(x)− 3 .


z W*(z)

z*z*-

Figure VIII.1: A computer drawing of the map (30.1.19). The smoothfit (30.1.23) holds at −z∗ and z∗ .

Since S −B = (St − Bt)0≤t≤1 is a Markov process for which the naturalfiltration (FS−B

t )t≥0 coincides with the natural filtration (FB)t≥0 , it followsfrom general theory of optimal stopping (see Subsection 2.2) that in (30.1.11) weneed only consider stopping times which are hitting times for S − B . Recallingmoreover that S − B =law |B| by Levy’s distributional theorem (see (4.4.24))and once more appealing to general theory, we see that (30.1.11) is equivalent tothe optimal stopping problem

V∗ = infτ∈M

E

(∫ τ

0

F

( |Bt|√1− t

)dt

)+ 1. (30.1.12)

In our treatment of this problem, we first make use of a deterministic change oftime (cf. Subsection 5.1 and Section 10).

2. Motivated by the form of (30.1.12), consider the process Z = (Zt)t≥0

given byZt = etB1−e−2t . (30.1.13)

By Ito’s formula (page 67) we find that Z is a (strong) solution of the linearstochastic differential equation

dZt = Zt dt +√

2 dβt (30.1.14)

where the process β = (βt)0≤t≤1 is given by

βt =1√2

∫ t

0

es dB1−e−2s =1√2

∫ 1−e−2t

0

1√1− s

dBs. (30.1.15)


As β is a continuous Gaussian martingale with mean zero and variance equalto t , it follows by Levy’s characterization theorem (see e.g. [174, p. 150]) thatβ is a standard Brownian motion. We thus may conclude that Z is a diffusionprocess with the infinitesimal generator given by

LZ = zd

dz+

d2

dz2. (30.1.16)

Substituting t = 1 − e−2s in (30.1.12) and using (30.1.13), we obtain

V∗ = 2 infτ∈M

E

(∫ στ

0

e−2sF(|Zs|

)ds

)+ 1 (30.1.17)

upon setting στ = log(1/√

1− τ ) . It is clear from (30.1.13) that τ is a stoppingtime with respect to (FB

t )0≤t≤1 if and only if στ is a stopping time with respectto (FZ

s )s≥0 . This shows that our initial problem (30.1.1) reduces to solving

W∗ = infσ

E

(∫ σ

0

e−2sF(|Zs|

)ds

)(30.1.18)

where the infimum is taken over all (FZs )s≥0 -stopping times σ with values in

[0,∞] . This problem belongs to the general theory of optimal stopping for time-homogeneous Markov processes (see Subsection 2.2).

3. To calculate (30.1.18) define

W∗(z) = infσ

Ez

(∫ σ

0

e−2sF(|Zs|

)ds

)(30.1.19)

for z ∈ R , where Z0 = z under Pz , and the infimum is taken as above. Generaltheory combined with basic properties of the map z → F (|z|) prompts that thestopping time

σ∗ = inf t > 0 : |Zt| ≥ z∗ (30.1.20)

should be optimal in (30.1.19), where z∗>0 is a constant to be found.

To determine z∗ and compute the value function z → W∗(z) in (30.1.19),it is a matter of routine to formulate the following free-boundary problem:(

LZ −2)W (z) = −F (|z|) for z ∈ (−z∗, z∗), (30.1.21)

W (±z∗) = 0 (instantaneous stopping), (30.1.22)W ′(±z∗) = 0 (smooth fit) (30.1.23)

where LZ is given by (30.1.16) above. We shall extend the solution of (30.1.21)–(30.1.23) by setting its value equal to 0 for z /∈ (−z∗, z∗) , and thus the map soobtained will be C2 everywhere on R but at −z∗ and z∗ where it is C1 .


Inserting LZ from (30.1.16) into (30.1.21) leads to the equation

W ′′(z) + zW ′(z) − 2W (z) = −F (|z|) (30.1.24)

for z ∈ (−z∗, z∗) . The form of the equation (30.1.14) and the value (30.1.18)indicates that z → W∗(z) should be even; thus we shall additionally impose

W ′(0) = 0 (30.1.25)

and consider (30.1.24) only for z ∈ [0, z∗) .

The general solution of the equation (30.1.24) for z ≥ 0 is given by

W (z) = C1(1+z2) + C2

(zϕ(z) + (1+z2)Φ(z)

)+ 2Φ(z) − 3

2 . (30.1.26)

The three conditions W (z∗) = W ′(z∗) = W ′(0) = 0 determine constants C1 , C2

and z∗ uniquely; it is easily verified that C1 = Φ(z∗) , C2 = −1 , and z∗ is theunique root of the equation (30.1.5). Inserting this back into (30.1.24), we obtainthe following candidate for the value (30.1.19):

W (z) = Φ(z∗)(1+z2) − zϕ(z) + (1− z2)Φ(z) − 32 (30.1.27)

when z ∈ [0, z∗] , upon extending it to an even function on R as indicated above(see Figure VIII.1).

To verify that this solution z → W (z) coincides with the value function(30.1.19), and that σ∗ from (30.1.20) is an optimal stopping time, we shall notethat z → W (z) is C2 everywhere but at ±z∗ where it is C1 . Thus by theIto–Tanaka–Meyer formula (page 68) we find

e−2tW (Zt) = W (Z0) +∫ t

0

e−2s(

LZW (Zs) − 2W (Zs))

ds (30.1.28)

+√

2∫ t

0

e−2sW ′(Zs) dβs.

Hence by (30.1.24) and the fact that LZW (z) − 2W (z) = 0 > −F (|z|) for z /∈[−z∗, z∗] , upon extending W ′′ to ±z∗ as we please and using that the Lebesguemeasure of those t > 0 for which Zt = ±z∗ is zero, we get

e−2tW (Zt) ≥ W (Z0) −∫ t

0

e−2sF (|Zs|) ds + Mt (30.1.29)

where Mt =√

2∫ t

0e−2sW ′(Zs) dβs is a continuous local martingale for t ≥ 0 .

Using further that W (z) ≤ 0 for all z , a simple application of the optionalsampling theorem (page 60) in the stopped version of (30.1.29) under Pz showsthat W∗(z) ≥ W (z) for all z . To prove equality one may note that the passagefrom (30.1.28) to (30.1.29) also yields

0 = W (Z0) −∫ σ∗

0

e−2sF (|Zs|) ds + Mσ∗ (30.1.30)


upon using (30.1.21) and (30.1.22). Since clearly Ezσ∗ < ∞ and thus Ez√

σ∗ < ∞as well, and z → W ′(z) is bounded on [−z∗, z∗] , we can again apply the optionalsampling theorem and conclude that EzMσ∗ = 0 . Taking the expectation underPz on both sides in (30.1.30) enables one therefore to conclude W∗(z) = W (z)for all z , and the proof of the claim is complete.

From (30.1.17)–(30.1.19) and (30.1.27) we find that V∗ = 2W∗(0) + 1 =2(Φ(z∗)− 1) + 1 = 2Φ(z∗) − 1 . This establishes (30.1.4). Transforming σ∗ from(30.1.20) back to the initial problem via the equivalence of (30.1.11), (30.1.12)and (30.1.17), we see that τ∗ from (30.1.6) is optimal. The proof is complete. Remark 30.2. Recalling that S − B =law |B| we see that τ∗ is identically dis-tributed as the stopping time τ = inf t > 0 : |Bt| = z∗

√1− t . This implies

Eτ∗ = Eτ = E|Beτ |2 = (z∗)2E(1− τ) = (z∗)2(1−Eτ∗) , and hence we obtain

Eτ∗ =(z∗)2

1 + (z∗)2= 0.55 . . . . (30.1.31)

Moreover, using that (B4t − 6tB2

t + 3t2)t≥0 is a martingale, similar argumentsshow that

E(τ∗)2 =(z∗)6 + 5(z∗)4

(1 + (z∗)2)(3 + 6(z∗)2 + (z∗)4)= 0.36 . . . . (30.1.32)

From (30.1.31) and (30.1.32) we find

Var(τ∗) =2(z∗)4

(1 + (z∗)2)2(3 + 6(z∗)2 + (z∗)4)= 0.05 . . . . (30.1.33)

Remark 30.3. For the sake of comparison with (30.1.4) and (30.1.31) it is inter-esting to note that

V0 = inf0≤t≤1

E

((Bt − max

0≤s≤1Bs

)2)=

1π

+12

= 0.81 . . . (30.1.34)

with the infimum being attained at t = 12 . For this, recall from (30.1.10) and

(30.1.8) that

E(Bt −S1)2 = E

(∫ t

0

F

(Ss −Bs√

1− s

)ds

)+ 1 (30.1.35)

where F (x) = 4Φ(x)− 3 . Using further that S −B =law |B| , elementary calcu-lations show

E(Bt −S1)2 = 4

(∫ t

0

E

(Φ( |Bs|√

1− s

))ds

)− 3t + 1 (30.1.36)

= 4∫ t

0

(1 − 1

πarctan

√1− s

s

)ds − 3t + 1

= − 4π

(t arctan

√1− t

t+

12

arctan√

t

1− t− 1

2

√t (1− t)

)+ t + 1.


Hence (30.1.34) is easily verified by standard means.

Remark 30.4. In view of the fact that σ∗ from (30.1.20) with z∗ = 1.12 . . .from (30.1.5) is optimal in the problem (30.1.19), it is interesting to observe thatthe unique solution of the equation F (z) = 0 is given by z = 0.67 . . . . Notingmoreover that the map z → F (z) is increasing on [0,∞) and satisfies F (0) = −1 ,we see that F (z) < 0 for all z ∈ [0, z) and F (z) > 0 for all z > z . The size ofthe gap between z and z∗ quantifies the tendency of the process |Z| to returnto the “favourable” set [0, z) where clearly it is never optimal to stop.

Remark 30.5. The case of a general time interval [0, T ] easily reduces to the caseof a unit time interval treated above by using the scaling property of Brownianmotion implying

inf0≤τ≤T

E(Bτ − max

0≤t≤TBt

)2= T inf

0≤τ≤1E(Bτ − max

0≤t≤1Bt

)2(30.1.37)

which further equals to T (2Φ(z∗)− 1) by (30.1.4). Moreover, the same argumentshows that the optimal stopping time in (30.1.37) is given by

τ∗ = inf

0 ≤ t ≤ T : Bt ≥ z∗√

T − t (30.1.38)

where z∗ is the same as in Theorem 30.1.

Remark 30.6. From the point of view of mathematical statistics, the “estimator”Bτ of S1 is biased, since EBτ = 0 for all 0 ≤ τ ≤ 1 but at the same timeES1 = 0 . Instead of V∗ and V0 it is thus desirable to consider the values

V∗ = infa∈R, τ∈M

E(a+Bτ −S1

)2 & V0 = infa∈R, 0≤t≤1

E(a+Bt−S1

)2 (30.1.39)

and compare them with the values from (30.1.1) and (30.1.34). However, by usingthat EBτ = 0 we also find at once that a∗ = ES1 is optimal in (30.1.39) withV∗ = V∗− 2

π = 0.09 . . . and V0 = V0 − 2π = 0.18 . . . .

3. Stochastic integral representation of the maximum process. In this para-graph we present a direct derivation of the stochastic integral representation(30.1.7) and (30.1.8) (cf. [178, pp. 89–93] and [107, pp. 363–369]). For the sake ofcomparison we shall deal with a standard Brownian motion with drift

Bµt = Bt + µt (30.1.40)

for 0 ≤ t ≤ 1 where µ is a real number. The maximum process Sµ = (Sµt )0≤t≤1

associated with Bµ = (Bµt )0≤t≤1 is given by

Sµt = sup

0≤s≤tBµ

s . (30.1.41)


1. To derive the analogue of (30.1.7) and (30.1.8) in this case, we shall firstnote that stationary independent increments of Bµ imply

E(Sµ

1 | FBt

)= Sµ

t + E

((sup

t≤s≤1Bµ

s − Sµt

)+ ∣∣∣ FBt

)(30.1.42)

= Sµt + E

((sup

t≤s≤1

(Bµ

s −Bµt

)− (Sµt −Bµ

t

))+ ∣∣∣FBt

)= Sµ

t + E(Sµ

1−t − (z−x))+∣∣∣

z=Sµt , x=Bµ

t

.

Using further the formula E(X − c)+ =∫∞

c P(X > z) dz , we see that (30.1.42)reads

E(Sµ

1 | FBt

)= Sµ

t +∫ ∞

Sµt −Bµ

t

(1−Fµ

1−t(z))

dz := f(t, Bµt , Sµ

t ) (30.1.43)

where we use the notation

Fµ1−t(z) = P

(Sµ

1−t ≤ z), (30.1.44)

and the map f = f(t, x, s) is defined accordingly.

2. Applying Ito’s formula (page 67) to the right-hand side of (30.1.43), andusing that the left-hand side defines a continuous martingale, we find upon settingaµ = ESµ

1 that

E(Sµ

1 | FBt

)= aµ +

∫ t

0

∂f

∂x(s, Bµ

s , Sµs ) dBs (30.1.45)

= aµ +∫ t

0

(1 − Fµ

1−s(Sµs −Bµ

s ))

dBs,

as a nontrivial continuous martingale cannot have paths of bounded variation.This reduces the initial problem to the problem of calculating (30.1.44).

3. The following explicit formula is well known (see (4.4.21)):

Fµ1−t(z) = Φ

(z − µ(1− t)√

1− t

)− e2µzΦ

(−z − µ(1− t)√1− t

). (30.1.46)

Inserting this into (30.1.45) we obtain the representation

Sµ1 = aµ +

∫ 1

0

Hµt dBt (30.1.47)

where the process Hµ is explicitly given by

Hµt = 1 − Φ

((Sµ

t −Bµt ) − µ(1− t)√1− t

)(30.1.48)

+ e2µ(Sµt −Bµ

t ) Φ(−(Sµ

t −Bµt ) − µ(1− t)√1− t

).


1

2

σt

Figure VIII.2: A computer simulation of a Brownian path (Bt(ω))0≤t≤1

with the maximum being attained at σ = 0.51 .

Setting µ = 0 in this expression, we recover (30.1.7) and (30.1.8).

4. Note that the argument above extends to a large class of processes withstationary independent increments (Levy processes) by reducing the initial prob-lem to calculating the analogue of (30.1.44). In particular, the following “pre-diction” result deserves a special note. It is derived in exactly the same wayas (30.1.43) above.

Let X = (Xt)0≤t≤T be a process with stationary independent incrementsstarted at zero, and let us denote St = sup0≤s≤t Xs for 0 ≤ t ≤ T . If EST < ∞then the predictor E(ST | FX

t ) of ST based on the observations Xs : 0 ≤ s ≤ tis given by the formula

E(ST | FX

t

)= St +

∫ ∞

St−Xt

(1−FT−t(z)

)dz (30.1.49)

where FT−t(z) = P(ST−t ≤ z) .

4. In the setting of the optimal prediction problem (30.1.1) above the follow-ing remarkable identity holds:

E |τ − θ| = E (Bτ − Bθ)2 − 12 (30.1.50)

for all stopping times τ of B (satisfying 0 ≤ τ ≤ 1 ) where θ is the ( P -a.s.unique) time at which the maximum of B on [0, 1] is attained (i.e. Bθ = S1 ).


1

2

σt

Figure VIII.3: A computer drawing of the maximum process (St(ω))0≤t≤1

associated with the Brownian path from Figure VIII.2.

To verify (30.1.50) note that

|τ − θ| = (τ − θ)+ + (τ − θ)− = (τ − θ)+ + θ − τ ∧ θ (30.1.51)

=∫ τ

0

I(θ ≤ t) dt + θ −∫ τ

0

I(θ > t) dt

= θ +∫ τ

0

(2I(θ ≤ t) − 1) dt.

Taking E on both sides we get

E |τ − θ| = Eθ + E

∫ τ

0

(2I(θ ≤ t) − 1) dt (30.1.52)

=12

+ E

∫ ∞

0

(2 E (θ ≤ t | FB

t ) − 1)I(t < τ) dt

=12

+ E

∫ τ

0

(2πt − 1) dt

where we set

πt = P(θ ≤ t | FBt ). (30.1.53)


1

2

t

Figure VIII.4: A computer drawing of the process (St(ω)−Bt(ω))0≤t≤1

from Figures VIII.2 and VIII.3.

By stationary and independent increments of B , upon using (30.1.46) above withµ = 0 , we get

πt = P(St ≤ max

t≤s≤1Bs

∣∣FBt

)= P

(St − Bt ≤ max

t≤s≤1Bs − Bt

∣∣FBt

)(30.1.54)

= P(z−x ≤ S1−t)∣∣z=St, x=Bt

= 1 − F1−t(St −Bt)

= 2Φ(

St −Bt√1− t

)− 1.

Inserting (30.1.54) into (30.1.52) and using (30.1.8)+(30.1.10) above we see that(30.1.50) holds as claimed.

Finally, taking the infimum on both sides of (30.1.50) over all stopping timesτ of B (satisfying 0 ≤ τ ≤ 1 ), we see that the stopping time τ∗ given in (30.1.6)above is optimal for both (30.1.1) as well as the optimal prediction problem

W = inf0≤τ≤1

E |τ − θ| (30.1.55)

where the infimum is taken over all stopping times τ of B (satisfying 0 ≤ τ ≤ 1 ).Thus, not only is τ∗ the optimal time to stop as close as possible to the ultimatemaximum, but also τ∗ is the optimal time to stop as close as possible to the timeat which the ultimate maximum is attained. This is indeed the most extraordinaryfeature of this particular stopping time.


1

1

τ

t z*√1- t

t*

Figure VIII.5: A computer drawing of of the optimal stopping strat-egy (30.1.6) for the Brownian path from Figures VIII.2–VIII.4. It turnsout that τ∗ = 0.62 in this case (cf. Figure VIII.2).

30.2. Brownian motion with drift

Let B = (Bt)t≥0 be a standard Brownian motion defined on a probability space(Ω,F , P) where B0 = 0 under P . Set

Bµt = Bt+µt (30.2.1)

for t ≥ 0 where µ ∈ R is given and fixed. Then Bµ = (Bµt )t≥0 is a standard

Brownian motion with drift µ . Define

Sµt = max

0≤s≤tBµ

s (30.2.2)

for t ≥ 0 . Then Sµ = (Sµt )t≥0 is the maximum process associated with Bµ .

1. The optimal prediction problem. Given T > 0 we consider the optimalprediction problem

V = inf0≤τ≤T

E (Bµτ −Sµ

T )2 (30.2.3)

where the infimum is taken over all stopping times τ of Bµ (the latter meansthat τ is a stopping time with respect to the natural filtration of Bµ that in turnis the same as the natural filtration of B given by FB

t = σ(Bs : 0 ≤ s ≤ t) fort ∈ [0, T ] ). The problem (30.2.3) consists of finding an optimal stopping time (atwhich the infimum is attained) and computing V as explicitly as possible.

1. The identity (30.2.4) below reduces the optimal prediction problem(30.2.3) above (where the gain process (Bµ

t − SµT )0≤t≤T is not adapted to the


natural filtration of Bµ ) to the optimal stopping problem (30.2.10) below (wherethe gain process is adapted). Similar arguments in the case case µ = 0 were usedin Subsection 30.1 above.

Lemma 30.7. The following identity holds [see (30.1.43) above]:

E((Sµ

T −Bµt )2

∣∣FBt

)= (Sµ

t −Bµt )2 + 2

∫ ∞

Sµt −Bµ

t

z(1−Fµ

T−t(z))dz (30.2.4)

for all 0 ≤ t ≤ T where

FµT−t(z) = P(Sµ

T−t ≤ z) = Φ(

z − µ(T − t)√T − t

)− e2µzΦ

(−z − µ(T − t)√T − t

)(30.2.5)

for z ≥ 0 .

Proof. By stationary independent increments of Bµ we have (cf. (30.1.42))

E((Sµ

T −Bµt )2

∣∣FBt

)= E

[(Sµ

t +(

maxt≤s≤T

Bµs − Sµ

t

)+− Bµ

t

)2 ∣∣FBt

](30.2.6)

= E[(

Sµt − Bµ

t +(

maxt≤s≤T

Bµs − Bµ

t − (Sµt − Bµ

t ))+)2 ∣∣FB

t

]=[E(x + (Sµ

T−t − x)+)2]∣∣∣

x=Sµt −Bµ

t

for 0 ≤ t ≤ T given and fixed. Integration by parts gives

E(x + (Sµ

T−t − x)+)2 = E

(x2I(Sµ

T−t ≤ x))

+ E((Sµ

T−t)2I(Sµ

T−t > x))

(30.2.7)

= x2P(SµT−t ≤ x) +

∫ ∞

x

z2 FµT−t(dz)

= x2FµT−t(x) +

(z2(Fµ

T−t(z) − 1))∣∣∣∞

x+ 2

∫ ∞

x

z(1−Fµ

T−t(z))dz

= x2 + 2∫ ∞

x

z(1−Fµ

T−t(z))dz

for all x ≥ 0 . Combining (30.2.6) and (30.2.7) we get (30.2.4). (The identity(30.2.5) is a well-known result of [39, p. 397] and [130, p. 526].)

2. Denoting FµT−t(z) = Fµ(T−t, z) , standard arguments based on the fact

that each stopping time is the limit of a decreasing sequence of discrete stoppingtimes imply that (30.2.4) extends as follows:

E((Sµ

T − Bµτ )2

∣∣FBτ

)= (Sµ

τ − Bµτ )2 + 2

∫ ∞

Sµτ −Bµ

τ

z(1 − Fµ(T − τ, z)

)dz (30.2.8)

for all stopping times τ of Bµ with values in [0, T ] . Setting

Xt = Sµt −Bµ

t (30.2.9)


for t ≥ 0 and taking expectations in (30.2.8) we find that the optimal predictionproblem (30.2.3) is equivalent to the optimal stopping problem

V = inf0≤τ≤T

E

(X2

τ + 2∫ ∞

Xτ

z(1−Fµ(T − τ, z)

)dz

)(30.2.10)

where the infimum is taken over all stopping times τ of X (upon recalling thatthe natural filtrations of Bµ and X coincide). The process X = (Xt)t≥0 isstrong Markov so that (30.2.10) falls into the class of optimal stopping problemsfor Markov processes (cf. Subsection 2.2). The structure of (30.2.10) is complicatedsince the gain process depends on time in a highly nonlinear way.

3. A successful treatment of (30.2.10) requires that the problem be extendedso that the process X can start at arbitrary points in the state space [0,∞) . Forthis, recall that (cf. [84]) the following identity in law holds:

Xlaw= |Y | (30.2.11)

where |Y | = (|Yt|)t≥0 and the process Y = (Yt)t≥0 is a unique strong solution tothe (“bang-bang”) stochastic differential equation

dYt = −µ sign (Yt) dt + dBt (30.2.12)

with Y0 = 0 . Moreover, it is known (cf. [84]) that under Y0 = x in (30.2.12)the process |Y | has the same law as a Brownian motion with drift −µ startedat |x| and reflected at 0 . The infinitesimal operator of |Y | acts on functionsf ∈ C2

b

([0,∞)

)satisfying f ′(0+) = 0 as −µf ′(x) + 1

2f ′′(x) . Since an optimalstopping time in (30.2.10) is the first entry time of the process to a closed set(this follows by general optimal stopping results of Chapter I and will be mademore precise below) it is possible to replace the process X in (30.2.10) by theprocess |Y | . On the other hand, since it is difficult to solve the equation (30.2.12)explicitly so that the dependence of X on x is clearly expressed, we will take adifferent route based on the following fact.

Lemma 30.8. The process Xx = (Xxt )t≥0 defined by

Xxt = x ∨ Sµ

t − Bµt (30.2.13)

is Markov under P making Px = Law(Xx | P) for x ≥ 0 a family of probabilitymeasures on the canonical space

(C+,B(C+)

)under which the coordinate process

X = (Xt)t≥0 is Markov with Px(X0 = x) = 1 .

Proof. Let x ≥ 0 , t ≥ 0 and h > 0 be given and fixed. We then have:

Xxt+h = x ∨ Sµ

t+h − Bµt+h (30.2.14)

= (x ∨ Sµt ) ∨

(max

t≤s≤t+hBµ

s

)− (Bµ

t+h − Bµt ) − Bµ

t

=(x ∨ Sµ

t − Bµt

) ∨ ( maxt≤s≤t+h

Bµs − Bµ

t

)− (Bµ

t+h − Bµt ).


Hence by stationary independent increments of Bµ we get:

E(f(Xx

t+h) | FBt

)= E

(f(z ∨ Sµ

h − Bµh ))∣∣

z=Xxt

(30.2.15)

for every bounded Borel function f . This shows that Xx is a Markov processunder P . Moreover, the second claim follows from (30.2.15) by a basic transfor-mation theorem for integrals upon using that the natural filtrations of B and Xx

coincide. This completes the proof. 4. By means of Lemma 30.8 we can now extend the optimal stopping prob-

lem (30.2.10) where X0 = 0 under P to the optimal stopping problem

V (t, x) = inf0≤τ≤T−t

E t,x

(X2

t+τ + 2∫ ∞

Xt+τ

z(1−Fµ(T−t−τ, z)

)dz

)(30.2.16)

where Xt = x under Pt,x with (t, x) ∈ [0, T ]× [0,∞) given and fixed. Theinfimum in (30.2.16) is taken over all stopping times τ of X .

In view of the fact that Bµ has stationary independent increments, it is norestriction to assume that the process X under Pt,x is explicitly given as

Xxt+s = x ∨ Sµ

s − Bµs (30.2.17)

under P for s ∈ [0, T − t] . Setting

R(t, z) = 1 − F µTt

(z) (30.2.18)

and introducing the gain function

G(t, x) = x2 + 2∫ ∞

x

zR(t, z) dz (30.2.19)

we see that (30.2.16) can be written as follows:

V (t, x) = inf0≤τ≤T−t

E t,xG(t+τ, Xt+τ ) (30.2.20)

for (t, x) ∈ [0, T ]×[0,∞) .

5. The preceding analysis shows that the optimal prediction problem(30.2.3) reduces to solving the optimal stopping problem (30.2.20). Introducingthe continuation set C = (t, x) ∈ [0, T ]× [0,∞) : V (t, x) < G(t, x) and thestopping set D = (t, x) ∈ [0, T ]×[0,∞) : V (t, x) = G(t, x) , we may infer fromgeneral theory of optimal stopping for Markov processes (cf. Chapter I) that theoptimal stopping time in (30.2.20) is given by

τD = inf 0 ≤ s ≤ T − t : (t+s, Xt+s) ∈ D . (30.2.21)

It then follows using (30.2.9) that the optimal stopping time in (30.2.3) is givenby

τ∗ = inf 0 ≤ t ≤ T : (t, Sµt −Bµ

t ) ∈ D . (30.2.22)


b1

D

γ1

T0

b2

γ2

C

t*

u*

C

Figure VIII.6: (The “black-hole” effect.) A computer drawing of the opti-mal stopping boundaries b1 and b2 when µ > 0 is away from 0 .

The problems (30.2.20) and (30.2.3) are therefore reduced to determining D andV (outside D ). We will see below that this task is complicated primarily becausethe gain function G depends on time in a highly nonlinear way. The main aim ofthe present subsection is to expose solutions to the problems formulated.

2. The free-boundary problem. Consider the optimal stopping problem(30.2.20). Recall that the problem reduces to determining the stopping set Dand the value function V outside D . It turns out that the shape of D dependson the sign of µ .

1. The case µ > 0 . It will be shown in the proof below that D = (t, x) ∈[t∗, T )×[0,∞) : b1(t) ≤ x ≤ b2(t) ∪ (T, x) : x ∈ 0,∞) where t∗ ∈ [0, T ) , thefunction t → b1(t) is continuous and decreasing on [t∗, T ] with b1(T ) = 0 , andthe function t → b2(t) is continuous and increasing on [t∗, T ] with b2(T ) = 1/2µ .If t∗ = 0 then b1(t∗) = b2(t∗) , and if t∗ = 0 then b1(t∗) ≤ b2(t∗) . We also haveb1(t) < b2(t) for all t∗ < t ≤ T . See Figures VIII.6+VIII.7.

It follows that the optimal stopping time (30.2.21) can be written as

τD = inf t∗ ≤ t ≤ T : b1(t) ≤ Xt ≤ b2(t) . (30.2.23)

Inserting this expression into (30.2.20) and recalling that C equals Dc in [0, T ]×[0,∞) , we can use Markovian arguments to formulate the following free-boundary


γ1

b2

γ2

C

u*

C

b1

D

T0

C

s*

C

Figure VIII.7: A computer drawing of the optimal stopping boundariesb1 and b2 when µ ≥ 0 is close to 0 .

problem:

Vt − µVx + 12Vxx = 0 in C, (30.2.24)

V (t, b1(t)) = G(t, b1(t)) for t∗ ≤ t ≤ T, (30.2.25)

V (t, b2(t)) = G(t, b2(t)) for t∗ ≤ t ≤ T, (30.2.26)

Vx(t, b1(t)−) = Gx(t, b1(t)) for t∗ ≤ t < T (smooth fit), (30.2.27)

Vx(t, b2(t)+) = Gx(t, b2(t)) for t∗ ≤ t < T (smooth fit), (30.2.28)

Vx(t, 0+) = 0 for 0 ≤ t < T (normal reflection), (30.2.29)

V < G in C, (30.2.30)

V = G in D. (30.2.31)

Note that the conditions (30.2.27)–(30.2.29) will be derived in the proof belowwhile the remaining conditions are obvious.

2. The case µ ≤ 0 . It will be seen in the proof below that D = (t, x) ∈[0, T )×[0,∞) : x ≥ b1(t) ∪ (T, x) : x ∈ [0,∞) where the continuous functiont → b1(t) is decreasing on [z∗, T ] with b1(T ) = 0 and increasing on [0, z∗) forsome z∗ ∈ [0, T ) (with z∗ = 0 if µ = 0 ). See Figures VIII.8+VIII.9.

It follows that the optimal stopping time (30.2.21) can be written as

τD = inf 0 ≤ t ≤ T : Xt ≥ b1(t) . (30.2.32)

Inserting this expression into (30.2.20) and recalling again that C equals Dc

in [0, T ]× [0,∞) , we can use Markovian arguments to formulate the following


T

b1

C

D

γ1

0

C

Figure VIII.8: A computer drawing of the optimal stopping boundary b1

when µ ≤ 0 is close to 0 .

free-boundary problem:

Vt − µVx + 12Vxx = 0 in C, (30.2.33)

V (t, b1(t)) = G(t, b1(t)) for 0 ≤ t ≤ T, (30.2.34)

Vx(t, b1(t)−) = Gx(t, b1(t)) for 0 ≤ t < T (smooth fit), (30.2.35)

Vx(t, 0+) = 0 for 0 ≤ t < T (normal reflection), (30.2.36)

V < G in C, (30.2.37)

V = G in D. (30.2.38)

Note that the conditions (30.2.35) and (30.2.36) can be derived similarly to theconditions (30.2.27) and (30.2.29) above while the remaining conditions are obvi-ous.

3. It will be clear from the proof below that the case µ ≤ 0 may be viewedas the case µ > 0 with b2 ≡ ∞ (and t∗ = 0 ). This is in accordance with thefacts that b2 ↑ ∞ as µ ↓ 0 and the point s∗ < T at which b1(s∗) = b2(s∗)tends to −∞ as µ ↓ 0 . (Note that t∗ equals s∗ ∨ 0 and that extending the timeinterval [0, T ] to negative values in effect corresponds to enlarging the terminalvalue T in the problem (30.2.20) above.) Since the case µ > 0 is richer and moreinteresting we will only treat this case in complete detail. The case µ ≤ 0 can bedealt with analogously and most of the details will be omitted.

4. It will follow from the result of Theorem 30.9 below that the free-boundary problem (30.2.24)–(30.2.31) characterizes the value function V andthe optimal stopping boundaries b1 and b2 in a unique manner. Motivated by


T

b1C

D

γ1

0

C

Figure VIII.9: (The “hump” effect.) A computer drawing of the optimalstopping boundary b1 when µ < 0 is away from 0 .

wider application, however, our main aim will be to express V in terms of b1 andb2 and show that b1 and b2 themselves satisfy a coupled system of nonlinearintegral equations (which may then be solved numerically). Such an approach wasapplied in Subsections 25.2, 26.2 and 27.1 above. The present problem, however,is in many ways different and more complicated. We will nonetheless succeed inproving (as in the cases above with one boundary) that the coupled system ofnonlinear equations derived for b1 and b2 cannot have other solutions. The keyargument in the proof relies upon a local time-space formula (see Subsection 3.5).The analogous facts hold for the free-boundary problem (30.2.33)–(30.2.38) andthe optimal stopping boundary b1 (see Theorem 30.9 below).

3. Solution to the problem. To solve the problems (30.2.3) and (30.2.20) letus introduce the function

H = Gt − µ Gx + 12 Gxx (30.2.39)

on [0, T ]× [0,∞) where G is given in (30.2.19). A lengthy but straightforwardcalculation shows that

H(t, x) =(2µ2(T − t) − 2µx + 3

)Φ(

x − µ(T − t)√T − t

)(30.2.40)

− 2µ√

T − t ϕ

(x − µ(T − t)√

T − t

)− e2µx Φ

(−x − µ(T − t)√T − t

)− 2

(1 + µ2(T − t)

)for (t, x) ∈ [0, T ]×[0,∞) .


Let P = (t, x) ∈ [0, T ]× [0,∞) : H(t, x) ≥ 0 and N = (t, x) ∈ [0, T ]×[0,∞) : H(t, x) < 0 . A direct analysis based on (30.2.40) shows that in thecase µ > 0 we have P = (t, x) ∈ [u∗, T ]× [0,∞) : γ1(t) ≤ x ≤ γ2(t) whereu∗ ∈ [0, T ) , the function t → γ1(t) is continuous and decreasing on [u∗, T ] withγ1(T ) = 0 , and the function t → γ2(t) is continuous and increasing on [u∗, T ]with γ2(T ) = 1/2µ . If u∗ = 0 then γ1(u∗) = γ2(u∗) , and if u∗ = 0 thenγ1(u∗) ≤ γ2(u∗) . We also have γ1(t) < γ2(t) for all u∗ < t ≤ T . See FiguresVIII.6+VIII.7. Similarly, a direct analysis based on (30.2.40) shows that in the caseµ ≤ 0 we have P = (t, x) ∈ [0, T ]× [0,∞) : x ≥ γ1(t) where the continuousfunction t → γ1(t) is decreasing on [w∗, T ] with γ1(T ) = 0 and increasing on[0, w∗) for some w∗ ∈ [0, T ) (with w∗ = 0 if µ = 0 ). See Figures VIII.8+VIII.9.

Below we will make use of the following functions:

J(t, x) = ExG(T, XT−t) (30.2.41)

=∫ ∞

0

ds

∫ s

−∞db G(T, x ∨ s − b) f(T − t, b, s),

K(t, x, t+u, y, z) = Ex

(H(t+u, Xu) I(y < Xu < z)

)(30.2.42)

=∫ ∞

0

ds

∫ s

−∞db H(t+u, x∨ s − b) I(y < x ∨ s − b < z) f(u, b, s),

L(t, x, t+u, y) = Ex

(H(t+u, Xu) I(Xu > y)

)(30.2.43)

=∫ ∞

0

ds

∫ s

−∞db H(t+u, x∨ s − b) I(x ∨ s − b > y) f(u, b, s)

for (t, x) ∈ [0, T ]×[0,∞) , u ≥ 0 and 0 < y < z , where (b, s) → f(t, b, s) is theprobability density function of (Bµ

t , Sµt ) under P given by

f(t, b, s) =

√2π

1t3/2

(2s− b) exp− (2s − b)2

2t+ µ

(b − µt

2

)(30.2.44)

for t > 0 , s ≥ 0 and b ≤ s (see e.g. [107, p. 368]).

The main results of the present subsection may now be stated as follows.

Theorem 30.9. Consider the problems (30.2.3) and (30.2.20). We can then distin-guish the following two cases.

1. The case µµµ >>> 0 . The optimal stopping boundaries in (30.2.20) can becharacterized as the unique solution to the coupled system of nonlinear Volterraintegral equations

J(t, b1(t)) = G(t, b1(t)) +∫ T−t

0

K(t, b1(t), t+u, b1(t+u), b2(t+u)) du, (30.2.45)

J(t, b2(t)) = G(t, b2(t)) +∫ T−t

0

K(t, b2(t), t+u, b1(t+u), b2(t+u)) du (30.2.46)


in the class of functions t → b1(t) and t → b2(t) on [t∗, T ] for t∗ ∈ [0, T )such that the function t → b1(t) is continuous and decreasing on [t∗, T ] , thefunction t → b2(t) is continuous and increasing on [t∗, T ] , and γ1(t) ≤ b1(t) <b2(t) ≤ γ2(t) for all t ∈ (t∗, T ] . The solutions b1 and b2 satisfy b1(T ) = 0 andb2(T ) = 1/2µ , and the stopping time τD from (30.2.23) is optimal in (30.2.20).The stopping time (30.2.22) given by

τ∗ = inf 0 ≤ t ≤ T : b1(t) ≤ Sµt −Bµ

t ≤ b2(t) (30.2.47)

is optimal in (30.2.3). The value function V from (30.2.20) admits the followingrepresentation:

V (t, x) = J(t, x) −∫ T−t

0

K(t, x, t+u, b1(t+u), b2(t+u)) du (30.2.48)

for (t, x) ∈ [0, T ]×[0,∞) . The value V from (30.2.3) equals V (0, 0) in (30.2.48).

2. The case µ ≤µ ≤µ ≤ 0 . The optimal stopping boundary in (30.2.20) can be char-acterized as the unique solution to the nonlinear Volterra integral equation

J(t, b1(t)) = G(t, b1(t)) +∫ T−t

0

L(t, b1(t), t+u, b1(t+u)) du (30.2.49)

in the class of continuous functions t → b1(t) on [0, T ] that are decreasing on[z∗, T ] and increasing on [0, z∗) for some z∗ ∈ [0, T ) and satisfy b1(t) ≥ γ1(t)for all t ∈ [0, T ] . The solution b1 satisfies b1(T ) = 0 and the stopping time τD

from (30.2.32) is optimal in (30.2.20). The stopping time (30.2.22) given by

τ∗ = inf 0 ≤ t ≤ T : Sµt −Bµ

t ≥ b1(t) (30.2.50)

is optimal in (30.2.3). The value function V from (30.2.20) admits the followingrepresentation:

V (t, x) = J(t, x) −∫ T−t

0

L(t, x, t+u, b1(t+u)) du (30.2.51)

for (t, x) ∈ [0, T ]×[0,∞) . The value V from (30.2.3) equals V (0, 0) in (30.2.51).

Proof. The proof will be carried out in several steps. We will only treat the caseµ > 0 in complete detail. The case µ ≤ 0 can be dealt with analogously anddetails in this direction will be omitted. Thus we will assume throughout thatµ > 0 is given and fixed. We begin by invoking a result from general theory ofoptimal stopping for Markov processes (cf. Chapter I).


1. We show that the stopping time τD in (30.2.21) is optimal in the problem(30.2.20). For this, recall that it is no restriction to assume that the process Xunder Pt,x is given explicitly by (30.2.17) under P . Since clearly (t, x) → EG(t+τ, Xx

τ ) is continuous (and thus usc) for each stopping time τ , it follows that(t, x) → V (t, x) is usc (recall that the infimum of usc functions defines an uscfunction). Since (t, x) → G(t, x) is continuous (and thus lsc) by general theory[see Corollary 2.9 (Finite horizon) with Remark 2.10] it follows that τD is optimalin (30.2.20) as claimed. Note also that C is open and D is closed in [0, T ]×[0,∞) .

2. The initial insight into the shape of D is provided by stochastic calculusas follows. By Ito’s formula (page 67) we have

G(t+s, Xt+s) = G(t, x) +∫ s

0

Gt(t+u, Xt+u) du (30.2.52)

+∫ s

0

Gx(t+u, Xt+u) dXt+u +12

∫ s

0

Gxx(t+u, Xt+u) d〈X, X〉t+u

for 0 ≤ s ≤ T − t and x ≥ 0 given and fixed. By the Ito–Tanaka formula (page68), recalling (30.2.11) and (30.2.12) above, we have

Xt = |Yt| = x +∫ t

0

sign (Ys) I(Ys = 0) dYs + 0t (Y ) (30.2.53)

= x − µ

∫ t

0

I(Ys = 0) ds +∫ t

0

sign (Ys) I(Ys = 0) dBs + 0t (Y )

where sign (0) = 0 and 0t (Y ) is the local time of Y at 0 given by

0t (Y ) = P - lim

ε↓012ε

∫ t

0

I(−ε < Ys < ε) ds (30.2.54)

upon using that d〈Y, Y 〉s = ds . It follows from (30.2.53) that

dXt = −µ I(Yt = 0) dt + sign (Yt) I(Yt = 0) dBt + d0t (Y ). (30.2.55)

Inserting (30.2.55) into (30.2.52), using that d〈X, X〉t = I(Yt = 0) dt and P(Yt =0) = 0 , we get

G(t+s, Xt+s) = G(t, x) +∫ s

0

(Gt − µGx + 1

2Gxx

)(t+u, Xt+u) du (30.2.56)

+∫ s

0

Gx(t+u, Xt+u) sign (Yt+u) dBt+u +∫ s

0

Gx(t+u, Xt+u) d0t+u(Y )

= G(t, x) +∫ s

0

H(t+u, Xt+u) du + Ms


where H is given by (30.2.39) above and Ms =∫ s

0 Gx(t+u, Xt+u) sign (Yt+u)dBt+u

is a continuous (local) martingale for s ≥ 0 . In the last identity in (30.2.56) weuse that (quite remarkably) Gx(t, 0) = 0 while d0

t+u(Y ) is concentrated at 0 sothat the final integral in (30.2.56) vanishes.

From the final expression in (30.2.56) we see that the initial insight into theshape of D is gained by determining the sets P and N as introduced following(30.2.40) above. By considering the exit times from small balls in [0, T )× [0,∞)and making use of (30.2.56) with the optional sampling theorem (page 60), we seethat it is never optimal to stop in N . We thus conclude that D ⊆ P .

A deeper insight into the shape of D is provided by the following arguments.Due to the fact that P is bounded by γ1 and γ2 as described following (30.2.40)above, it is readily verified using (30.2.56) above and simple comparison argumentsthat for each x ∈ (0, 1/2µ) there exists t = t(x) ∈ (0, T ) close enough to T suchthat every point (x, u) belongs to D for u ∈ [t, T ] . Note that this fact is fully inagreement with intuition since after starting at (u, x) close to (T, x) there willnot be enough time to reach either of the favourable sets below γ1 or above γ2

to compensate for the loss incurred by strictly positive H via (30.2.56). Thesearguments in particular show that D \ (T, x) : x ∈ R+ is nonempty.

To prove the existence claim above, let x ∈ (0, 1/2µ) be given and fixed. Ifthe claim is not true, then one can find a > 0 small enough such that the rectangleR = [t, T ]×[x− a, x + a] is contained in C ∩ P . Indeed, if there are x′ < x andx′′ > x such that (u′, x′) ∈ D and (u′′, x′′) ∈ D for some u′, u′′ ∈ [t, T ] , thentaking the larger of u′ and u′′ , say u′ , we know that all the points (x, u) foru ≥ u′ must belong to D , violating the fact that the existence claim is not true.The former conclusion follows by combining the facts stated (and independentlyverified) in the first sentence of paragraphs following (30.2.62) and (30.2.63) below.On the other hand, if there is no such x′ < x as above, then we can replace xby x − ε for ε > 0 small enough, find a rectangle R for x − ε in place of x asclaimed above, prove the initial existence claim for x− ε , and then apply the factof the first sentence following (30.2.63) below to derive the initial existence claimfor x . Likewise, if there is no such x′′ > x , we can proceed in a symmetric way toderive the initial existence claim for x . Thus, we may assume that the rectangleR above is contained in C ∩ P for small enough a > 0 .


τa = inf s ≥ 0 : Xt+s /∈ (x − a, x + a) (30.2.57)

under the measure Pt,x . Using equation (30.2.56) together with the optional sam-pling theorem (page 60) we see that


V (t, x) − G(t, x) = Et,x

(G(t+τD, Xt+τD )

)− G(t, x) (30.2.58)

= Et,x

(∫ τD

0

H(t+u, Xt+u) du

)= Et,x

(∫ τa∧(T−t)

0

H(t+u, Xt+u) du

)

+ Et,x

(∫ τD

τa∧(T−t)

H(t+u, Xt+u) du

)≥ c Et,x

(τa ∧ (T−t)

)− d Et,x

(∫ τD

τa∧(T−t)

(1+Xt+u) du

).

since H ≥ c > 0 on R by continuity of H , and H(u, x) ≥ −d (1+x) for(u, x) ∈ [t, T ]×R+ with d > 0 large enough by (30.2.40) above. Considering thefinal term, we find from the Holder inequality that

Et,x

(∫ τD

τa∧(1−t)

(1+Xt+u) du

)(30.2.59)

≤ Et,x

((1+ max

t≤s≤TXs

)((T−t) − τa ∧ (T − t)

))≤√

Et,x

(1+ max

0≤s≤TXs

)2√

Et,x

((T−t)− τa ∧ (T−t)

)2

≤ C

√Et,x

((T−t−τa

)2I(τa < T − t)

)where C is a positive constant independent of t . Since (T − t−τa)2 ≤ (T − t)2 ≤T − t on the set τa < T − t for t < T close to T , we may conclude that

V (t, x) − G(t, x) ≥ c Et,x

((T−t) I(τa > T−t)

)(30.2.60)

− D√

(T−t)Pt,x(τa < T−t)

= (T−t)(

c Pt,x(τa ≥ T−t) − D

√Pt,x(τa < T−t)

T−t

)where c > 0 and D > 0 are constants independent of t . Since clearly Pt,x(τa ≥T−t) → 1 and Pt,x(τa < T−t)/(T−t) → 0 as t ↑ T (the distribution function ofthe first exit time of (−µt+Bt)t≥0 from (−a, a) has a zero derivative at zero), itthus follows that V (t, x)−G(t, x) > 0 for t < T close to T . As this is impossiblewe see that the initial existence claim must be true.

The final insight into the shape of D is obtained by the following fortunatefact:

t → H(t, x) is increasing on [0, T ] (30.2.61)


whenever x ≥ 0 . Indeed, this can be verified by a direct differentiation in (30.2.40)which yields

Ht(t, x) = 2(

x + µ(T − t)(T − t)3/2

)ϕ

(x − µ(T − t)√

T − t

)(30.2.62)

+ 2µ2

(1 − Φ

(x − µ(T − t)√

T − t

))from where one sees that Ht ≥ 0 on [0, T )×[0,∞) upon recalling that µ > 0 byassumption.

We next show that (t1, x) ∈ D implies that (t2, x) ∈ D whenever 0 ≤ t1 ≤t2 ≤ T and x ≥ 0 . For this, assume that (t2, x) ∈ C for some t2 ∈ (t1, T ) . Letτ∗ = τD(t2, x) denote the optimal stopping time for V (t2, x) . Then by (30.2.56)and (30.2.61) using the optional sampling theorem (page 60) we have

V (t1, x) − G(t1, x) ≤ EG(t1+τ∗, Xxτ∗) − G(t1, x) (30.2.63)

= E

(∫ τ∗

0

H(t1+u, Xxu) du

)≤ E

(∫ τ∗

0

H(t2+u, Xxu) du

)= E

(G(t2+τ∗, Xx

τ∗) − G(t2, x))

= V (t2, x) − G(t2, x) < 0.

Hence (t1, x) belongs to C which is a contradiction. This proves the initial claim.

Finally we show that for (t, x1) ∈ D and (t, x2) ∈ D with x1 ≤ x2 in(0,∞) we have (t, z) ∈ D for every z ∈ [x1, x2] . For this, fix z ∈ (x1, x2) andlet τ∗ = τD(t, z) denote the optimal stopping time for V (t, z) . Since (u, x1)and (u, x2) belong to D for all u ∈ [t, T ] we see that τ∗ must be smaller thanor equal to the exit time from the rectangle R with corners at (t, x1) , (t, x2) ,(T, x1) and (T, x2) . However, since H > 0 on R we see from (30.2.56) uponusing the optional sampling theorem (page 60) that V (t, z) > G(t, z) . This showsthat (t, z) cannot belong to C , thus proving the initial claim.

Summarising the facts derived above we can conclude that D equals the setof all (t, x) in [t∗, T ]×[0,∞) with t∗ ∈ [0, T ) such that b1(t) ≤ x ≤ b2(t) , wherethe function t → b1(t) is decreasing on [t∗, T ] with b1(T ) = 0 , the functiont → b2(t) is increasing on [t∗, T ] with b2(T ) = 1/2µ , and γ1(t) ≤ b1(t) ≤ b2(t) ≤γ2(t) for all t ∈ [t∗, T ] . See Figures VIII.6+VIII.7. It follows in particular thatthe stopping time τD from (30.2.23) is optimal in (30.2.20) and the stopping timefrom (30.2.47) is optimal in (30.2.3).

3. We show that V is continuous on [0, T ]×[0,∞) . For this, we will firstshow that x → V (t, x) is continuous on [0,∞) uniformly over t ∈ [0, T ] . Indeed,if x < y in [0,∞) are given and fixed, we then have

V (t, x) − V (t, y) = inf0≤τ≤T−t

EG(t+τ, Xxτ ) − inf

0≤τ≤T−tEG(t+τ, Xy

τ ) (30.2.64)

≥ inf0≤τ≤T−t

E(G(t+τ, Xx

τ ) − G(t+τ, Xyτ ))


for all t ∈ [0, T ] . It is easily verified that x → G(t, x) is increasing so thatx → V (t, x) is increasing on [0,∞) for every t ∈ [0, T ] . Hence it follows from(30.2.64) that

0 ≤ V (t, y) − V (t, x) ≤ sup0≤τ≤T−t

E(G(t+τ, Xy

τ ) − G(t+τ, Xxτ ))

(30.2.65)

for all t ∈ [0, T ] . Using (30.2.19) we find

G(t+τ, Xyτ ) − G(t+τ, Xx

τ ) = (Xyτ )2 − (Xx

τ )2 − 2∫ Xy

τ

Xxτ

zR(t+τ, z) dt (30.2.66)

≤ (Xyτ − Xx

τ

)(Xy

τ + Xxτ + 2c

)=(y ∨ Sµ

τ − x ∨ Sµτ

)(y ∨ Sµ

τ − Bµτ + x ∨ Sµ

τ − Bµτ + 2c

)≤ (y−x)Z

where c = supz≥0 zR(t, z) ≤ ESµT−t ≤ ESµ

T < ∞ by Markov’s inequality andZ = 2(y + 1) + 4 max 0≤t≤T |Bµ

t | + 2c belongs to L1(P) . From (30.2.65) and(30.2.66) we find

0 ≤ V (t, y) − V (t, x) ≤ (y−x)EZ (30.2.67)

for all t ∈ [0, T ] implying that x → V (t, x) is continuous on [0,∞) uniformlyover t ∈ [0, T ] .

To complete the proof of the initial claim it is sufficient to show that t →V (t, x) is continuous on [0, T ] for each x ∈ [0,∞) given and fixed. For this, fixx in [0,∞) and t1 < t2 in [0, T ] . Let τ1 = τD(t1, x) and τ2 = τD(t2, x) beoptimal for V (t1, x) and V (t2, x) respectively. Setting τε

1 = τ1 ∧ (T − t2) withε = t2 − t1 we have

E(G(t2+τ2, X

xτ2

) − G(t1+τ2, Xxτ2

))

(30.2.68)≤ V (t2, x) − V (t1, x)

≤ E(G(t2+τε

1 , Xxτε1) − G(t1+τ1, X

xτ1

)).

Note that

Gt(t, x) = −2∫ ∞

x

zfµT−t(z) dz (30.2.69)

where fµT−t(z) = (dFµ

T−t/dz)(z) so that

|Gt(t, x)| ≤ 2∫ ∞

0

zfµT−t(z) dz = 2 ESµ

T−t ≤ 2 ESµT (30.2.70)

for all t ∈ [0, T ] . Hence setting β = 2 ESµT by the mean value theorem we get

|G(u2, x) − G(u1, x)| ≤ β(u2 −u1) (30.2.71)


for all u1 < u2 in [0, T ] . Using (30.2.71) in (30.2.68) upon subtracting and addingG(t1+τ1, X

xτε1) we obtain

−β(t2− t1) ≤ V (t2, x) − V (t1, x) (30.2.72)

≤ 2β(t2 − t1) + E(G(t1+τ1, X

xτε1) − G(t1+τ1, X

xτ1

)).

Note thatGx(t, x) = 2xFµ

T−t(x) ≤ 2x (30.2.73)

so that the mean value theorem implies

|G(t1+τ1, Xxτε1) − G(t1+τ1, X

xτ1

)| = |Gx(t1+τ1, ξ)| |Xxτε1−Xx

τ1| (30.2.74)

≤ 2(Xx

τε1∨ Xx

τ1

) |Xxτε1−Xx

τ1|

where ξ lies between Xxτε1

and Xxτ1

. Since Xxτ is dominated by the random

variable x + 2 max 0≤t≤T |Bµt | which belongs to L1(P) for every stopping time

τ , letting t2 − t1 → 0 and using that τε1 − τ1 → 0 we see from (30.2.72) and

(30.2.74) that V (t2, x) − V (t1, x) → 0 by dominated convergence. This showsthat t → V (t, x) is continuous on [0, T ] for each x ∈ [0,∞) , and thus V iscontinuous on [0, T ]×[0,∞) as claimed. Standard arguments based on the strongMarkov property and classic results from PDEs (cf. Chapter III) show that V isC1,2 on C and satisfies (30.2.24). These facts will be freely used below.

4. We show that x → V (t, x) is differentiable at bi(t) for i = 1, 2 andthat Vx(t, bi(t)) = Gx(t, bi(t)) for t ∈ [t∗, T ) . For this, fix t ∈ [t∗, T ) and setx = b2(t) (the case x = b1(t) can be treated analogously). We then have

V (t, x+ε) − V (t, x)ε

≤ G(t, x+ε) − G(t, x)ε

(30.2.75)

for all ε > 0 . Letting ε ↓ 0 in (30.2.75) we find

lim supε↓0

V (t, x+ε) − V (t, x)ε

≤ Gx(t, x). (30.2.76)

Let τε = τD(t, x + ε) be optimal for V (t, x + ε) . Then by the mean valuetheorem we have

V (t, x+ε) − V (t, x)ε

≥ 1ε

(EG(t+τε, X

x+ετε

) − EG(t+τε, Xxτε

))

(30.2.77)

=1ε

E(Gx(t+τε, ξε)(Xx+ε

τε−Xx

τε))

where ξε lies between Xxτε

and Xx+ετε

. Using that t → b2(t) is increasing andthat t → λt is a lower function for B at 0+ for every λ ∈ R , it is possible


to verify that τε → 0 as ε ↓ 0 . Hence it follows that ξε → x as ε ↓ 0 so thatGx(t+τε, ξε) → Gx(t, x) as ε ↓ 0 . Moreover, using (30.2.73) we find

Gx(t+τε, ξε) ≤ 2ξε ≤ 2Xx+ετε

= 2((x+ε) ∨ Sµ

τε− Bµ

τε

)(30.2.78)

≤ 2(x + ε + 2 max

0≤t≤T|Bµ

t |)

where the final expression belongs to L1(P) (recall also that Gx ≥ 0 ). Finally,we have

1ε

(Xx+ε

τε−Xx

τε

)= 1

ε

((x+ε) ∨ Sµ

τε− x ∨ Sµ

τε

)→ 1 (30.2.79)

when ε ↓ 0 as well as0 ≤ 1

ε

(Xx+ε

τε−Xx

τε

) ≤ 1 (30.2.80)

for all ε > 0 . Letting ε ↓ 0 in (30.2.77) and using (30.2.78)–(30.2.80), we mayconclude that

lim infε↓0

V (t, x+ε) − V (t, x)ε

≥ Gx(t, x) (30.2.81)

by dominated convergence. Combining (30.2.76) and (30.2.81) we see that x →V (t, x) is differentiable at b2(t) with Vx(t, b2(t)) = Gx(t, b2(t)) as claimed. Anal-ogously one finds that x → V (t, x) is differentiable at b1(t) with Vx(t, b1(t)) =Gx(t, b1(t)) and further details of this derivation will be omitted.

A small modification of the proof above shows that x → V (t, x) is C1 atb2(t) . Indeed, let τδ = τD(t, x+δ) be optimal for V (t, x+δ) where δ > 0 is givenand fixed. Instead of (30.2.75) above we have by the mean value theorem that

V (t, x+δ+ε)−V (t, x+δ)ε

≤ 1ε

(EG(t+τδ, X

x+δ+ετδ

)− EG(t+τδ, Xx+δτδ

))(30.2.82)

=1ε

E(Gx(t+τδ, ηε)

(Xx+δ+ε

τδ−Xx+δ

τδ

))where ηε lies between Xx+δ

τδand Xx+δ+ε

τδfor ε > 0 . Clearly ηε → Xx+δ

τδas

ε ↓ 0 . Letting ε ↓ 0 in (30.2.82) and using the same arguments as in (30.2.78)–(30.2.80) we can conclude that

Vx(t, x+δ) ≤ EGx(t+τδ, Xx+δτδ

). (30.2.83)

Moreover, in exactly the same way as in (30.2.77)–(30.2.81) we find that the reverseinequality in (30.2.83) also holds, so that we have

Vx(t, x+δ) = EGx(t+τδ, Xx+δτδ

). (30.2.84)

Letting δ ↓ 0 in (30.2.84), recalling that τδ → 0 , and using the same argumentsas in (30.2.78), we find by dominated convergence that

limδ↓0

Vx(t, x+δ) = Gx(t, x) = Vx(t, x). (30.2.85)


Thus x → V (t, x) is C1 at b2(t) as claimed. Similarly one finds that x →V (t, x) is C1 at b1(t) with Vx(t, b1(t)+) = Gx(t, b1(t)) and further details of thisderivation will be omitted. This establishes the smooth fit conditions (30.2.27)–(30.2.28) and (30.2.35) above.

5. We show that t → b1(t) and t → b2(t) are continuous on [t∗, T ] . Againwe only consider the case of b2 in detail, since the case of b1 can be treatedsimilarly. Note that the same proof also shows that b2(T−) = 1/2µ and thatb1(T−) = 0 .

Let us first show that b2 is right-continuous. For this, fix t ∈ [t∗, T ) andconsider a sequence tn ↓ t as n → ∞ . Since b2 is increasing, the right-hand limitb2(t+) exists. Because (tn, b2(tn)) belongs to D for all n ≥ 1 , and D is closed,it follows that (t, b2(t+)) belongs to D . Hence by (30.2.23) we may conclude thatb2(t+) ≤ b2(t) . Since the fact that b2 is increasing gives the reverse inequality, itfollows that b2 is right-continuous as claimed.

Let us next show that b2 is left-continuous. For this, suppose that thereexists t ∈ (t∗, T ) such that b2(t−) < b2(t) . Fix a point x ∈ (b2(t−), b2(t)) andnote by (30.2.28) that we have

V (s, x) − G(s, x) =∫ x

b2(s)

∫ y

b2(s)

(Vxx(s, z) − Gxx(s, z)

)dz dy (30.2.86)

for any s ∈ (t∗, t) . By (30.2.24) and (30.2.39) we find that

12 (Vxx −Gxx) = Gt − Vt + µ(Vx −Gx) − H. (30.2.87)

From (30.2.63) we derive the key inequality

Vt(t, x) ≥ Gt(t, x) (30.2.88)

for all (t, x) ∈ [0, T )×[0,∞) . Inserting (30.2.87) into (30.2.86) and using (30.2.88)and (30.2.26) we find

V (s, x) − G(s, x) ≤∫ x

b2(s)

∫ y

b2(s)

2(µ(Vx −Gx)(s, z) − H(s, z)

)dz dy (30.2.89)

=∫ x

b2(s)

2µ(V (s, y)−G(s, y)

)dy −

∫ x

b2(s)

∫ y

b2(s)

2H(s, z) dz dy

≤ −∫ x

b2(s)

∫ y

b2(s)

2H(s, z) dz dy

for any s ∈ (t∗, t) . From the properties of the function γ2 it follows that thereexists s∗ < t close enough to t such that (s, z) belongs to P for all s ∈ [s∗, t)and z ∈ [b2(s), x] . Moreover, since H is continuous and thus attains its infimum


on a compact set, it follows that 2H(s, z) ≥ m > 0 for all s ∈ [s∗, t) andz ∈ [b2(s), x] . Using this fact in (30.2.89) we get

V (s, x) − G(s, x) ≤ −m(x− b2(s))2

2< 0 (30.2.90)

for all s ∈ [s∗, t) . Letting s ↑ t in (30.2.90) we conclude that V (t, x) < G(t, x)violating the fact that (t, x) ∈ D . This shows that b2 is left-continuous and thuscontinuous. The continuity of b1 is proved analogously.

6. We show that the normal reflection condition (30.2.29) holds. For this,note first since x → V (t, x) is increasing on [0,∞) that Vx(t, 0+) ≥ 0 for allt ∈ [0, T ) (note that the limit exists since V is C1,2 on C ). Suppose that thereexists t ∈ [0, T ) such that Vx(t, 0+) > 0 . Recalling that V is C1,2 on C sothat t → Vx(t, 0+) is continuous on [0, T ) , we see that there exists δ > 0 suchthat Vx(s, 0+) ≥ ε > 0 for all s ∈ [t, t + δ] with t + δ < T . Setting τδ = τD ∧ δit follows by the Ito–Tanaka formula (as in (30.2.56) above) upon using (30.2.24)and the optional sampling theorem (recall (30.2.83) and (30.2.73) for the latter)that we have

E t,0

(V (t+τδ, Xt+τδ

))

= V (t, 0) + E t,0

(∫ τδ

0

Vx(t+u, Xt+u) d0t+u(Y )

)(30.2.91)

≥ V (t, 0) + ε E t,0

(0t+τδ

(Y )).

Since (V (t+s∧τD, Xt+s∧τD)0≤s≤T−t is a martingale under Pt,0 by general theoryof optimal stopping for Markov processes (cf. Chapter I) we see from (30.2.91) thatEt,0

(0t+τδ

(Y ))

must be equal to 0 . Since however properties of the local timeclearly exclude this, we must have V (t, 0+) equal to 0 as claimed in (30.2.29)above.

7. We show that V is given by the formula (30.2.48) and that b1 and b2

solve the system (30.2.45)–(30.2.46). For this, note that by (30.2.24) and (30.2.88)we have 1

2Vxx = −Vt +µVx ≤ −Gt +µVx in C . It is easily verified using (30.2.73)and (30.2.83) that Vx(t, x) ≤ M/2µ for all t ∈ [0, T ) and all x ∈ [0, (1/2µ)+1]with some M > 0 large enough. Using this inequality in the previous inequalitywe get Vxx ≤ −Gt + M in A = C ∩ ([0, T ) × [0, (1/2µ)+1]) . Setting h(t, x) =∫ x

0

∫ y

0 (−Gt(t, z) + M) dz dy we easily see that h is C1,2 on [0, T ) × [0,∞) andthat hxx = −Gt + M . Thus the previous inequality reads Vxx ≤ hxx in A ,and setting F = V − h we see that x → F (t, x) is concave on [0, b1(t)] and[b2(t), (1/2µ)+1] for t ∈ [t∗, T ) . We also see that F is C1,2 on C and D = (t, x) ∈ [t∗, T ) × [0,∞) : b1(t) < x < b2(t) since both V and G are so.Moreover, it is also clear that Ft − µFx + 1

2Fxx is locally bounded on C ∪ D

in the sense that the function is bounded on K ∩ (C ∪ D) for each compactset K in [0, T )× [0,∞) . Finally, we also see using (30.2.27) and (30.2.28) thatt → Fx(t, bi(t)∓) = Vx(t, bi(t)∓) − hx(t, bi(t)∓) = Gx(t, bi(t)) − hx(t, bi(t)) iscontinuous on [t∗, T ) since bi is continuous for i = 1, 2 .


Since the previous conditions are satisfied we know that the local time-spaceformula (cf. Subsection 3.5) can be applied to F (t+s, Xt+s) . Since h is C1,2

on [0, T )×[0,∞) we know that the Ito–Tanaka–Meyer formula (page 68) can beapplied to h(t+s, Xt+s) as in (30.2.56) above (upon noting that hx(t, 0+) = 0) .Adding the two formulae, using in the former that Fx(t, 0+) = −hx(t, 0+) = 0since Vx(t, 0+) = 0 by (30.2.29) above, we get

V (t+s, Xt+s) = V (t, x) (30.2.92)

+∫ s

0

(Vt − µVx + 1

2Vxx

)(t+u, Xt+u)

× I(Xt+u /∈ b1(t+u), b2(t+u)) du

+∫ s

0

Vx(t+u, Xt+u) sign (Yt+u) I(Xt+u /∈ b1(t+u), b2(t+u))dBt+u

+2∑

i=1

∫ s

0

(Vx(t+u, Xt+u+) − Vx(t+u, Xt+u−)

)× I

(Xt+u = bi(t+u)

)dbi

t+u(X)

for t ∈ [0, T ) and x ∈ [0,∞) . Making use of (30.2.24)+(30.2.31) in the firstintegral and (30.2.27)–(30.2.28) in the final integral (which consequently vanishes),we obtain

V (t+s, Xt+s) = V (t, x) (30.2.93)

+∫ s

0

H(t+u, Xt+u)I(b1(t+u) < Xt+u < b2(t+u)

)du + Ms

for t ∈ [0, T ) and x ∈ [0,∞) where Ms =∫ s

0Vx(t+u, Xt+u) dBt+u is a continuous

(local) martingale for s ≥ 0 .

Setting s = T − t , using that V (T, x) = G(T, x) for all x ≥ 0 , and takingthe Pt,x -expectation in (30.2.93), we find by the optional sampling theorem (page60) that

V (t, x) = E t,xG(T, XT ) (30.2.94)

−∫ T − t

0

E t,x

[H(t+u, Xt+u)I

(b1(t+u) < Xt+u < b2(t+u)

)]du

for t ∈ [0, T ) and x ∈ [0,∞) . Making use of (30.2.41) and (30.2.42) we see that(30.2.94) is the formula (30.2.48). Moreover, inserting x = bi(t) in (30.2.94) andusing that V (t, bi(t)) = G(t, bi(t)) for i = 1, 2 we see that b1 and b2 satisfy thesystem (30.2.45)–(30.2.46) as claimed.

8. We show that b1 and b2 are the unique solution to the system (30.2.45)–(30.2.46) in the class of continuous functions t → b1(t) and t → b2(t) on [t∗, T ]for t∗ ∈ [0, T ) such that γ1(t) ≤ b1(t) < b2(t) ≤ γ2(t) for all t ∈ (t∗, T ] . Note


that there is no need to assume that b1 is decreasing and b2 is increasing asestablished above. The proof of uniqueness will be presented in the final threesteps of the main proof below.

9. Let c1 : [t∗, T ] → R and c2 : [t∗, T ] → R be a solution to the system(30.2.45)–(30.2.46) for t∗ ∈ [0, T ) such that c1 and c2 are continuous and satisfyγ1(t) ≤ c1(t) < c2(t) ≤ γ2(t) for all t ∈ (t∗, T ] . We need to show that these c1 andc2 must then be equal to the optimal stopping boundaries b1 and b2 respectively.

Motivated by the derivation (30.2.92)–(30.2.94) which leads to the formula(30.2.48), let us consider the function U c : [0, T )×[0,∞) → R defined as follows:

U c(t, x) = E t,xG(T, XT ) (30.2.95)

−∫ T−t

0

E t,x

(H(t+u, Xt+u) I

(c1(t+u) < Xt+u < c2(t+u)

))du

for (t, x) ∈ [0, T ) × [0,∞) . In terms of (30.2.41) and (30.2.42) note that U c isexplicitly given by

U c(t, x) = J(t, x) −∫ T−t

0

K(t, x, t+u, c1(t+u), c2(t+u)

)du (30.2.96)

for (t, x) ∈ [0, T )× [0,∞) . Observe that the fact that c1 and c2 solve thesystem (30.2.45)–(30.2.46) means exactly that U c(t, ci(t)) = G(t, ci(t)) for allt ∈ [t∗, T ] and i = 1, 2 . We will moreover show that U c(t, x) = G(t, x) for allx ∈ [c1(t), c2(t)] with t ∈ [t∗, T ] . This is the key point in the proof (cf. Subsections25.2, 26.2, 27.1) that can be derived using martingale arguments as follows.

If X = (Xt)t≥0 is a Markov process (with values in a general state space)and we set F (t, x) = ExG(XT−t) for a (bounded) measurable function G withP(X0 = x) = 1 , then the Markov property of X implies that F (t, Xt) is a martin-gale under Px for 0 ≤ t ≤ T . Similarly, if we set F (t, x) = Ex

( ∫ T−t

0H(Xs) ds

)for a (bounded) measurable function H with P(X0 = x) = 1 , then the Markovproperty of X implies that F (t, Xt) +

∫ t

0 H(Xs) ds is a martingale under Px for0 ≤ t ≤ T .

Combining the two martingale facts applied to the time-space Markov process(t+s, Xt+s) instead of Xs , we find that

U c(t+s, Xt+s) −∫ s

0

H(t+u, Xt+u) I(c1(t+u) < Xt+u < c2(t+u)

)du (30.2.97)

is a martingale under Pt,x for 0 ≤ s ≤ T − t . We may thus write

U c(t+s, Xt+s) −∫ s

0


)du (30.2.98)

= U c(t, x) + Ns


where (Ns)0≤s≤T−t is a martingale under Pt,x . On the other hand, we know from(30.2.56) that

G(t+s, Xt+s) = G(t, x) +∫ s

0

H(t+u, Xt+u) du + Ms (30.2.99)

where Ms =∫ s

0 Gx(t+u, Xt+u) sign (Yt+u) dBt+u is a continuous (local) martin-gale under Pt,x for 0 ≤ s ≤ T − t .

For x ∈ [c1(t), c2(t)] with t ∈ [t∗, T ] given and fixed, consider the stoppingtime

σc = inf 0 ≤ s ≤ T − t : Xt+s ≤ c1(t+s) or Xt+s ≥ c2(t+s) (30.2.100)

under Pt,x . Using that U c(t, ci(t)) = G(t, ci(t)) for all t ∈ [t∗, T ] (since c1 andc2 solve the system (30.2.45)–(30.2.46) as pointed out above) and that U c(T, x) =G(T, x) for all x ≥ 0 , we see that U c(t+σc, Xt+σc) = G(t+σc, Xt+σc) . Hencefrom (30.2.98) and (30.2.99) using the optional sampling theorem (page 60) wefind

U c(t, x) = E t,xU c(t+σc, Xt+σc) (30.2.101)

− E t,x

[ ∫ σc

0


)du

]= E t,xG(t+σc, Xt+σc) − E t,x

[ ∫ σc

0

H(t+u, Xt+u) du

]= G(t, x)

since Xt+u ∈ (c1(t+u), c2(t+u)) for all u ∈ [0, σc) . This proves that U c(t, x) =G(t, x) for all x ∈ [c1(t), c2(t)] with t ∈ [t∗, T ] as claimed.

10. We show that U c(t, x) ≥ V (t, x) for all (t, x) ∈ [0, T ]×[0,∞) . For this,consider the stopping time

τc = inf 0 ≤ s ≤ T − t : c1(t+s) ≤ Xt+s ≤ c2(t+s) (30.2.102)

under Pt,x with (t, x) ∈ [0, T ]× [0,∞) given and fixed. The same arguments asthose following (30.2.100) above show that U c(t + τc, Xt+τc) = G(t + τc, Xt+τc) .Inserting τc instead of s in (30.2.98) and using the optional sampling theorem(page 60), we get

U c(t, x) = Et,x

(U c(t+τc, Xt+τc)

)= Et,x

(G(t+τc, Xt+τc)

) ≥ V (t, x) (30.2.103)

proving the claim.

11. We show that c1 ≤ b1 and c2 ≥ b2 on [t∗, T ] . For this, suppose thatthere exists t ∈ [t∗, T ) such that c2(t) < b2(t) and examine first the case whenc2(t) > b1(t) . Choose a point x ∈ (b1(t)∨c1(t), c2(t)] and consider the stoppingtime

σb = inf 0 ≤ s ≤ T − t : Xt+s ≤ b1(t+s) or Xt+s ≥ b2(t+s) (30.2.104)


under Pt,x . Inserting σb in the place of s in (30.2.93) and (30.2.98) and usingthe optional sampling theorem (page 60), we get

Et,x

(V (t+σb, Xt+σb

))

= V (t, x) + E t,x

(∫ σb

0

H(t+u, Xt+u) du

), (30.2.105)

Et,x

(U c(t+σb, Xt+σb

))

= U c(t, x) (30.2.106)

+ Et,x

(∫ σb

0


)du

).

Since U c ≥ V and V (t, x) = U c(t, x) = G(t, x) for x ∈ [b1(t)∨c1(t), b2(t)∧c2(t)]with t ∈ [t∗, T ] , it follows from (30.2.105) and (30.2.106) that

E t,x

[ ∫ σb

0

H(t+u, Xt+u)I(Xt+u≤ c1(t+u) or Xt+u≥ c2(t+u)

)du

]≤ 0. (30.2.107)

Due to the fact that H(t + u, Xt+u) > 0 for u ∈ [0, σb) we see by the continuityof bi and ci for i = 1, 2 that (30.2.107) is not possible. Thus under c2(t) < b2(t)we cannot have c2(t) > b1(t) . If however c2(t) ≤ b1(t) , then due to the factsthat b1 is decreasing with b1(T ) = 0 and c2(T ) > 0 , there must exist u ∈ (t, T )such that c2(u) ∈ (b1(u), b2(u)) . Applying then the preceding arguments at timeu instead of time t , we again arrive at a contradiction. Hence we can concludethat c2(t) ≥ b2(t) for all t ∈ [t∗, T ] . In exactly the same way (or by symmetry)one can derive that c1(t) ≤ b1(t) for t ∈ [t∗, T ] completing the proof of the initialclaim.

12. We show that c1 must be equal to b1 and c2 must be equal to b2 .For this, let us assume that there exists t ∈ [t∗, T ) such that c1(t) < b1(t) orc2(t) > b2(t) . Pick an arbitrary point x from (c1(t), b1(t)) or (b2(t), c2(t)) andconsider the stopping time τD from (30.2.23) under Pt,x . Inserting τD insteadof s in (30.2.93) and (30.2.98), and using the optional sampling theorem (page60), we get

E t,x

(G(t+τD, Xt+τD)

)= V (t, x), (30.2.108)

E t,x

(G(t+τD, Xt+τD)

)= U c(t, x) (30.2.109)

+ E t,x

(∫ τD

0


)du

)where we also use that V (t+ τD, Xt+τD ) = U c(t+ τD, Xt+τD) = G(t+ τD, Xt+τD )upon recalling that c1 ≤ b1 and c2 ≥ b2 , and U c = G either between c1 andc2 or at T . Since U c ≥ V we see from (30.2.108) and (30.2.109) that

E t,x

(∫ τD

0


)du

)≤ 0. (30.2.110)

Due to the fact that H(t+u, Xt+u) > 0 for Xt+u ∈ (c1(t+u), c2(t+u)) we seefrom (30.2.110) by the continuity of bi and ci for i = 1, 2 that such a point


(t, x) cannot exist. Thus ci must be equal to bi for i = 1, 2 and the proof iscomplete.

Remark 30.10. The following simple method can be used to solve the system(30.2.45)–(30.2.46) numerically. Better methods are needed to achieve higher pre-cision around the singularity point t = T and to increase the speed of calculation.These issues are worthy of further consideration.

Set tk = kh for k = 0, 1, . . . , n where h = T/n and denote (recalling(30.2.41) and (30.2.42) above for more explicit expressions)

I(t, bi(t)) = J(t, bi(t)) − G(t, bi(t)) (30.2.111)

= E bi(t)

(G(T, XT−t)

)− G(t, bi(t)), (30.2.112)

K(t, bi(t), t+u, b1(t+u), b2(t+u)) (30.2.113)

= E bi(t)

(H(t+u, Xt+u) I

(b1(t+u) < Xt+u < b2(t+u)

))for i = 1, 2 . Note that K always depends on both b1 and b2 .

The following discrete approximation of the integral equations (30.2.45) and(30.2.46) is then valid:

I(tk, bi(tk)) =n−1∑j=k

K(tk, bi(tk), tj+1, b1(tj+1), b2(tj+1)

)h (30.2.114)

for k = 0, 1, . . . , n − 1 where i = 1, 2 . Setting k = n − 1 with b1(tn) = 0and b2(tn) = 1/2µ we can solve the system (30.2.114) for i = 1, 2 numeri-cally and get numbers b1(tn−1) and b2(tn−1) . Setting k = n − 2 and usingvalues b1(tn−1) , b1(tn) , b2(tn−1) , b2(tn) we can solve (30.2.114) numericallyand get numbers b1(tn−2) and b2(tn−2) . Continuing the recursion we obtaingi(tn) , gi(tn−1) , . . . , gi(t1) , gi(t0) as an approximation of the optimal bound-ary bi at the points T , T −h , . . . , h , 0 for i = 1, 2 (see Figures VIII.6+VIII.7).The equation (30.2.49) can be treated analogously (see Figures VIII.8+VIII.9).

Notes. Stopping a stochastic process as close as possible to its ultimate maxi-mum is an undertaking of great practical and theoretical interest (e.g. in “financialengineering”). Mathematical problems of this type may be referred to as optimalprediction problems. Variants of these problems have appeared in the past underdifferent names (the optimal selection problem, the best choice problem, the secre-tary problem, the house selling problem) concerning which the older papers [108],[76], [21], [86] are interesting to consult. Most of this work has been done in thecase of discrete time.

The case of continuous time (Subsection 30.1) has been studied in the paper[85] when the process is a standard Brownian motion (see also [152] for a related


problem). This hypothesis leads to an explicit solution of the problem using themethod of time change. Motivated by wider applications, our aim in Subsection30.2 (following [42]) is to continue this study when the process is a standardBrownian motion with drift. It turns out that this extension is not only far frombeing routine, but also requires a different line of argument to be developed, whichin turn is applicable to a broader class of diffusions and Markov processes. Theidentity (30.1.50) was observed by Urusov [213].

The continuation set of the problem turns out to be “humped” when the driftis negative. This is rather unexpected and indicates that the problem is stronglytime dependent. The most surprising discovery revealed by the solution, however,is the existence of a nontrivial stopping set (a “black hole” as we call it) whenthe drift is positive. This fact is not only counter-intuitive but also has importantpractical implications. For example, in a growing economy where the appreciationrate of a stock price is strictly positive, any financial strategy based on optimalprediction of the ultimate maximum should be thoroughly re-examined in the lightof this new phenomenon.

Bibliography

[1] Abramowitz, M. and Stegun, I. A. (eds.) (1964). Handbook of Math-ematical Functions with Formulas, Graphs, and Mathematical Tables. U.S.Department of Commerce, Washington.

[2] Alili, L. and Kyprianou, A. E. (2005). Some remarks on first passage ofLevy processes, the American put and pasting principles. Ann. Appl. Probab.15 (2062–2080).

[3] Alvarez, L. H. R. (2001). Reward functionals, salvage values, and optimalstopping. Math. Methods Oper. Res. 54 (315–337).

[4] Andre, D. (1887). Solution directe du probleme resolu par M. Bertrand.C. R. Acad. Sci. Paris 105 (436–437).

[5] Arrow, K. J., Blackwell, D. and Girshick, M. A. (1949). Bayes andminimax solutions of sequential decision problems. Econometrica 17 (213–244).

[6] Azema, J. and Yor, M. (1979). Une solution simple au probleme de Sko-rokhod. Sem. Probab. XIII, Lecture Notes in Math. 721, Springer, Berlin(90–115).

[7] Bachelier, L. (1900). Theorie de la speculation. Ann. Sci. Ecole Norm.Sup. (3) 17 (21–86). English translation “Theory of Speculation” in “TheRandom Character of Stock Market Prices”, MIT Press, Cambridge, Mass.1964 (ed. P. H. Cootner) (17–78).

[8] Barrieu, P., Rouault, A. and Yor, M. (2004). A study of the Hartman–Watson distribution motivated by numerical problems related to the pricingof asian options. J. Appl. Probab. 41 (1049–1058).

[9] Bayraktar, E., Dayanik, S. and Karatzas, I. (2006). Adaptive Poissondisorder problem. To appear in Ann. Appl. Probab.

[10] Bather, J. A. (1962). Bayes procedures for deciding the sign of a normalmean. Proc. Cambridge Philos. Soc. 58 (599–620).

478 Bibliography

[11] Bather, J. (1970). Optimal stopping problems for Brownian motion. Adv.in Appl. Probab. 2 (259–286).

[12] Beibel, M. (2000). A note on sequential detection with exponential penaltyfor the delay. Ann. Statist. 28 (1696–1701).

[13] Beibel, M. and Lerche, H. R. (1997). A new look at optimal stoppingproblems related to mathematical finance. Empirical Bayes, sequential anal-ysis and related topics in statistics and probability (New Brunswick, NJ,1995). Statist. Sinica 7 (93–108).

[14] Beibel, M. and Lerche, H. R. (2002). A note on optimal stopping ofregular diffusions under random discounting. Theory Probab. Appl. 45 (547–557).

[15] Bellman, R. (1952). On the theory of dynamic programming. Proc. Natl.Acad. Sci. USA 38 (716–719).

[16] Bellman, R. (1957). Dynamic Programming. Princeton Univ. Press,Princeton.

[17] Bhat, B. R. (1988). Optimal properties of SPRT for some stochastic pro-cesses. Contemp. Math. 80 (285–299).

[18] Blackwell, D. and Girshick M. A. (1954). Theory of Games and Sta-tistical Decisions. Wiley, New York; Chapman & Hall, London.

[19] Bliss, G. A. (1946). Lectures on the Calculus of Variations. Univ. ChicagoPress, Chicago.

[20] Bolza, O. (1913). Uber den “Anormalen Fall” beim Lagrangeschen undMayerschen Problem mit gemischten Bedingungen und variablen Endpunk-ten. Math. Ann. 74 (430–446).

[21] Boyce, W. M. (1970). Stopping rules for selling bonds. Bell J. Econom.Management Sci. 1 (27–53).

[22] Breakwell, J. and Chernoff H. (1964). Sequential tests for the meanof a normal distribution. II (Large t ). Ann. Math. Statist. 35 (162–173).

[23] Brekke, K. A. and Øksendal, B. (1991). The high contact principle asa sufficiency condition for optimal stopping. Stochastic Models and OptionValues (Loen, 1989), Contrib. Econom. Anal. 200, North-Holland (187–208).

[24] Buonocore, A., Nobile, A. G. and Ricciardi, L. M. (1987). A new in-tegral equation for the evaluation of first-passage-time probability densities.Adv. in Appl. Probab. 19 (784–800).

Bibliography 479

[25] Burkholder, D. L. (1991). Explorations in martingale theory and its ap-plications. Ecole d’Ete de Probabilites de Saint-Flour XIX—1989, LectureNotes in Math. 1464, Springer-Verlag, Berlin (1–66).

[26] Carlstein, E., Muller, H.-G. and Siegmund, D. (eds.) (1994). Change-Point Problems. IMS Lecture Notes Monogr. Ser. 23. Institute of Mathemat-ical Statistics, Hayward.

[27] Carr, P., Jarrow, R. and Myneni, R. Alternative characterizations ofAmerican put options. Math. Finance 2 (78–106).

[28] Chapman, S. (1928). On the Brownian displacements and thermal diffusionof grains suspended in a non-uniform fluid. Proc. R. Soc. Lond. Ser. A 119(34–54).

[29] Chernoff, H. (1961). Sequential tests for the mean of a normal distribu-tion. Proceedings of the 4th Berkeley Symposium on Mathematical Statisticsand Probability. Vol. I. Univ. California Press, Berkeley, Calif. (79–91).

[30] Chernoff, H. (1968). Optimal stochastic control. Sankhya Ser. A 30 (221–252).

[31] Chow, Y. S., Robbins, H. and Siegmund, D. (1971). Great Expectations:The Theory of Optimal Stopping. Houghton Mifflin, Boston, Mass.

[32] Cox, D. C. (1984). Some sharp martingale inequalities related to Doob’sinequality. IMS Lecture Notes Monograph Ser. 5 (78–83).

[33] Darling, D. A., Liggett, T. and Taylor, H. M. (1972). Optimal stop-ping for partial sums. Ann. Math. Statist. 43 (1363–1368).

[34] Davis, B. (1976). On the Lp norms of stochastic integrals and other mar-tingales. Duke Math. J. 43 (697–704).

[35] Davis, M. H. A. (1976). A note on the Poisson disorder problem. Math.Control Theory, Proc. Conf. Zakopane 1974, Banach Center Publ. 1 (65–72).

[36] Davis, M. H. A. and Karatzas, I. (1994). A deterministic approach to op-timal stopping. Probability, statistics and optimisation. Wiley Ser. Probab.Math. Statist., Wiley, Chichester (455–466).

[37] Dayanik, S. and Karatzas, I. (2003). On the optimal stopping problemfor one-dimensional diffusions. Stochastic Process. Appl. 107 (173–212).

[38] Dayanik, S. and Sezer, S. O. (2006). Compound Poisson disorder prob-lem. To appear in Math. Oper. Res.

[39] Doob, J. L. (1949). Heuristic approach to the Kolmogorov–Smirnov theo-rems. Ann. Math. Statist. 20 (393–403).

480 Bibliography

[40] Doob, J. L. (1953). Stochastic Processes. Wiley, New York.

[41] Dupuis, P. and Wang, H. (2005). On the convergence from discrete tocontinuous time in an optimal stopping problem. Ann. Appl. Probab. 15(1339–1366).

[42] Du Toit, J. and Peskir, G. (2006). The trap of complacency in predictingthe maximum. To appear in Ann. Probab.

[43] Dubins, L. E. and Gilat, D. (1978). On the distribution of maxima ofmartingales. Proc. Amer. Math. Soc. 68 (337–338).

[44] Dubins, L. E. and Schwarz, G. (1988). A sharp inequality for sub-martingales and stopping times. Asterisque 157–158 (129–145).

[45] Dubins, L. E., Shepp, L. A. and Shiryaev, A. N. (1993). Optimal stop-ping rules and maximal inequalities for Bessel processes. Theory Probab.Appl. 38 (226–261).

[46] Dufresne, D. (2001). The integral of geometric Brownian motion. Adv. inAppl. Probab. 33 (223–241).

[47] Duistermaat, J. J., Kyprianou, A. E. and van Schaik, K. (2005).Finite expiry Russian options. Stochastic Process. Appl. 115 (609–638).

[48] Durbin, J. (1985). The first-passage density of a continuous Gaussian pro-cess to a general boundary. J. Appl. Probab. 22 (99–122).

[49] Durbin, J. (1992). The first-passage density of the Brownian motion processto a curved boundary (with an appendix by D. Williams). J. Appl. Probab.29 (291–304).

[50] Durrett, R. (1984). Brownian Motion and Martingales in Analysis.Wadsworth, Belmont.

[51] Dvoretzky, A., Kiefer, J. and Wolfowitz, J. (1953). Sequential de-cision problems for processes with continuous time parameter. Testing hy-potheses. Ann. Math. Statist. 24 (254–264).

[52] Dynkin, E. B. (1963). The optimum choice of the instant for stopping aMarkov process. Soviet Math. Dokl. 4 (627–629).

[53] Dynkin, E. B. (1965). Markov Processes. Vols. I, II. Academic Press, NewYork; Springer-Verlag, Berlin-Gottingen-Heidelberg. (Russian edition pub-lished in 1963 by “Fizmatgiz”.)

[54] Dynkin, E. B. (2002). Diffusions, Superdiffusions and Partial DifferentialEquations. Amer. Math. Soc., Providence.

Bibliography 481

[55] Dynkin, E. B. and Yushkevich A. A. (1969). Markov Processes: Theo-rems and Problems. Plenum Press, New York. (Russian edition published in1967 by “Nauka”.)

[56] Einstein, A. (1905). Uber die von der molekularkinetischen Theorie derWarme geforderte Bewegung von in ruhenden Flussigkeiten suspendiertenTeilchen. Ann. Phys. (4) 17 (549–560). English translation “On the motionof small particles suspended in liquids at rest required by the molecular-kinetictheory of heat” in the book ‘Einstein’s Miraculous Year’ by Princeton Univ.Press 1998 (85–98).

[57] Ekstrom, E. (2004). Russian options with a finite time horizon. J. Appl.Probab. 41 (313–326).

[58] Ekstrom, E. (2004). Convexity of the optimal stopping boundary for theAmerican put option. J. Math. Anal. Appl. 299 (147–156).

[59] Engelbert H. J. (1973). On the theory of optimal stopping rules forMarkov processes. Theory Probab. Appl. 18 (304–311).

[60] Engelbert H. J. (1974). On optimal stopping rules for Markov processeswith continuous time. Theory Probab. Appl. 19 (278–296).

[61] Engelbert H. J. (1975). On the construction of the payoff s(x) in theproblem of optimal stopping of a Markov sequence. (Russian) Math. Oper.Forschung und Statistik. 6 (493–498).

[62] Fakeev, A. G. (1970). Optimal stopping rules for stochastic processes withcontinuous parameter. Theory Probab. Appl. 15 (324–331).

[63] Fakeev, A. G. (1971). Optimal stopping of a Markov process. TheoryProbab. Appl. 16 (694–696).

[64] Feller, W. (1952). The parabolic differential equations and the associatedsemi-groups of transformations. Ann. of Math. (2) 55 (468–519).

[65] Ferebee, B. (1982). The tangent approximation to one-sided Brownian exitdensities. Z. Wahrscheinlichkeitstheor. Verw. Geb. 61 (309–326).

[66] Fick, A. (1885). Ueber diffusion. (Poggendorff’s) Annalen der Physik undChemie 94 (59–86).

[67] Fleming, W. H. and Rishel, R. W. (1975). Deterministic and StochasticOptimal Control. Springer-Verlag, Berlin–New York.

[68] Fokker, A. D. (1914). Die mittlere Energie rotierender elektrischer Dipoleim Strahlungsfeld. Ann. Phys. 43 (810–820).

482 Bibliography

[69] Fortet, R. (1943). Les fonctions aleatoires du type de Markoff associees acertaines equations lineaires aux derivees partielles du type parabolique. J.Math. Pures Appl. (9) 22 (177–243).

[70] Friedman, A. (1959). Free boundary problems for parabolic equations. I.Melting of solids. J. Math. Mech. 8 (499–517).

[71] Gapeev, P. V. and Peskir, G. (2004). The Wiener sequential testingproblem with finite horizon. Stoch. Stoch. Rep. 76 (59–75).

[72] Gapeev, P. V. and Peskir, G. (2006). The Wiener disorder problem withfinite horizon. To appear in Stochastic Process. Appl.

[73] Gal’chuk, L. I. and Rozovskii, B. L. (1972). The ‘disorder’ problem fora Poisson process. Theory Probab. Appl. 16 (712–716).

[74] Gilat, D. (1986). The best bound in the L logL inequality of Hardy andLittlewood and its martingale counterpart. Proc. Amer. Math. Soc. 97 (429–436).

[75] Gilat, D. (1988). On the ratio of the expected maximum of a martingaleand the Lp -norm of its last term. Israel J. Math. 63 (270–280).

[76] Gilbert, J. P. and Mosteller, F. (1966). Recognizing the maximum ofa sequence. J. Amer. Statist. Assoc. 61 (35–73).

[77] Girsanov, I. V. (1960). On transforming a certain class of stochastic pro-cesses by absolutely continuous substitution of measures. Theory Probab.Appl. 5 (285–301).

[78] Graversen, S. E. and Peskir, G. (1997). On Wald-type optimal stoppingfor Brownian motion. J. Appl. Probab. 34 (66–73).

[79] Graversen, S. E. and Peskir, G. (1997). On the Russian option: Theexpected waiting time. Theory Probab. Appl. 42 (564–575).

[80] Graversen, S. E. and Peskir, G. (1997). On Doob’s maximal inequalityfor Brownian motion. Stochastic Process. Appl. 69 (111–125).

[81] Graversen, S. E. and Peskir, G. (1998). Optimal stopping and maximalinequalities for geometric Brownian motion. J. Appl. Probab. 35 (856–872).

[82] Graversen, S. E. and Peskir, G. (1998). Optimal stopping and maximalinequalities for linear diffusions. J. Theoret. Probab. 11 (259–277).

[83] Graversen, S. E. and Peskir, G. (1998). Optimal stopping in theL logL -inequality of Hardy and Littlewood. Bull. London Math. Soc. 30(171–181).

Bibliography 483

[84] Graversen, S. E. and Shiryaev, A. N. (2000). An extension of P.Levy’s distributional properties to the case of a Brownian motion with drift.Bernoulli 6 (615–620).

[85] Graversen, S. E. Peskir, G. and Shiryaev, A. N. (2001). StoppingBrownian motion without anticipation as close as possible to its ultimatemaximum. Theory Probab. Appl. 45 (125–136).

[86] Griffeath, D. and Snell, J. L. (1974). Optimal stopping in the stockmarket. Ann. Probab. 2 (1–13).

[87] Grigelionis, B. I. (1967). The optimal stopping of Markov processes. (Rus-sian) Litovsk. Mat. Sb. 7 (265–279).

[88] Grigelionis, B. I. and Shiryaev, A. N. (1966). On Stefan’s problemand optimal stopping rules for Markov processes. Theory Probab. Appl. 11(541–558).

[89] Hansen, A. T. and Jørgensen, P. L. (2000). Analytical valuation ofAmerican-style Asian options. Management Sci. 46 (1116–1136).

[90] Hardy, G. H. and Littlewood, J. E. (1930). A maximal theorem withfunction-theoretic applications. Acta Math. 54 (81–116).

[91] Haugh, M. B. and Kogan, L. (2004). Pricing American options: a dualityapproach. Oper. Res. 52 (258–270).

[92] Hochstadt, H. (1973). Integral Equations. Wiley, New York–London–Sydney.

[93] Hou, C., Little, T. and Pant, V. (2000). A new integral representationof the early exercise boundary for American put options. J. Comput. Finance3 (73–96).

[94] Ikeda, N. and Watanabe, S. (1981). Stochastic Differential Equationsand Diffusion Processes. North-Holland, Amsterdam-New York; Kodansha,Tokyo.

[95] Irle, A. and Paulsen, V. (2004). Solving problems of optimal stoppingwith linear costs of observations. Sequential Anal. 23 (297–316).

[96] Irle, A. and Schmitz, N. (1984). On the optimality of the SPRT forprocesses with continuous time parameter. Math. Operationsforsch. Statist.Ser. Statist. 15 (91–104).

[97] Ito, K. (1944). Stochastic integral. Imperial Academy. Tokyo. Proceedings.20 (519–524).

484 Bibliography

[98] Ito, K. (1946). On a stochastic integral equation. Japan Academy. Pro-ceedings 22 (32–35).

[99] Ito, K. (1951). On stochastic differential equations. Mem. Amer. Math.Soc. 4 (1–51).

[100] Ito, K.and McKean, H. P., Jr. (1965). Diffusion Processes and TheirSample Paths. Academic Press, New York; Springer-Verlag, Berlin–NewYork. Reprinted in 1996 by Springer-Verlag.

[101] Jacka, S. D. (1988). Doob’s inequalities revisited: A maximal H1 -embedding. Stochastic Process. Appl. 29 (281–290).

[102] Jacka, S. D. (1991). Optimal stopping and the American put. Math. Fi-nance 1 (1–14).

[103] Jacka, S. D. (1991). Optimal stopping and best constants for Doob-likeinequalities I: The case p = 1 . Ann. Probab. 19 (1798–1821).

[104] Jacka, S. D. (1993). Local times, optimal stopping and semimartingales.Ann. Probab. 21 (329–339).

[105] Jacka, S. D. and Lynn, J. R. (1992). Finite-horizon optimal stoppingobstacle problems and the shape of the continuation region. StochasticsStochastics Rep. 39 (25–42).

[106] Jacod, J. and Shiryaev, A. N. (1987). Limit Theorems for StochasticProcesses. Springer-Verlag, Berlin. Second ed.: 2003.

[107] Karatzas, I. and Shreve, S. E. (1998). Methods of Mathematical Fi-nance. Springer-Verlag, New York.

[108] Karlin, S. (1962). Stochastic models and optimal policy for selling an as-set. Studies in Applied Probability and Management Science, Stanford Univ.Press, Standord (148–158).

[109] Karlin, S. and Taylor, H. M. (1981). A Second Course in StochasticProcesses. Academic Press, New York–London.

[110] Kim, I. J. (1990). The analytic valuation of American options. Rev. Finan-cial Stud. 3 (547–572).

[111] Kolmogoroff, A. (1931). Uber die analytischen Methoden in derWahrscheinlichkeitsrechnung. Math. Ann. 104 (415–458). English transla-tion “On analytical methods in probability theory” in “Selected works ofA. N. Kolmogorov” Vol. II (ed. A. N. Shiryayev), Kluwer Acad. Publ., Dor-drecht, 1992 (62–108).

Bibliography 485

[112] Kolmogoroff, A. (1933). Zur Theorie der stetigen zufalligen Prozesse.Math. Ann. 108 (149–160). English translation “On the theory of continuousrandom processes” in “Selected works of A. N. Kolmogorov” Vol. II (ed. A.N. Shiryayev) Kluwer Acad. Publ., Dordrecht, 1992 (156–168).

[113] Kolmogorov, A. N., Prokhorov, Yu. V. and Shiryaev, A. N. (1990).Probabilistic-statistical methods of detecting spontaneously occuring effects.Proc. Steklov Inst. Math. 182 (1–21).

[114] Kolodner, I. I. (1956). Free boundary problem for the heat equation withapplications to problems of change of phase. I. General method of solution.Comm. Pure Appl. Math. 9 (1–31).

[115] Kramkov, D. O. and Mordecki, E. (1994). Integral option. TheoryProbab. Appl. 39 (162–172).

[116] Kramkov, D. O. and Mordecki, E. (1999). Optimal stopping and max-imal inequalities for Poisson processes. Publ. Mat. Urug. 8 (153–178).

[117] Krylov, N. V. (1970). On a problem with two free boundaries for an ellipticequation and optimal stopping of a Markov process. Soviet Math. Dokl. 11(1370–1372).

[118] Kuznetsov, S. E. (1980). Any Markov process in a Borel space has atransition function. Theory Probab. Appl. 25 (384–388).

[119] Kyprianou, A. E. and Surya, B. A. (2005). On the Novikov-Shiryaevoptimal stopping problems in continuous time. Electron. Comm. Probab. 10(146–154).

[120] Lamberton, D. and Rogers, L. C. G. (2000). Optimal stopping andembedding. J. Appl. Probab. 37 (1143–1148).

[121] Lamberton, D. and Villeneuve, S. (2003). Critical price near maturityfor an American option on a dividend-paying stock. Ann. Appl. Probab. 13(800–815).

[122] Lawler G. F. (1991). Intersections of Random Walks. Birkhauser, Boston.

[123] Lehmann, E. L. (1959). Testing Statistical Hypotheses. Wiley, New York;Chapman & Hall, London.

[124] Lerche, H. R. (1986). Boundary Crossing of Brownian Motion. LectureNotes in Statistics 40. Springer-Verlag, Berlin–Heidelberg.

[125] Levy, P. (1939). Sur certains processus stochastiques homogenes. Compo-sitio Math. 7 (283–339).

486 Bibliography

[126] Lindley, D. V. (1961). Dynamic programming and decision theory. Appl.Statist. 10 (39–51).

[127] Liptser, R. S. and Shiryayev, A. N. (1977). Statistics of Random Pro-cesses I. Springer-Verlag, New York–Heidelberg. (Russian edition publishedin 1974 by “Nauka”.) Second, revised and expanded English edition: 2001.

[128] Liptser, R. S. and Shiryayev, A. N. (1978). Statistics of Random Pro-cesses II. Springer-Verlag, New York–Heidelberg. (Russian edition publishedin 1974 by “Nauka”.) Second, revised and expanded English edition: 2001.

[129] Liptser, R. S. and Shiryayev, A. N. (1989). Theory of Martingales.Kluwer Acad. Publ., Dordrecht. (Russian edition published in 1986 by“Nauka”.)

[130] Malmquist, S. (1954). On certain confidence contours for distribution func-tions. Ann. Math. Statist. 25 (523–533).

[131] Marcellus, R. L. (1990). A Markov renewal approach to the Poisson dis-order problem. Comm. Statist. Stochastic Models 6 (213–228).

[132] McKean, H. P., Jr. (1960/1961). The Bessel motion and a singular inte-gral equation. Mem. Coll. Sci. Univ. Kyoto Ser. A. Math. 6 (317–322).

[133] McKean, H. P., Jr. (1965). Appendix: A free boundary problem for theheat equation arising from a problem of mathematical economics. Ind. Man-agement Rev. 6 (32–39).

[134] Meyer, P.-A. (1966). Probability and Potentials. Blaisdell Publishing Co.,Ginn and Co., Waltham–Toronto–London.

[135] Mikhalevich, V. S. (1956). Sequential Bayes solutions and optimal meth-ods of statistical acceptance control. Theory Probab. Appl. 1 (395–421).

[136] Mikhalevich, V. S. (1958). A Bayes test of two hypotheses concerning themean of a normal process. (Ukrainian) Vısn. Kiıv. Unıv. No. 1 (254–264).

[137] Miranker, W. L. (1958). A free boundary value problem for the heatequation. Quart. Appl. Math. 16 (121–130).

[138] Miroshnichenko, T. P. (1975). Optimal stopping of the integral of aWiener process. Theory Probab. Apll. 20 (397–401).

[139] Mordecki, E. (1999). Optimal stopping for a diffusion with jumps. FinanceStoch. 3 (227–236).

[140] Mordecki, E. (2002). Optimal stopping and perpetual options for Levyprocesses. Finance Stoch. 6 (473–493).

Bibliography 487

[141] Mucci, A. G. (1978). Existence and explicit determination of optimal stop-ping times. Stochastic Process. Appl. 8 (33–58).

[142] Myneni, R. (1992). The pricing of the American option. Ann. Appl. Probab.2 (1–23).

[143] Novikov, A. A. (1971). On stopping times for a Wiener process. TheoryProbab. Appl. 16 (449–456).

[144] Novikov, A. A. and Shiryaev, A. N. (2004). On an effective solution ofthe optimal stopping problem for random walks. Theory Probab. Appl. 49(344–354).

[145] Øksendal, B. (1990). The high contact principle in optimal stopping andstochastic waves. Seminar on Stochastic Processes, 1989 (San Diego, CA,1989). Progr. Probab. 18, Birkhauser, Boston, MA (177–192).

[146] Øksendal, B. and Reikvam, K. (1998). Viscosity solutions of optimalstopping problems. Stochastics Stochastics Rep. 62 (285–301).

[147] Øksendal, B. (2005). Optimal stopping with delayed information. Stoch.Dyn. 5 (271–280).

[148] Park, C. and Paranjape, S. R. (1974). Probabilities of Wiener pathscrossing differentiable curves. Pacific J. Math. 53 (579–583).

[149] Park, C. and Schuurmann, F. J. (1976). Evaluations of barrier-crossingprobabilities of Wiener paths. J. Appl. Probab. 13 (267–275).

[150] Pedersen, J. L. (1997). Best bounds in Doob’s maximal inequality forBessel processes. J. Multivariate Anal. 75, 2000 (36–46).

[151] Pedersen, J. L. (2000). Discounted optimal stopping problems for themaximum process. J. Appl. Probab. 37 (972–983).

[152] Pedersen, J. L. (2003). Optimal prediction of the ultimate maximum ofBrownian motion. Stochastics Stochastics Rep. 75 (205–219).

[153] Pedersen, J. L. (2005). Optimal stopping problems for time-homogeneousdiffusions: a review. Recent advances in applied probability, Springer, NewYork (427–454).

[154] Pedersen, J. L. and Peskir, G. (1998). Computing the expectation ofthe Azema–Yor stopping times. Ann. Inst. H. Poincare Probab. Statist. 34(265–276).

[155] Pedersen, J. L. and Peskir, G. (2000). Solving non-linear optimal stop-ping problems by the method of time-change. Stochastic Anal. Appl. 18(811–835).

488 Bibliography

[156] Peskir, G. (1998). Optimal stopping inequalities for the integral of Brown-ian paths. J. Math. Anal. Appl. 222 (244–254).

[157] Peskir, G. (1998). The integral analogue of the Hardy–Littlewood L logL -inequality for Brownian motion. Math. Inequal. Appl. 1 (137–148).

[158] Peskir, G. (1998). The best Doob-type bounds for the maximum of Brown-ian paths. High Dimensional Probability (Oberwolfach 1996), Progr. Probab.43, Birkhauser, Basel (287–296).

[159] Peskir, G. (1998). Optimal stopping of the maximum process: The maxi-mality principle. Ann. Probab. 26 (1614–1640).

[160] Peskir, G. (1999). Designing options given the risk: The optimalSkorokhod-embedding problem. Stochastic Process. Appl. 81 (25–38).

[161] Peskir, G. (2002). On integral equations arising in the first-passage problemfor Brownian motion. J. Integral Equations Appl. 14 (397–423).

[162] Peskir, G. (2002). Limit at zero of the Brownian first-passage density.Probab. Theory Related Fields 124 (100–111).

[163] Peskir, G. (2005). A change-of-variable formula with local time on curves.J. Theoret. Probab. 18 (499–535).

[164] Peskir, G. (2005). On the American option problem. Math. Finance 15(169–181).

[165] Peskir, G. (2005). The Russian option: Finite horizon. Finance Stoch. 9(251–267).

[166] Peskir, G. (2006). A change-of-variable formula with local time on surfaces.To appear in Sem. de Probab. (Lecture Notes in Math.) Springer.

[167] Peskir, G. (2006). Principle of smooth fit and diffusions with angles. Re-search Report No. 7, Probab. Statist. Group Manchester (11 pp).

[168] Peskir, G. and Shiryaev, A. N. (2000). Sequential testing problems forPoisson processes. Ann. Statist. 28 (837–859).

[169] Peskir, G. and Shiryaev, A. N. (2002). Solving the Poisson disorderproblem. Advances in Finance and Stochastics. Essays in Honour of DieterSondermann. Springer, Berlin (295–312).

[170] Peskir, G. and Uys, N. (2005). On Asian options of American type. Ex-otic Option Pricing and Advanced Levy Models (Eindhoven, 2004), Wiley(217–235).

Bibliography 489

[171] Pham, H. (1997). Optimal stopping, free boundary, and American optionin a jump-diffusion model. Appl. Math. Optim. 35 (145–164).

[172] Planck, M. (1917). Uber einen Satz der statistischen Dynamik und seineErweiterung in der Quantentheorie. Sitzungsber. Preuß. Akad. Wiss. 24(324–341).

[173] Poor, H. V. (1998). Quickest detection with exponential penalty for delay.Ann. Statist. 26 (2179–2205).

[174] Revuz, D. and Yor, M. (1999). Continuous Martingales and BrownianMotion. Springer, Berlin.

[175] Ricciardi, L. M., Sacerdote, L. and Sato, S. (1984). On an inte-gral equation for first-passage-time probability densities. J. Appl. Probab.21 (302–314).

[176] Rogers, L. C. G. (2002). Monte Carlo valuation of American options.Math. Finance 12 (271–286).

[177] Rogers, L. C. G. and Shi, Z. (1995). The value of an Asian option. J.Appl. Probab. 32 (1077–1088).

[178] Rogers, L. C. G. and Williams, D. (1987). Diffusions, Markov Processes,and Martingales; Vol. 2: Ito Calculus. Wiley, New York.

[179] Romberg, H. F. (1972). Continuous sequential testing of a Poisson processto minimize the Bayes risk. J. Amer. Statist. Assoc. 67 (921–926).

[180] Salminen, P. (1985). Optimal stopping of one-dimensional diffusions. Math.Nachr. 124 (85–101).

[181] Schroder, M. (2003). On the integral of geometric Brownian motion. Adv.in Appl. Probab. 35 (159–183).

[182] Schrodinger, E. (1915). Zur Theorie der Fall- und Steigversuche anTeilchen mit Brownscher Bewegung. Physik. Zeitschr. 16 (289–295).

[183] Shepp, L. A. (1967). A first passage time for the Wiener process. Ann.Math. Statist. 38 (1912–1914).

[184] Shepp, L. A. (1969). Explicit solutions of some problems of optimal stop-ping. Ann. Math. Statist. 40 (993–1010).

[185] Shepp, L. A. and Shiryaev, A. N. (1993). The Russian option: Reducedregret. Ann. Appl. Probab. 3 (631–640).

[186] Shepp, L. A. and Shiryaev, A. N. (1994). A new look at pricing of the“Russian option”. Theory Probab. Appl. 39 (103–119).

490 Bibliography

[187] Shiryaev, A. N. (1961). The detection of spontaneous effects. Soviet Math.Dokl. 2 (740–743).

[188] Shiryaev, A. N. (1961). The problem of the most rapid detection of adisturbance of a stationary regime. Soviet Math. Dokl. 2 (795–799).

[189] Shiryaev, A. N. (1961). A problem of quickest detection of a disturbanceof a stationary regime. (Russian) PhD Thesis. Steklov Institute of Mathe-matics, Moscow. 130 pp.

[190] Shiryaev, A. N. (1963). On optimal methods in quickest detection prob-lems. Theory Probab. Appl. 8 (22–46).

[191] Shiryaev, A. N. (1966). On the theory of decision functions and controlof a process of observation based on incomplete information. Select. Transl.Math. Statist. Probab. 6 (162–188).

[192] Shiryaev, A. N. (1965). Some exact formulas in a “disorder” problem.Theory Probab. Appl. 10 (349–354).

[193] Shiryaev, A. N. (1967). Two problems of sequential analysis. Cybernetics3 (63–69).

[194] Shiryaev, A. N. (1969). Optimal stopping rules for Markov processes withcontinuous time. (With discussion.) Bull. Inst. Internat. Statist. 43 (1969),book 1 (395–408).

[195] Sirjaev, A. N. (1973). Statistical Sequential Analysis: Optimal StoppingRules. American Mathematical Society, Providence. (First Russian editionpublished by “Nauka” in 1969.)

[196] Shiryayev, A. N. (1978). Optimal Stopping Rules. Springer, New York–Heidelberg. (Russian editions published by “Nauka”: 1969 (first ed.), 1976(second ed.).)

[197] Shiryaev, A. N. (1999). Essentials of Stochastic Finance. Facts, Models,Theory. World Scientific, River Edge. (Russian edition published by FASISin 1998.)

[198] Shiryaev, A. N. (2002). Quickest detection problems in the technical anal-ysis of the financial data. Mathematical Finance—Bachelier Congress (Paris,2000), Springer, Berlin (487–521).

[199] Shiryaev, A. N. (2004). Veroyatnost’. Vol. 1, 2. MCCME, Moscow (Rus-sian). English translation: Probability. To appear in Springer.

[200] Shiryaev, A. N. (2004). A remark on the quickest detection problems.Statist. Decisions 22 (79–82).

Bibliography 491

[201] Siegert, A. J. F. (1951). On the first passage time probability problem.Phys. Rev. II 81 (617–623).

[202] Siegmund, D. O. (1967). Some problems in the theory of optimal stoppingrules. Ann. Math. Statist. 38 (1627–1640).

[203] Siegmund, D. O. (1985). Sequential Analysis. Tests and Confidence Inter-vals. Springer, New York.

[204] Smoluchowski, M. v. (1913). Einige Beispiele Brown’scher Molekularbe-wegung unter Einfluß außerer Krafte. Bull. Intern. Acad. Sc. Cracovie A(418–434).

[205] Smoluchowski, M. v. (1915). Notiz uber die Berechnung der BrownschenMolekularbewegung bei der Ehrenhaft-Millikanschen Versuchsanordnung.Physik. Zeitschr. 16 (318–321).

[206] Snell, J. L. (1952). Applications of martingale system theorems. Trans.Amer. Math. Soc. 73 (293–312).

[207] Strassen, V. (1967). Almost sure behavior of sums of independent randomvariables and martingales. Proceedings of the Fifth Berkeley Symposium onMathematical Statistics and Probability (Berkeley, 1965/66) II, Part 1, Univ.California Press, Berkeley (315–343).

[208] Stratonovich, R. L. (1962). Some extremal problems in mathematicalstatistics and conditional Markov processes. Theory Probab. Appl. 7 (216–219).

[209] Stroock D. W., Varadhan S. R. S. (1979) Multidimensional DiffusionProcesses. Springer, Berlin–New York.

[210] Taylor, H. M. (1968). Optimal stopping in a Markov process. Ann. Math.Statist. 39 (1333–1344).

[211] Thompson, M. E. (1971). Continuous parameter optimal stopping prob-lems. Z. Wahrscheinlichkeitstheor. verw. Geb. 19 (302–318).

[212] Tricomi, F. G. (1957). Integral Equations. Interscience Publishers, NewYork–London.

[213] Urusov, M. On a property of the moment at which Brownian motion at-tains its maximum and some optimal stopping problems. Theory Probab.Appl. 49 (2005) (169–176).

[214] van Moerbeke, P. (1974). Optimal stopping and free boundary problems.Rocky Mountain J. Math. 4 (539–578).

492 Bibliography

[215] van Moerbeke, P. (1976). On optimal stopping and free boundary prob-lems. Arch. Ration. Mech. Anal. 60 (101–148).

[216] Wald, A. (1947). Sequential Analysis. Wiley, New York; Chapman & Hall,London.

[217] Wald, A. (1950). Statistical Decision Functions. Wiley, New York; Chap-man & Hall, London.

[218] Wald, A. and Wolfowitz, J. (1948). Optimum character of the sequentialprobability ratio test. Ann. Math. Statist. 19 (326–339).

[219] Wald, A. and Wolfowitz, J. (1949). Bayes solutions of sequential deci-sion problems. Proc. Natl. Acad. Sci. USA 35 (99–102). Ann. Math. Statist.21, 1950 (82–99).

[220] Walker, L. H. (1974). Optimal stopping variables for Brownian motion.Ann. Probab. 2 (317–320).

[221] Wang, G. (1991). Sharp maximal inequalities for conditionally symmetricmartingales and Brownian motion. Proc. Amer. Math. Soc. 112 (579–586).

[222] Whittle, P. (1964). Some general results in sequential analysis. Biometrika51 (123–141).

[223] Wu, L., Kwok, Y. K. and Yu, H. (1999). Asian options with the Americanearly exercise feature. Int. J. Theor. Appl. Finance 2 (101–111).

[224] Yor, M. (1992). On some exponential functionals of Brownian motion. Adv.in Appl. Probab. 24 (509–531).

Subject Index

a posteriori probability process, 288,309, 335, 357

adapted process, 54admissible function, 207American option, 375angle-bracket process, 59Appell polynomial, 24arbitrage-free price, 375, 379, 395Asian option, 416average delay, 356average number of visits, 81

backward equation, 95backward Kolmogorov equation, 90Bayesian problem, xiiiBellman’s principle, 6Bessel inequalities, 251Blumenthal’s 0-1 law, 230

for Brownian motion, 97Bouleau–Yor formula, 68Brownian motion, 93, 94Burkholder–Davis–Gundy

inequalities, 63, 284

cadlag function, 54cag function, 55canonical representation, 105

for semimartingales, 69Cauchy distribution, 105Cauchy problem, 135, 137

killed, 136–138Cauchy–Euler equation, 376, 397change of measure, 115change of scale, 194change of space, 111, 193

change of time, 106, 109change of variables, 195change-of-variable formula, 74Chapman–Kolmogorov equations, 79,

88, 108, 113of Volterra type, 221

characteristic operator, 101, 128compensator, 56compound Poisson process, 104concave conjugate, 248condition of linear growth, 73condition of normal reflection, xixcondition of smooth fit, xixcontinuation set, xvii, 35continuous fit, 49, 144cost function, 200creation, 119cumulant, 103cumulant function, 70

Dambis–Dubins–Schwarz theorem, 110differential characteristics, 88differential equation

normal form, 211diffusion, 101diffusion coefficient, 88, 199diffusion process, 72, 101

with jumps, 72diffusions with angles, 155dimension of problem, 126Dirichlet class, 56Dirichlet problem, 84, 130

for the Poisson equation, 85inhomogeneous, 86

494 Subject Index

Dirichlet/Poisson problem, 132discounted problem, 127discounting, 119discounting rate, 102, 127, 215Doob convergence theorem, 61Doob inequalities, 255, 269

expected waiting time, 263in mean, 62in probability, 62

Doob stopping time theorem, 60Doob type bounds, 269Doob–Meyer decomposition, 56drift coefficient, 88, 199dynamic programming, 6

early exercise premium representation,385, 403, 411, 420

ellipticity condition, 102Esscher measure, 119essential supremum, 7Euclidean velocity, 187excessive function, 83

Follmer–Protter–Shiryaev formula, 68Feynman–Kac formula, 137, 138filtered probability space, 54filtration, 53finite horizon, 125, 146finite horizon formulation, 36first boundary problem, 84first-passage equation, 221fixed-point theorem

for contractive mappings, 237forward equation, 95forward Kolmogorov equation, 90free-boundary equation, 219, 221, 393free-boundary problem, 48, 143

gain function, 35, 203generalized Markov property, 78generating operator, 82Girsanov theorem

for local martingales, 117Green function, 81, 200

Hardy–Littlewood inequalities, 272harmonic function, 83Hermite polynomial, 193Hunt stopping time theorem, 60

inequality of L logL type, 283infinite horizon, 125, 144infinite horizon formulation, 36infinitesimal generator, 129infinitesimal operator, 101, 129information, 53initial distribution, 76innovation process, 344instantaneous stopping, 264integral process, 124integral representation

of the maximum process, 447invariant function, 83inverse problem, 240Ito formula, 67Ito–Clark representation theorem, 442Ito–Levy representation, 70, 106Ito–Tanaka formula, 67Ito–Tanaka–Meyer formula, 67iterative method, 19iterative procedure, 48

Khintchine inequalities, 62killed problem, 127killing, 119killing coefficient, 102killing rate, 127Kolmogorov backward equation, 139

semigroup formulation, 140Kolmogorov inequalities, 61Kolmogorov test, 230Kolmogorov–Chapman equations, 79,

88, 108, 113Kolmogorov–Levy–Khintchine formula,

103semimartingale analogue, 72

Kummer confluenthypergeometric function, 192

Levy characterization theorem, 94

Subject Index 495

Levy convergence theorem, 61Levy distributional theorem, 96Levy measure, 69Levy process, 102Levy–Khintchine representation, 104Levy–Smirnov distribution, 105Lagrange functional, 132Laplacian, 86law of the iterated logarithm

at infinity, 97at zero, 97

likelihood ratio process, 288, 309, 336,357

linear problem, 196linear programming, 49

dual problem, 50primal problem, 50

local Lipschitz condition, 73local martingale, 55

first decomposition, 58purely discontinuous, 58second decomposition, 58

local submartingale, 55local supermartingale, 55local time, 67

on curve, 74on surfaces, 75

local time-space formula, 74localized class, 55localizing sequence, 55lower function, 230

Markov chain, 76in a wide sense, 76time-homogeneous, 76

Markov kernel, 76Markov process, 76, 88Markov property, 91

generalized, 78in a strict sense, 76in a wide sense, 76strong, 79

Markov sequence, 76Markov time, 1, 27

finite, 54Markovian cost problem, 217martingale, 53, 55

basic definitions, 53fundamental theorems, 60

martingale convergence theorem, 61martingale maximal inequalities, 61master equation, 227, 228maximal equality, ximaximal inequality, xii

for geometric Brownian motion,271

maximality principle, 207maximum process, 395Mayer functional, 130method of backward induction, 3method of essential supremum, 6method of measure change, 197method of space change, 193method of time change, 165MLS formulation, 124MLS functional, 128, 135

Neumann problem, 134, 135Newton potential, 81nonlinear integral equation, 219nonlinear problem, 196normal distribution, 105normal reflection, xix, 264Novikov condition, 197number of visits, 81

obstacle problem, 146occupation times formula, 69optimal prediction problem, 437

ultimate integral, 438ultimate maximum, 441ultimate position, 437

optimal stoppingcontinuous time, 26discrete time, 1Markovian approach, 12, 34martingale approach, 1, 26of maximum process, 199

496 Subject Index

optimal stopping boundary, 207optimal stopping problem, 2optimal stopping time, 2optional σ -algebra, 57optional process, 57optional sampling theorem, 60orthogonality of local martingales, 58

parabolic cylinder function, 192parabolic differential equation

backward, 88forward, 89

perpetual option, 395Picard method, 271PIDE problem, 128Poisson disorder problem, 356Poisson equation, 81, 82, 85potential measure, 81potential of a function, 81potential of a Markov chain, 80potential of an operator, 80potential theory, 79predictable σ -algebra, 55predictable process, 56predictable quadratic covariation, 59predictable quadratic variation, 59principle of continuous fit, 153principle of smooth fit, 149probability of a false alarm, 356probability of an error

of the first kind, 335of the second kind, 335

probability-statistical space, 287process of bounded variation, 55progressive measurability, 58

quadratic characteristic, 59, 65quadratic covariation, 66quadratic variation, 65quickest detection

of Poisson process, 355of Wiener process, 308

quickest detection problemfor Poisson process, 356

for Wiener process, 308

Radon–Nikodym derivative, 288random element, 54reflection principle, 229

for Brownian motion, 96regular boundary, 129regular diffusion process, 150, 156regular point, 152, 156Russian option, 395

S -concave function, 157scale function, 114, 200scaling property, 227self-similarity property, 95, 104semimartingale, 55

special, 59sequential analysis, xiiisequential testing

of a Poisson process, 334of Wiener process, 287

shift operator, 77smallest supermartingale, 9, 14smooth fit, 49, 144, 160smooth fit through scale, 158smooth-fit condition, xixSnell envelope, 8, 28solution-measure, 73solution-process, 73space change, 193speed measure, 107, 200squared Bessel process, 188state space, 76statistical experiment, 287Stefan free-boundary problem, 147stochastic basis, 53stochastic differential equation, 72

of “bang-bang” type, 454stochastic exponential, 72, 103stochastic integral, 63stochastic process

adapted to a filtration, 54Markov in a strict sense, 91Markov in a wide sense, 91

Subject Index 497

progressively measurable, 58with independent increments, 69with stationary

independent increments, 69stopped process, 54stopping set, xvii, 35stopping time, 1, 27, 54strike price, xivstrong Markov property, 79, 92, 99strong solution, 73submartingale, 55superdiffusion, 101superharmonic characterization, 147superharmonic function, 16, 17, 37supermartingale, 55supremum functional, 133supremum process, 124

Tanaka formula, 67time-space Feller condition, 154transition function, 76transition kernel, 72transition operator, 80triplet of predictable characteristics,

71truncation function, 70, 71

unilateral stable distribution, 105upper function, 230

value function, 2, 35physical interpretation, 146, 147

variational problem, xiiiVolterra integral equation

of the first kind, 229of the second kind, 240

Wald identities, 61Wald inequalities, 244Wald’s optimal stopping

of Brownian motion, 245Wald–Bellman equation, 14–16, 84

uniqueness, 19Wald–Bellman inequality, 83

weak solution, 73Whittaker equation, 189Wiener disorder problem, 308Wiener process, 93

List of Symbols

A , 101A , 101A , 56(B, C, ν) , 71B = (Bt)t≥0 , 375bE+ , 80Bx = (Bx

t (ω))t≥0 , 98C (continuation set), 16C (space of continuous functions),

54Cε , 41Cg , xixCλ , 43(D) (Dirichlet class), 56D (space of cadlag functions), 54D (stopping set), 16∂C (boundary of C ), 129, 130Dε , 41Dg , xixDiF , 66DijF , 66Dλ , 43DP , 80(E, E) , 54E+ , 80E+ , 82E(λ) , 103Et(λ) , 72Fτ , 54F+

t , 97F

t , 97γa(t) , 96H · X , 64Kt(λ) , 103

L , xviiiL′

loc(M) , 65lsc (lower semicontinuous), 36L(s, x) , 90Lt , 96La

t , 67L∗(t, y) , 90Lvar(A) , 65L(X) , 65〈M〉 , 58M , 55M (family of all stopping times), 2M (family of all Markov times), 2MN = MN

0 , 2Mloc , 55M2

loc , 58Mn = M∞

n , 2MN

n = τ ∈ M | n ≤ τ ≤ N , 2Mt , 29µ , 70N = (Nt)t≥0 , 104NB , 81ν , 70O , 57(Ω,F , (Ft)t≥0, P) , 53P , 55Pg , 80ϕ (standard normal density

function), 438Φ (standard normal distribution

function), 441PII , 69PIIS , 69

Ploc P , 117

500 List of Symbols

Px , 79P (x; B) , 79Q , 15Q , 84Q

(η)k (x) , 24

Qn , 15SDE , 145S1(σ, 0, µ) , 105S2(σ, 0, µ) , 105S1/2(σ, 1, µ) , 105Sα(σ, β, µ) , 105(SN

n )0≤n≤N , 3SN

n = esssup n≤τ≤N E (Gτ | Fn) , 8S∞

n , 11SemiM , 55, 59σB , 79Sp-SemiM , 59subM , 55(subM)loc , 55supM , 55(supM)loc , 55T (θ) , 106Ta , 96Ta,b , 97τB , 79θ , 77Tτ(U) , 101U , 80usc (upper semicontinuous), 36V , 55Vn , 8Vn(x) , 24V N

n , 2, 3V ∞

n , 11V N (x) , 12Vn(x) , 24V (t, x) , 36[X ] , 65x+ = max(x, 0) , 24X = X T , 107[X, Y ] , 66Xc , 60Xloc , 55

Xτ (stopped process), 54Zd = 0 ± 1,±2, . . .d , 86

Documents

Shiryaev a., Peskir G. Optimal Stopping and Free-Boundary Problems s