J Optim Theory ApplDOI 10.1007/s10957-013-0386-5

Stochastic Maximum Principle for ControlledBackward Delayed System via Advanced StochasticDifferential Equation

Li Chen · Jianhui Huang

Received: 12 April 2012 / Accepted: 25 July 2013© Springer Science+Business Media New York 2014

Abstract The main contributions of this paper are three old. First, our primary con-cern is to investigate a class of stochastic recursive delayed control problems thatnaturally arise with strong backgrounds but have not been well studied yet. For illus-tration, some concrete examples are provided here. Second, it is interesting that a newclass of time-advanced stochastic differential equations (ASDEs) is introduced as theadjoint process via duality relation. To our knowledge, such equations have neverbeen discussed in literature, although they have considerable research values. Anexistence and uniqueness result for ASDEs is presented. Third, to illustrate our the-oretical results, some dynamic optimization problems are discussed based on ourstochastic maximum principles. It is interesting that the optimal controls are de-rived explicitly by solving the associated time-advanced ordinary differential equa-tion (AODE), the counterpart of the ASDE in its deterministic setup.

Keywords Advanced stochastic differential equation · Backward delayed system ·Backward stochastic differential equation · Maximum principle ·Stochastic recursive control

1 Introduction

Our starting point is a backward stochastic differential equation (BSDE) with time-delayed generator. Two remarkable features of such an equation are: (i) Instead of the

L. ChenDepartment of Mathematics, China University of Mining and Technology, Beijing 100083, Chinae-mail: chenli@cumtb.edu.cn

J. Huang (B)Department of Applied Mathematics, Hong Kong Polytechnic University, Hong Kong, Hong Konge-mail: majhuang@polyu.edu.hk

J Optim Theory Appl

initial condition, the terminal condition is specified; (ii) The generator depends notonly on the instantaneous state, but also on the past history of the state through thetime-delayed parameter. The feature (i) makes the BSDEs with time-delayed genera-tors essentially different from the well-studied stochastic delay differential equations(SDDEs) (see, e.g., [1, 2], etc.), in which the initial state conditions are given be-forehand. The BSDEs with delay also differ from the standard BSDEs due to theirtime-delayed generators from feature (ii). In particular, they are different from theanticipated backward stochastic differential equations (ABSDEs) introduced by [3],which are dual to SDDEs. Since this kind of BSDEs with time-delayed generatorswas recently introduced by [4], it has already found many real backgrounds in eco-nomics, finance, management, and decision sciences. More details can be found in[4–6] and the references therein. Due to the interesting structure and wide-range ap-plications, it is very natural and necessary to study the dynamic optimizations ofequations of this type. However, to our best knowledge, very few works have beendone in this direction, and thus we aim to fill this research gap in some systematicway.

In principle, the dynamic optimization problem can be investigated using two dif-ferent methods: the dynamic programming principle (DPP), which leads to someHamilton–Jacobi–Bellman (HJB) equation for the value function, or the stochasticmaximum principle, which leads to some Hamiltonian system for the optimality.Note that the investigation becomes more complicated when we study the controlledstochastic (forward) delayed systems: for instance, the state should be embedded intosome infinite-dimensional function space when we apply the DPP. Keep this in mind,we introduce and discuss a class of general controlled stochastic (backward) delayedsystems using the maximum principle method. It turns out that there is a perfect dual-ity between the controlled forward system (i.e., SDDEs) and its adjoint equation (i.e.,ABSDEs). Concerning our controlled backward stochastic delayed system, we willintroduce a new type of advanced stochastic differential equations (ASDEs) and es-tablish the corresponding duality relation. Based on it, the maximum principle for thedelayed backward controlled system can be established with the help of variationalanalysis.

The rest of this paper is organized as follows. In Sect. 2, we present some illus-trating examples for motivation of our problem. Some preliminary results on delayedBSDEs and ASDEs are also given. The stochastic recursive delayed control prob-lems are formulated in Sect. 3, and necessary and sufficient conditions of the max-imum principle are derived based on the duality between the ASDEs and BSDEswith delayed generator. As an application of our theoretical results, in Sect. 4, werevisit the examples given in Sect. 2, and the optimal controls are derived explicitlyby solving the associated time-advanced ordinary differential equation (AODE). Theconclusions are given in Sect. 5.

2 Preliminaries

2.1 Notation and Motivations

Let T > 0 be some finite time horizon, and let R denote the set of real numbers.For any Euclidean space H , we denote by 〈·, ·〉 (resp. | · |) the scalar product (resp.

J Optim Theory Appl

norm) of H . Let W(·) be a standard d-dimensional Brownian motion on a completeprobability space (Ω,F ,P ). The information structure is given by the filtration F ={Ft }t≥0 generated by W(·) and augmented by all P -null sets. For p ≥ 1, we use thefollowing notation:

Lp(Ω,Ft , P ;H) :={ξ is an H -valued Ft -measurable random variable satisfying

E[ξp

]< +∞};

Lp

F(t1, t2;H) :=

{ϕ(t), t1 ≤ t ≤ t2, is an F-adapted process satisfying

E

∫ t2

t1

∣∣ϕ(t)∣∣p dt < +∞

};

L∞F

(t1, t2;H) :={ϕ(t), t1 ≤ t ≤ t2, is an H -valuedF-adapted bounded process

}.

We introduce the following more general controlled backward delayed system:

⎧⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎩

−dy(t) = f

(t, y(t),

∫ t

t−δ

φ(t, s)y(s)α(ds), z(t),

∫ t

t−δ

φ(t, s)z(s)α(ds), v(t),

∫ t

t−δ

φ(t, s)v(s)α(ds)

)dt − z(t) dW(t), 0 ≤ t ≤ T ,

y(T ) = ξ, y(t) = ϕ(t), z(t) = ψ(t), −δ ≤ t < 0.

(1)Here, δ is a time delay parameter, α is a σ -finite measure, and φ(·, ·) is a locallybounded process. The relevance of our optimization problems can be demonstratedby the following concrete examples.

Remark 2.1 B. Øksendal et al. [7, 8] considered delays of the form

∫ t

t−δ

e−ρ(t−s)x(s) ds.

It is a kind of the moving average terms

∫ t

t−δ

φ(t, s)x(s)α(ds)

that we used in (1).

Example 2.1 (Optimization of forward–backward stochastic differential utility) Thisexample originates from [9], in which the decision makers have recursive utility withdelayed generators. Such a utility can be used to characterize the habit information,disappointment effects, and volatility aversion in decision-making. Accordingly, theobjective of a decision maker is to maximize his/her utility by selecting suitable in-stantaneous consumption process c(t). This leads to the dynamic optimization prob-

J Optim Theory Appl

lem

infc(·) y

c(0),

where the recursive utility y(t) satisfies the following BSDE with time-delayed gen-erator:

⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

−dy(t) = f

(t, y(t),

∫ t

0y(s) ds, z(t),

∫ t

0z(s) ds, c(t)

)dt − z(t) dW(t),

0 ≤ t ≤ T ,

y(T ) = ξ, y(t) = ϕ(t), z(t) = ψ(t), −δ ≤ t < 0.

(2)

Equation (2) can be viewed as a special case of (1) by noting that

∫ t

0y(s) ds =

∫ t

t−T

y(s)T χ{s≥0}α(ds),

where α is the uniform measure on [t − T , t]. It can characterize the nonmonotonicutility to volatility aversion.

Example 2.2 (Pension fund with delayed surplus) This example comes from [10],where the pension fund manager can invest two assets: the riskless asset P0(t) satisfy-ing dP0(t) = rP0(t) dt with instantaneous return rate r ≥ 0 and the risky asset P1(t)

satisfying dP1(t) = μP1(t) dt + σP1(t) dW(t) with expected return rate μ ≥ r andvolatility rate σ > 0. Denote by λ := μ−r

σthe risk premium, by θ(t) ∈ [0,1] the pro-

portion of fund invested in risky asset, and by S(t) the surplus premium to fund mem-bers. Suppose that the pension fund at time t is y(t), and it is reasonable to assumethat S(t) depends on the performance of fund growth during the past period. Thus,we assume that S(t) = g(y(t) − κy(t − δ)) for some κ > 0 and g : R → [0,+∞),which is increasing, convex, and Lipschitz continuous, and δ > 0 is the time delay.Moreover, there should be some operation cost or consumption for fund management(represented by the instantaneous rate c(t)). Hence, the pension fund y(t) evolves as

⎧⎪⎪⎨

⎪⎪⎩

dy(t) = ([θ(t)σλ + r

]y(t) − g

(y(t) − κy(t − δ)

) − c(t))dt + σθ(t)y(t) dW(t),

0 ≤ t ≤ T ,

y(0) = y0, y(t) = 0, −δ ≤ t < 0.

(3)Note that in practice, the pension fund is required to provide some minimum guaran-tee, i.e., to pay some part of the due benefits ξ (which should be some random vari-able) at some given future time T . Keep this in mind, the objective of fund manageris to choose θ(t) and c(t) to meet terminal condition y(T ) = ξ and also maximizesome given cost functional at the same time. By setting z(t) = σθ(t)y(t), (3) can be

J Optim Theory Appl

reformulated by the following controlled backward delayed system:⎧⎪⎪⎨

⎪⎪⎩

dy(t) = {ry(t) + λz(t) − g

(y(t) − κy(t − δ)

) − c(t)}dt + z(t) dW(t),

0 ≤ t ≤ T ,

y(t) = 0, −δ ≤ t < 0, y(T ) = ξ.

(4)

Equation (4) is a special case of (1) by setting α(ds) to be Dirac measure at −δ, thepointwise delay with lag δ.

Example 2.3 It is remarkable that there exists a considerably rich literature discussingthe controlled stochastic delay differential equations (SDDEs) (see, e.g., [7, 8, 11,12], etc.), which naturally arise due to the time lag between the observation and reg-ulator, or the possible after-effect of control. The SDDEs and their optimization haveattracted extensive research attention in last few decades and have been applied inwide-range domains including physics, biology, engineering, etc. (see [1, 2] for moredetails). Note that these works are discussed in the forward setup because the initialcondition is given a priori. On the other hand, as suggested by [13, 14], the forwardcontrolled systems can be reformulated by some backward controlled systems. Forexample, in case of some state constraints (e.g., no short selling), it is better to refor-mulate the controlled forward systems into some backward systems, which are moreconvenient to be analyzed in some cases (see [15, 16]). Furthermore, inspired by [17],we aim to investigate the following controlled linear backward delayed system (seeSect. 4 for details):⎧⎪⎪⎨

⎪⎪⎩

dy(t) = (β1y(t) + β2y(t − δ) + γ1z(t) + γ2z(t − δ) + αv(t)

)dt + z(t) dW(t),

0 ≤ t ≤ T ,

y(T ) = ξ,

which can be transformed from some linear constrained forward controlled delaysystem by penalty approach, or can be viewed as the limit of a family of linear un-constrained forward delayed systems.

2.2 Results of Delayed Backward Stochastic Differential Equations andTime-Advanced Stochastic Differential Equations

We set

yδ(t) =∫ t

t−δ

φ(t, s)y(s)α(ds), zδ(t) =∫ t

t−δ

φ(t, s)z(s)α(ds).

Then the backward delayed system (1) can be rewritten as{−dy(t) = f

(t, y(t), yδ(t), z(t), zδ(t)

)dt − z(t) dW(t), 0 ≤ t ≤ T ,

y(T ) = ξ, y(t) = ϕ(t), z(t) = ψ(t), −δ ≤ t < 0.(5)

J Optim Theory Appl

We introduce the following assumptions:

(H2.1) The function f : Ω × [0, T ] ×Rn ×R

n ×Rn×d ×R

n×d → Rn is F-adapted

and satisfies∣∣f (t, y, yδ, z, zδ) − f

(t, y′, y′

δ, z′, z′

δ

)∣∣

≤ C(|y − y′| + |yδ − y′

δ| + |z − z′| + |zδ − z′δ|

)

for all y, yδ, y′, y′

δ ∈ Rn, z, zδ, z

′, z′δ ∈R

n×d with constant C > 0.

(H2.2) The fixed time delay satisfies 0 ≤ δ ≤ T , ξ ∈ L2(Ω,FT ,P ;Rn), the ini-tial path of (y, z) is given by given square-integrable functions ϕ(·) and ψ(·), andφ(t, s) ≤ M for any s, t ∈ [−δ, T ] and some M > 0.

(H2.3) E[∫ T

0 |f (t,0,0,0,0)|2 dt] < +∞.

Then we have the following existence and uniqueness result of the delayed BSDE(1):

Theorem 2.1 Suppose that (H2.1)–(H2.3) hold. Then for sufficiently small time de-lay δ, the delayed BSDE (1) has a unique solution (y(·), z(·)) ∈ L2

F(−δ, T ;Rn) ×

L2F(−δ, T ;Rn×d).

Proof Let us introduce the following norm in Banach space L2F(−δ, T ;Rn):

∥∥ν(·)∥∥β

:=(E

[∫ T

−δ

∣∣ν(s)∣∣2

eβs ds

]) 12

.

Set⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

y(t) = ξ +∫ T

t

f(s, Y (s), Yδ(s),Z(s),Zδ(s)

)ds −

∫ T

t

z(s) dW(s),

0 ≤ t ≤ T ,

y(t) = ϕ(t), z(t) = ψ(t), −δ ≤ t < 0.

(6)

Define a mapping h : L2F(−δ, T ;Rn×R

n×d) −→Rn×R

n×d such that h[(Y (·),Z(·))]= (y(·), z(·)). So, if we can prove that h is a contraction mapping under the norm‖ · ‖β , then the desired result can be obtained by the fixed-point theorem. For twoarbitrary elements (Y (·),Z(·)) and (Y ′(·),Z′(·)) in L2

F(−δ, T ;Rn ×R

n×d), set

(y(·), z(·)) = h

[(Y(·),Z(·))], (

y′(·), z′(·)) = h[(

Y ′(·),Z′(·))].Denote their difference by

(Y (·), Z(·)) := (

Y(·) − Y ′(·),Z(·) − Z′(·)),(y(·), z(·)) := (

y(·) − y′(·), z(·) − z′(·)).

J Optim Theory Appl

In fact, (6) is a classical BSDE, and it follows that

E

[∫ T

0

(β

2|y(s)|2 + |z(s)|2

)eβs ds

]

≤ 2

βE

[∫ T

0

∣∣f(s, Y (s), Yδ(s),Z(s),Zδ(s)

)

− f(s, Y ′(s), Y ′

δ(s),Z′(s),Z′

δ(s))∣∣2

eβs ds

]

≤ 2C2

βE

[∫ T

0

(∣∣Y (s)∣∣ + ∣

∣Yδ(s)∣∣ + ∣

∣Z(s)∣∣ + ∣

∣Zδ(s)∣∣)2

eβs ds

]

≤ 6C2

βE

[∫ T

0

(∣∣Y (s)∣∣2 + ∣∣Z(s)

∣∣2 + 2∣∣Yδ(s)

∣∣2 + 2∣∣Zδ(s)

∣∣2)eβs ds

]

≤ 6C2

β

[1 + 2M2δ

∫ 0

−δ

e−βrα(dr)

]E

[∫ T

−δ

(∣∣Y (s)∣∣2 + ∣∣Z(s)

∣∣2)eβs ds

]

= K(C,M,δ,α,β)E

[∫ T

−δ

(∣∣Y (s)∣∣2 + ∣∣Z(s)

∣∣2)eβs ds

].

Note that

E

∫ T

0

∣∣Yδ(s)∣∣2

eβs ds

= E

∫ T

0

∣∣∣∣

∫ 0

−δ

φ(s, s + r)(Y(s + r) − Y ′(s + r)

)α(dr)

∣∣∣∣

2

eβs ds

≤ M2δE

∫ T

0

∫ 0

−δ

∣∣Y(s + r) − Y ′(s + r)∣∣2

α(dr)eβs ds

= M2δE

∫ 0

−δ

e−βr

∫ T

0

∣∣Y(s + r) − Y ′(s + r)∣∣2

eβ(s+r) dsα(dr)

= M2δE

∫ 0

−δ

e−βr

∫ T +r

r

∣∣Y(u) − Y ′(u)∣∣2

eβu duα(dr)

≤ M2δE

∫ 0

−δ

e−βrα(dr)

∫ T

−δ

∣∣Y (s)∣∣2

eβs ds.

If we choose β = 1δ

, then

K(C,M,δ,α,β) = 6C2δ[1 + 2M2δeα

([−δ,0])].Therefore, if δ is sufficiently small so that K(C,M,δ,α,β) < 1, then h is a contrac-tion mapping under the norm ‖ · ‖β . Our proof is completed. �

J Optim Theory Appl

Using a similar method, we can get the following result.

Lemma 2.1 Suppose that (H2.1)–(H2.3) hold. Then for sufficiently small time delayδ, we have the following estimate of the solution of the BSDE with delay (5):

E

[sup

0≤t≤T

∣∣y(t)∣∣2 +

∫ T

0

∣∣z(t)∣∣2

dt

]≤ CE

[∣∣ξ∣∣2 +

∫ T

0

∣∣f (t,0,0,0,0)∣∣2

dt

]

with some constant C > 0.

Now, let us introduce the following time-advanced stochastic differential equation(ASDE):

⎧⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎩

dx(t) = b

(t, x(t),

∫ t+δ

t

φ(t, s)x(s)α(ds)

)dt

+ σ

(t, x(t),

∫ t+δ

t

φ(t, s)x(s)α(ds)

)dW(t), 0 ≤ t ≤ T ,

x(0) = x0, x(t) = λ(t), T < t ≤ T + δ.

(7)

It is notable that there exist some results on the time-advanced ordinary differen-tial equations (AODEs) (e.g., refer to [18–23], etc.) that have been applied in var-ious areas including traveling waves in physics, cell-growth in population dynam-ics, capital market in economics, life-cycle models, electronics, etc. However, to ourbest knowledge, the stochastic differential equations of advanced type (ASDEs) havenever been discussed in literature before. Nevertheless, these stochastic advancedequations should also have considerable importance besides the control study only(as implied by the broad-range applications of AODES, their deterministic counter-part). Now we aim to study the Ft -adapted solution x(·) ∈ L2

F(0, T + δ;Rn) of the

ASDE (7). Suppose that for all t ∈ [0, T ],b :Ω ×R

n × L2(Ω,Fr ,P ;Rn) → L2(Ω,Ft , P ;Rn

),

σ :Ω ×Rn × L2(Ω,Fr ,P ;Rn

) → L2(Ω,Ft , P ;Rn×d),

where r ∈ [t, T + δ]. We also assume that b and σ satisfy the following conditions:

(H2.4) There exists a constant C > 0 such that for all t ∈ [0, T ], x, x′ ∈ Rn,

ζ(·), ζ ′(·) ∈ L2F(t, T + δ;Rn), r ∈ [t, T + δ], we have

∣∣b(t, x, ζ(r)

) − b(t, x′, ζ ′(r)

)∣∣ + ∣∣σ(t, x, ζ(r)

) − σ(t, x′, ζ ′(r)

)∣∣

≤ C(∣∣x − x′∣∣ +E

Ft[∣∣ζ(r) − ζ ′(r)

∣∣]).

(H2.5) sup0≤t≤T (|b(t,0,0) + σ(t,0,0)|) < +∞.

Under these conditions, b(t, ·, ·) and σ(t, ·, ·) are Ft -measurable, and this ensuresthat the solution of the advanced SDE is Ft -adapted. We have the following result onthe ASDE (7).

J Optim Theory Appl

Theorem 2.2 Assume that b and σ satisfy (H2.4) and (H2.5), E supT ≤t≤T +δ |λ(t)|2 <

+∞, E|x0|2 < +∞, and the time delay δ is sufficiently small. Then the ASDE (7) ad-mits a unique Ft -adapted solution.

Proof Similarly to Theorem 2.1, let us define the following norm, which is moreconvenient for us to construct a contraction mapping in the Banach space L2

F(0, T +

δ;Rn):

∥∥ν(·)∥∥

β:=

(E

[∫ T +δ

0

∣∣ν(s)

∣∣2

e−βs ds

]) 12

.

For simplicity, we denote∫ t+δ

tφ(t, s)x(s)α(ds) by xδ+(t), and set

⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

x(t) = x0 +∫ t

0b(s,X(s),Xδ+(s)

)ds +

∫ t

0σ(s,X(s),Xδ+(s)

)dW(s),

0 ≤ t ≤ T ,

x(t) = λ(t), T < t ≤ T + δ.

Then we can define a mapping I : L2F(0, T + δ;Rn) =⇒ L2

F(0, T + δ;Rn) such that

I [X(·)] = x(·). For arbitrary X(·),X′(·) ∈ L2F(0, T + δ;Rn), we introduce the fol-

lowing notation:

I[X(·)] = x(·), I

[X′(·)] = x′(·),

X(·) := X(·) − X′(·), x(·) := x(·) − x′(·).Consequently, x(·) satisfies

⎧⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎩

x(t) =∫ t

0

[b(s,X(s),Xδ+(s)

) − b(s,X′(s),X′

δ+(s))]

ds

+∫ t

0

[σ(s,X(s),Xδ+(s)

) − σ(s,X′(s),X′

δ+(s))]

dW(s), 0 ≤ t ≤ T ,

x(0) = 0, x(t) = 0, T < t ≤ T + δ.

Applying Itô’s formula to e−βt |x(t)|2 on [0, T ], we get

E[e−βT

∣∣x(T )∣∣2] + βE

[∫ T

0e−βt

∣∣x(t)∣∣2

dt

]

= E

[∫ T

0

(2e−βt

⟨b(t), x(t)

⟩ + e−βt⟨σ (t), σ (t)

⟩)dt

]

with

b(t) := b(t,X(t),Xδ+(t)

) − b(t,X′(t),X′

δ+(t)),

σ (t) := σ(t,X(t),Xδ+(t)

) − σ(t,X′(t),X′

δ+(t)).

J Optim Theory Appl

Since b,σ satisfy (H2.4), we have

βE

[∫ T

0e−βt

∣∣x(t)∣∣2

dt

]

≤ E

[∫ T

0e−βt

∣∣x(t)∣∣2

dt

]+E

[∫ T

0e−βt

∣∣b(t)∣∣2

dt

]+E

[∫ T

0e−βt

∣∣σ (t)∣∣2

dt

]

≤ E

[∫ T

0e−βt

∣∣x(t)∣∣2

dt

]+ 2C2

E

[∫ T

0e−βt

(∣∣X(t)∣∣ +E

Ft[∣∣Xδ+(t)

∣∣])2dt

].

Moreover, it follows that

(β − 1)E

[∫ T

0e−βt

∣∣x(t)∣∣2

dt

]

≤ 4C2E

[∫ T

0e−βt

∣∣X(t)∣∣2

dt

]+ 4C2

E

[∫ T

0e−βt

∣∣Xδ+(t)∣∣2

dt

]

≤ 4C2[

1 + M2δ

∫ δ

0eβsα(ds)

]E

[∫ T +δ

0e−βt

∣∣X(t)∣∣2

dt

],

due to the fact

E

[∫ T

0e−βt

∣∣Xδ+(t)∣∣2

dt

]≤ M2δE

[∫ T

0e−βt

∫ δ

0

∣∣X(s)∣∣2

α(ds) dt

]

≤ M2δ

∫ δ

0eβsα(ds)E

[∫ T +δ

0e−βt

∣∣X(t)∣∣2

dt

].

Set

K ′(C,M,δ,α,β) = 4C2[1 + M2δ∫ δ

0 eβsα(ds)]β − 1

.

If we choose β = 1δ

, then for sufficiently small δ, we have

K ′(C,M,δ,α,β) ≤ 4C2δ[1 + M2δeα([0, δ])]1 − δ

< 1.

It follows that the mapping I is a contraction, and hence the result. �

3 Optimal Control Problem for Backward Stochastic System with Delay

In this section, we study a class of stochastic recursive delayed control problemsdescribed by (1). Here, we give the following assumptions: f : Ω × [0, T ] ×R

n × Rn × R

n×d × Rn×d × R

k × Rk → R

n is a given measurable function, ξ ∈L2(Ω,FT ,P ;Rn), ϕ(·),ψ(·) are deterministic functions, and v(·) is the control pro-cess with initial path η. The stochastic recursive control problem is to find the optimal

J Optim Theory Appl

control to achieve a pregiven goal ξ at the terminal time T and also to maximize somegiven cost functional. Let U be a nonempty and convex subset. We denote by U theset of all admissible control processes v(·) of the form

v(t) ={

η(t), −δ ≤ t < 0,

v(t) ∈ L2F

(0, T ;Rk

), v(t) ∈ U, a.s. ,0 ≤ t ≤ T .

The objective is to maximize the following functional over U :

J(v(·)) = E

[∫ T

0l

(t, y(t),

∫ t

t−δ

φ(t, s)y(s)α(ds), z(t),

∫ t

t−δ

φ(t, s)z(s)α(ds), v(t),

∫ t

t−δ

φ(t, s)v(s)α(ds)

)dt + γ

(y(0)

)].

For simplicity, denote

(∫ t

t−δ

φ(t, s)y(s)α(ds),

∫ t

t−δ

φ(t, s)z(s)α(ds),

∫ t

t−δ

φ(t, s)v(s)α(ds)

)

by (yδ(t), zδ(t), vδ(t)) if no confusion occurs. Introduce the following assumptions.

(H3.1) f is continuously differentiable in (y, yδ, z, zδ, v, vδ). Moreover, the partialderivatives fy,fyδ , fz, fzδ , fv , and fvδ of f with respect to (y, yδ, z, zδ, v, vδ) areuniformly bounded.

Then, if v(·) is admissible control and assumption (H3.1) holds, then the de-layed BSDE (1) has a unique solution (yv(·), zv(·)) ∈ L2

F(0, T + δ;Rn) × L2

F(0, T +

δ;Rn×d) on [0, T + δ] for sufficiently small 0 ≤ δ ≤ T . From now on, we let δ besufficiently small such that our system equation have solutions, and we can give therange of δ in some examples (see in Sect. 4).

(H3.2) For each v(·) ∈ U , l(·, yv(·), yvδ (·), zv(·), zv

δ (·), v(·), vδ(·)) ∈ L1F(0, T ;R), l is

differentiable with respect to (y, yδ, z, zδ, v, vδ), γ is differentiable with respect toy, and all the derivatives are bounded. In what follows, we will derive necessary andsufficient conditions for the maximum principle of the above problem.

3.1 Necessary Conditions

Now let (u(·), yu(·), zu(·)) be an optimal solution of our problem. Take an arbi-trary v(·) in U . Then, for each 0 ≤ ρ ≤ 1, vρ(·) = u(·) + ρ(v(·) − u(·)) ∈ U . Let(yρ(·), zρ(·)) be the state processes of system (1) with vρ(·).

J Optim Theory Appl

To derive a first-order necessary condition in terms of small ρ, we let (y(·), z(·))be the solution of the following delayed BSDE:

⎧⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎩

−dy(t) = [f u

y (t)y(t) + f uyδ

(t)yδ(t) + f uz (t)z(t) + f u

zδ(t)zδ(t) + f u

v (t)v(t)

+ f uvδ

(t)vδ(t)]dt − z(t) dW(t), 0 ≤ t ≤ T ,

y(T ) = 0,

y(t) = z(t) = v(t) = 0, −δ ≤ t < 0,

(8)

where f uk (t) = fk(t, y

u(t), yuδ (t), zu(t), zu

δ (t), u(t), uδ(t)), k = y, yδ, z, zδ, v, vδ ,and v(t) = vρ(t) − u(t). Equation (8) is called the variational equation.

Set

yρ(t) = ρ−1[yρ(t) − yu(t)] − y(t),

zρ(t) = ρ−1[zρ(t) − zu(t)] − z(t).

Lemma 3.1 Assume (H3.1) and (H3.2) hold. Then we have:

limρ→0

E

[sup

0≤t≤T

∣∣yρ(t)∣∣2

]= 0, lim

ρ→0E

[∫ T

0

∣∣zρ(t)∣∣2

dt

]= 0. (9)

Since u(·)is an optimal control of our problem, clearly, the following inequalityholds for any v(·) ∈ U and the corresponding vρ(·):

ρ−1[J(vρ(·)) − J

(u(·))] ≤ 0. (10)

From this point we derive the following variational inequality.

Lemma 3.2 If (H3.1) and (H3.2) hold, then we have

E

[γy

(yu(0)

)y(0) +

∫ T

0

(luy (t)y(t) + luyδ

(t)yδ(t)

+ luz (t)z(t) + luzδ(t)zδ(t) + luv (t)v(t) + luvδ

(t)vδ(t))dt

]≤ 0 ∀v(·) ∈ U , (11)

where luk (t) = lk(t, yu(t), yu

δ (t), zu(t), zuδ (t), u(t), uδ(t)), k = y, yδ, z, zδ, v, vδ .

Remark 3.1 It is straightforward to prove Lemmas 3.1 and 3.2 by using Lemma 2.1,the Lebesgue dominated convergence theorem, and Taylor expansion. Thus, we omitthe details and only state the main result for simplicity of presentation.

J Optim Theory Appl

In order to derive the maximum principle, we introduce the dual equation of thevariational equation (8) as follows:

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

dp(t) ={−luy (t) + f u

y (t)p(t)

+EFt

[∫ t+δ

t

(f u

yδ(s) − luyδ

(s))φ(s, t)χ[0,T ](s) ds

]α(dt)

dt

}dt

+{−luz (t) + f u

z (t)p(t)

+EFt

[∫ t+δ

t

(f u

zδ(s) − luzδ

(s))φ(s, t)χ[0,T ](s) ds

]α(dt)

dt

}dW(t),

0 ≤ t ≤ T ,

p(0) = −γy

(yu(0)

).

(12)

Define the Hamiltonian function H : [0, T ]×Rn ×R

n ×Rn×d ×R

n×d ×Rk ×R

k ×R

n → R by

H(t, y, yδ, z, zδ, v, vδ,p) := l(t, y, yδ, z, zδ, v, vδ) − ⟨f (t, y, yδ, z, zδ, v, vδ),p

⟩.

Then the associated adjoint equation (12) can be rewritten as⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

dp(t) ={−Hy

(t,Θ(t), u(t), uδ(t),p(t)

)

−EFt

[∫ t+δ

t

Hyδ

(s,Θ(s), u(s), uδ(s),p(s)

)φ(s, t)χ[0,T ](s) ds

]α(dt)

dt

}dt

+{−Hz

(t,Θ(t), u(t), uδ(t),p(t)

)

−EFt

[∫ t+δ

t

Hzδ

(s,Θ(s), u(s), uδ(s),p(s)

)φ(s, t)χ[0,T ](s) ds

]

× α(dt)

dt

}dW(t),

0 ≤ t ≤ T ,

p(0) = −γy

(y(0)

),

(13)with Θ(t) = (yu(t), yu

δ (t), zu(t), zuδ (t)), where α(dt)

dtis the Radon–Nikodym deriva-

tive.

Remark 3.2 For a given admissible control v(·), (12) and (13) are ASDEs. It is notnecessary to give the value of p(t) for T < t ≤ T + δ by the virtue of the indicativefunction χ[0,T ](s). Moreover, the ASDEs (12) and (13) admit a unique solution underconditions (H3.1) and (H3.2) due to Theorem 2.2.

J Optim Theory Appl

Now we can give the first main result of this paper.

Theorem 3.1 (Necessary conditions of optimality) Let (H3.1) and (H3.2) hold. Sup-pose that u(·) is an optimal control of our problem and (yu(·), zu(·)) is the corre-sponding optimal state trajectory. Then we have

⟨Hv

(t,Θ(t), u(t), uδ(t),p(t)

)

+EFt

∫ t+δ

t

Hvδ

(s,Θ(s), u(s), uδ(s),p(s)

)φ(s, t)χ[0,T ](s) ds

α(dt)

dt, v − u(t)

⟩

≤ 0 (14)

for any v ∈ U , a.e., a.s., where p(·) is the solution of the adjoint equation (13).

Proof Applying Itô’s formula to 〈p(t), y(t)〉, we have

E[γy

(yu(0)

) y(0)

]

= E

[∫ T

0

⟨p(t),−f u

yδ(t)yδ(t) − f u

zδ(t)zδ(t)

⟩dt

]

+E

[∫ T

0

⟨p(t),−f u

v v(t) − f uvδ

(t)vδ(t)⟩dt

]

+E

[∫ T

0

(⟨luy (t), y(t)

⟩ + ⟨luz (t), z(t)

⟩)dt

]

+E

[∫ T

0

⟨EFt

[∫ t+δ

t

(f u

yδ(s) − luyδ

(s))φ(s, t)χ[0,T ](s) ds

]α(dt)

dt, y(t)

⟩dt

]

+E

[∫ T

0

⟨EFt

[∫ t+δ

t

(f u

zδ(s) − luzδ

(s))φ(s, t)χ[0,T ](s) ds

]α(dt)

dt, z(t)

⟩dt

].

Fortunately, we have the following result, making the above equation to be of a simpleform:

E

[∫ T

0

⟨p(t), f u

yδ(t)yδ(t)

⟩dt

]

= E

[∫ T

0

⟨p(s), f u

yδ(s)

∫ s

s−δ

φ(s, r)y(r)α(dr)

⟩ds

]

= E

[∫ T

0

⟨EFr

∫ r+δ

r

f uyδ

(s)p(s)φ(s, r)χ[0,T ](s) ds, y(r)

⟩α(dr)

]

= E

[∫ T

0

⟨EFt

[∫ t+δ

t

f uyδ

(s)p(s)φ(s, t)χ[0,T ](s) ds

]α(dt)

dt, y(t)

⟩dt

].

J Optim Theory Appl

Moreover, the following relationships also hold:

E

[∫ T

0

⟨p(t), f u

zδ(t)zδ(t)

⟩dt

]

= E

[∫ T

0

⟨EFt

[∫ t+δ

t

f uzδ

(s)p(s)φ(s, t)χ[0,T ](s) ds

]α(dt)

dt, z(t)

⟩dt

],

E

[∫ T

0luyδ

(t)yδ(t) dt

]

= E

[∫ T

0

⟨EFt

[∫ t+δ

t

luyδ(s)φ(s, t)χ[0,T ](s) ds

]α(dt)

dt, y(t)

⟩dt

],

E

[∫ T

0luzδ

(t)zδ(t) dt

]

= E

[∫ T

0

⟨EFt

[∫ t+δ

t

luzδ(s)φ(s, t)χ[0,T ](s) ds

]α(dt)

dt, z(t)

⟩dt

].

Consequently, by (11) we derive

E

∫ T

0

{−⟨f u

v (t)p(t), v(t)⟩− ⟨

f uvδ

(t)p(t), vδ(t)⟩+ ⟨

luv (t), v(t)⟩+ ⟨

luvδ(t), vδ(t)

⟩}dt ≤ 0,

i.e.,

E

[∫ T

0

⟨Hv

(t,Θ(t), u(t), uδ(t),p(t)

), v(t)

⟩dt

]

+ E

[∫ T

0

⟨EFt

∫ t+δ

t

Hvδ

(s,Θ(s), u(s), uδ(s),p(s)

)φ(s, t)χ[0,T ](s) ds

α(dt)

dt, v(t)

⟩]

≤ 0.

Referring to the proof of Theorem 1.5 in [24], for v ∈ U , we have that (14) holds a.e.,a.s.. The left-hand side of (14) is equivalent to⟨Hv

(t,Θ(t), u(t), uδ(t),p(t)

)

+EFt

[∫ t+δ

t

Hvδ

(s,Θ(s), u(t), uδ(s),p(s)

)φ(s, t)χ[0,T ](s) ds

]α(dt)

dt, u(t)

⟩

= maxv∈U

⟨Hv

(t,Θ(t), u(t), uδ(t),p(t)

)

+EFt

[∫ t+δ

t

Hvδ

(s,Θ(s), u(t), uδ(s),p(s)

)φ(s, t)χ[0,T ](s) ds

]α(dt)

dt, v

⟩.

(15)�

J Optim Theory Appl

3.2 Sufficient Conditions

In this subsection, we will show that maximum conditions (14) or (15) plus someconcavity conditions constitute sufficient conditions of optimality. We call them thesufficient conditions of optimal control.

Theorem 3.2 (Sufficient conditions of optimality) Let (H3.1) and (H3.2) hold. Sup-pose that for u(·) ∈ U , (y(·), z(·)) is the corresponding trajectory and p(·) is the cor-responding solution of the adjoint equation (13). If condition (14) or (15) holds and,moreover, if H(t, y, yδ, z, zδ, v, vδ,p(t)) is a concave function of (y, yδ, z, zδ, v, vδ)

and γ is concave in y, then u(·) is an optimal control for our problem.

Proof Choose a v(·) ∈ U and let (yv(·), zv(·)) be the corresponding solution of (1).To simplify the notation, we also use

Θv(t) = (yv(t), yv

δ (t), zv(t), zvδ (t)

), Θ(t) = (

yu(t), yuδ (t), zu(t), zu

δ (t)).

Let

I := E

[∫ T

0

{l(t, y(t), yδ(t), z(t), zδ(t), u(t), uδ(t)

)

− l(t, yv(t), yv

δ (t), zv(t), zvδ (t), v(t), vδ(t)

)}dt

],

II := [γ(y(0)

) − γ(yv(0)

)].

We want to prove that

J(u(·)) − J

(v(·)) = I + II ≥ 0. (16)

Since γ is concave on y, II ≥ γy(y(0)) (y(0)−yv(0)) = −p(0) (y(0)−yv(0)).Applying Itô’s formula to 〈p(·), y(·) − yv(·)〉, we have

p(0) (y(0) − yv(0)

)

= E

∫ T

0

⟨p(t), f

(t,Θ(t), u(t), uδ(t)

) − f(t,Θv(t), v(t), vδ(t)

)⟩dt

+E

∫ T

0

⟨Hy

(t,Θ(t), u(t), uδ(t),p(t)

)

+EFt

[∫ t+δ

t

Hyδ

(s,Θ(s), u(s), uδ(s),p(s)

)φ(s, t)χ[0,T ](s) ds

]α(dt)

dt,

y(t) − yv(t)

⟩dt

+E

∫ T

0

⟨Hz

(t,Θ(t), u(t), uδ(t),p(t)

)

+EFt

[∫ t+δ

t

Hzδ

(s,Θ(s), u(s), uδ(s),p(s)

)φ(s, t)χ[0,T ](s) ds

]α(dt)

dt,

z(t) − zv(t)

⟩dt. (17)

J Optim Theory Appl

On the other hand,

I = E

∫ T

0

[H

(t,Θ(t), u(t), uδ(t),p(t)

) − H(t,Θv(t), v(t), vδ(t),p(t)

)]dt

+E

∫ T

0

⟨p(t), f

(t,Θ(t), u(t), uδ(t)

) − f(t,Θv(t), v(t), vδ(t)

)⟩dt. (18)

Since (Θ,v, vδ) → H(t,Θ,v, vδ,p(t)) is concave, we have

I ≥ −E

∫ T

0

⟨Hy

(t,Θ(t), u(t), uδ(t),p(t)

), yv(t) − y(t)

⟩dt

−E

∫ T

0

⟨Hyδ

(t,Θ(t), u(t), uδ(t),p(t)

), yv

δ (t) − yδ(t)⟩dt

−E

∫ T

0

⟨Hz

(t,Θ(t), u(t), uδ(t),p(t)

), zv(t) − z(t)

⟩dt

−E

∫ T

0

⟨Hzδ

(t,Θ(t), u(t), uδ(t),p(t)

), zv

δ (t) − zδ(t)⟩dt

−E

∫ T

0

⟨Hv

(t,Θ(t), u(t), uδ(t),p(t)

), v(t) − u(t)

⟩dt

−E

∫ T

0

⟨Hvδ

(t,Θ(t), u(t), uδ(t),p(t)

), vδ(t) − uδ(t)

⟩dt

+E

∫ T

0

⟨p(t), f

(t,Θ(t), u(t), uδ(t)

) − f(t,Θv(t), v(t), vδ(t)

)⟩dt. (19)

Moreover, we have

E

∫ T

0

⟨Hvδ

(t,Θ(t), u(t), uδ(t),p(t)

), vδ(t) − uδ(t)

⟩dt

= E

∫ T

0

⟨Hvδ

(s,Θ(s), u(s), uδ(s),p(s)

),

∫ s

s−δ

φ(s, r)(v(r) − u(r)

)α(dr)

⟩ds

= E

∫ T

0

⟨EFr

∫ r+δ

r

Hvδ

(s,Θ(s), u(s), uδ(s),p(s)

)φ(s, r)χ[0,T ](s) ds,

v(r) − u(r)

⟩α(dr)

= E

∫ T

0

⟨EFt

[∫ t+δ

t

Hvδ

(s,Θ(s), u(s), uδ(s),p(s)

)φ(s, t)χ[0,T ](s) ds

]α(dt)

dt,

v(t) − u(t)

⟩dt. (20)

J Optim Theory Appl

By the maximum condition (15) we obtain⟨Hv

(t,Θ(t), u(t), uδ(t),p(t)

), v(t) − u(t)

⟩dt

+E

∫ T

0

⟨Hvδ

(t,Θ(t), u(t), uδ(t),p(t)

), vδ(t) − uδ(t)

⟩dt

= 0. (21)

From (16)–(21) we easily get

J (u(·) − J(v(·))

≥ −E

∫ T

0

⟨Hy

(t,Θ(t), u(t), uδ(t),p(t)

), yv(t) − y(t)

⟩dt

−E

∫ T

0

⟨Hyδ

(t,Θ(t), u(t), uδ(t),p(t)

), yv

δ (t) − yδ(t)⟩dt

−E

∫ T

0

⟨Hz

(t,Θ(t), u(t), uδ(t),p(t)

), zv(t) − z(t)

⟩dt

−E

∫ T

0

⟨Hzδ

(t,Θ(t), u(t), uδ(t),p(t)

), zv

δ (t) − zδ(t)⟩dt

+E

∫ T

0

⟨Hy

(t,Θ(t), u(t), uδ(t),p(t)

)

+EFt

[∫ t+δ

t

Hyδ

(s,Θ(s), u(s), uδ(s),p(s)

)φ(s, t)χ[0,T ](s) ds

]α(dt)

dt,

yv(t) − y(t)

⟩dt

+E

∫ T

0

⟨Hz

(t,Θ(t), u(t), uδ(t),p(t)

)

+EFt

[∫ t+δ

t

Hzδ

(s,Θ(s), u(s), uδ(s),p(s)

)φ(s, t)χ[0,T ](s) ds

]α(dt)

dt,

zv(t) − z(t)

⟩dt

= 0.

So, we have verified that J (u(·)) − J (v(·)) ≥ 0 for any v(·) ∈ U , and it follows thatu(·) is an optimal control. �

Corollary 3.1 If α(dt) is the Dirac measure at −δ, then the system involves point-wise delay, i.e., yδ(t) = y(t − δ), zδ(t) = z(t − δ), vδ(t) = v(t − δ). In this case, asufficient condition of optimality is

Hv

(t,Θ(t), u(t), u(t − δ),p(t)

)

+EFt

[Hvδ

(t + δ,Θ(t + δ), u(t), u(t + δ),p(t + δ)

)] = 0

J Optim Theory Appl

with adjoint equation

⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

dp(t) = {−Hy

(t,Θ(t), u(t), u(t − δ),p(t)

)

−EFt

[Hyδ

(t + δ,Θ(t + δ), u(t + δ), u(t),p(t + δ)

)]}dt

{−Hz

(t,Θ(t), u(t), u(t − δ),p(t)

)

−EFt

[Hzδ

(t + δ,Θ(t + δ), u(t + δ), u(t),p(t + δ)

)]}dW(t),

0 ≤ t ≤ T ,

p(0) = −γy

(yu(0)

),

p(t) = 0, T < t ≤ T + δ,

(22)

where Θ(t) = (yu(t), yu(t − δ), zu(t), zu(t − δ)).

4 Applications

4.1 Application I: Dynamic Optimization of Recursive Utility with Moving Average

In this subsection, we investigate Example 2.1 of the dynamic optimization of recur-sive utility with moving average, already given in Sect. 2. The state equation satisfiesthe following dynamics:

y(t) = ξ −∫ T

t

[αc(s) + β

∫ s

0y(u)du

]ds −

∫ T

t

z(s) dWs, (23)

where α,β > 0 are some constants, and the control variable is a consumption processc(·). The class of admissible controls is denoted by C := {c(·) ∈ L2

F(0, T ;R),0 ≤

t ≤ T }. Given some standard utility function U , e.g., U(x) = xR

Rfor 0 < R < 1, we

consider the following dynamic optimization problem:

infc(·)∈C

J(c(·)),

where the objective functional is given by

J(c(·)) = −E

[∫ T

0U

(c(t)

)dt

]+ yc(0).

A similar model has been studied in [5], which is also introduced to describe thedynamic optimization models of recursive utility with moving average. The state of(23) can be reformulated as

y(t) = ξ −∫ T

t

[αc(s) + β

∫ s

s−T

T y(u)χ{u≥0}α(du)

]ds −

∫ T

t

z(s) dWs,

J Optim Theory Appl

where α is the uniform measure. Introduce the Hamiltonian function

H(t, y, yδ, z, c,p) = −U(c(t)

) +[αc(t) + β

∫ t

t−T

T y(u)χ{u≥0}(u)α(du)

]p(t).

The associated adjoint equation satisfies⎧⎪⎨

⎪⎩

dp(t) =(∫ T

t

βp(s) ds

)dt, 0 ≤ t ≤ T ,

p(0) = 1.

(24)

It follows that (24) can be reduced to the following ordinary differential equation:

p(t) =∫ T

t

βp(s) ds, p(t) = −βp(t),

which is solvable, and by Theorem 3.2 we have the following result.

Proposition 4.1 The optimal consumption is given by c(t) = (αp(t))1

R−1 , where p(t)

satisfies (24).

4.2 Application II: Dynamic Optimization of Pension Fund with Delayed Surplus

In this subsection, let us turn to Example 2.2 in Sect. 2. We will use the results ob-tained in Sect. 3 to derive the optimal control. For simplicity, suppose that g(·) is thefollowing linear function:

g(y(t) − κy(t − δ)

) = αy(t) − ακy(t − δ),

where α > 0. Then our model can be rewritten as{

dy(t) = {(r − α)y(t) + λz(t) + ακy(t − δ) − c(t)

}dt + z(t) dW(t), 0 ≤ t ≤ T ,

y(t) = 0, −δ ≤ t < 0, y(T ) = ξ.

(25)Denote the admissible control set by C := {c(·) ∈ L2

F(0, T ;R),0 ≤ t ≤ T }. It follows

that, if δ is sufficiently small, then (25) admits a unique solution pair (y(·), z(·)).Introduce the following objective functional of the fund manager:

J(c(·)) = E

[∫ T

0Le−ρt (c(t))

1−R

1 − Rdt

]− Ky(0), (26)

where L and K are positive constants, ρ is a discount factor, and 0 < R < 1 is the in-dex of risk aversion. The manager aims to maximize the expected objective functionalby taking into account both the cumulative consumption and initial reserve requite-ment. The optimal control problem is to maximize J (c(·)) over C. The Hamiltonianfunction is given by

H(t, y, yδ, c,p) = Le−ρt (c(t))1−R

1 − R+{

(r −α)y(t)+λz(t)+ακy(t −δ)−c(t)}p(t).

J Optim Theory Appl

The adjoint equation is

{dp(t) = {

(α − r)p(t) − ακEFt[p(t + δ)

]}dt − λp(t) dW(t), 0 ≤ t ≤ T ,

p(0) = K, p(t) = 0, T < t ≤ T + δ.

(27)Then from Corollary 3.1 we have the following result.

Proposition 4.2 If p(t) is the solution of ASDE (27), then the optimal consumption is

given by c(t) = (p(t)eρt

L)− 1

R , and the optimal fund proportion in risky asset is θ(t) =z(t)

σy(t), where (y(t), z(t)) satisfies (25).

In the following, we aim to get an explicit solution of ASDE (27). To this end, wefirst set

M(t) := e∫ t

0 −λdW(s)− 12

∫ t0 λ2 ds, 0 ≤ t ≤ T + δ.

It follows that M(t) is an exponential martingale and satisfies dM(t) = −λM(t) dW(t).Let p(t) = q(t)M(t), where q(t) is a deterministic function defined on [0, T + δ].Then, applying Itô’s formula to p(t), we have

dp(t) = q ′(t)M(t) dt − λq(t)M(t) dW(t), 0 ≤ t ≤ T . (28)

On the other hand, plugging p(t) = q(t)M(t) into (27), we have

dp(t) = {(α − r)q(t)M(t) − ακq(t + δ)EFt

[M(t + δ)

]}dt − λq(t)M(t) dW(t)

= {(α − r)q(t)M(t) − ακq(t + δ)M(t)

}dt − λq(t)M(t) dW(t),

0 ≤ t ≤ T .

(29)Comparing (28) and (29), we see that if the AODE

{q ′(t) = (α − r)q(t) − ακq(t + δ), 0 ≤ t ≤ T ,

q(0) = K, q(t) = 0, T < t ≤ T + δ,(30)

has a solution, then p(t) = q(t)M(t) is a solution of ASDE (27). The solution ofAODE (30) can be obtained via the characteristic function as follows: q(t) = Keht

for 0 ≤ t ≤ T , and q(t) = 0 for T < t ≤ T + δ. Here, h satisfies the following char-acteristic equation:

h + ακehδ = (α − r).

Note that α, r, κ are the parameters of the state equation, so the above characteristicequation has a solution h if the delayed parameter δ is small enough. In fact, if wedenote

F(h) := h + ακehδ,

then it follows that limh→+∞ F(h) = +∞. In addition, since F ′(h) > 0, F(h) is anincreasing function of h, so there exists a unique h such that F(h) = (α − r), and

J Optim Theory Appl

thus q(t) and p(t) are uniquely determined. One remark to the parameter range is asfollows. If we set L = max{|α − r|, ακ,λ}, we have the following parameter range towell-posedness of BSDE (25) and ASDE (27):

{6L2δ

(1 + 2δ2e

)< 1,

4L2δ(1 + δ2e

) + δ < 1.(31)

4.3 Application III: The Dynamic Optimization of Linear Delayed System

Here, we revisit Example 2.3 of a backward system with time-delayed generator. Thestate equation is

y(t) = ξ −∫ T

t

[β1y(s)+β2y(s−δ)+γ1z(s)+γ2z(s−δ)+αv(s)

]ds−

∫ T

t

z(s) dWs,

where α,β1, β2, γ1, γ2 are some constants, v(·) is the control process, and the classof admissible controls is denoted by Uad := {v(·) ∈ L2

F(0, T ;R),0 ≤ t ≤ T }. The

dynamic optimization problem is as follows:

infv∈Uad

J(v(·)),

where the objective functional is given by

J(v(·)) = 1

2E

[∫ T

0R(t)v2(t) dt

]+ Ky(0)

for some constant K and a nonnegative function R(t) defined on [0, T ]. By Corol-lary 3.1, the Hamiltonian function of our optimization problem becomes

H(t, y, yδ, z, zδ, v,p)

= −1

2R(t)v2(t) + (

αv(t) + β1y(t) + β2y(t − δ) + γ1z(t) + γ2z(t − δ))p(t),

and the adjoint equation becomes

⎧⎪⎪⎨

⎪⎪⎩

dp(t) = (−β1p(t) − β2EFt

[p(t + δ)

])dt + (−γ1p(t) − γ2E

Ft[p(t + δ)

])dW(t),

0 ≤ t ≤ T ,

p(0) = K, p(t) = 0, T < t ≤ T + δ.

(32)Similar to Application II, we can introduce the exponential martingale satisfyingdM(t) = γM(t) dW(t), where γ is some coefficient to be determined. Let p(t) =q(t)M(t). Then we get the following AODE:

{q ′(t) = −β1q(t) − β2q(t + δ),

γ q(t) = γ1q(t) + γ2q(t + δ).(33)

J Optim Theory Appl

The first equation q(t) can be solved using the same method to (30). Based on it, wecan plug q(t) into second equation to get the value of γ , and thus the exponentialmartingale M(t) can be uniquely determined. Consequently, the optimal control isgiven by u(t) = αp(t)

R(t), where p(t) = q(t)M(t) is the solution of the ASDE (32).

5 Conclusions

We investigate the optimal control problems for stochastic backward delayed systemthat naturally arises with various concrete backgrounds such as recursive utility opti-mization, pension fund management, etc. Under mild conditions, necessary and suffi-cient conditions for optimality are obtained based on convex variational method andthe introduction of time-advanced stochastic differential equations (ASDEs) as ad-joint processes. To illustrate our theoretical results, some real examples are discussedin detail, for which optimal controls are derived explicitly by solving the associatedASDEs. Moreover, it is remarkable that the value range of time delay parameter δ

plays an important role to our maximum principle and the well-posedness of the cor-responding ASDEs. This current work also suggests some interesting topics that willbe discussed in our future research.

Acknowledgements The first author acknowledges the support from the Fundamental Research Fundsfor the Central Universities (2010QS05), P.R. China. The first author also thanks Department of AppliedMathematics, The Hong Kong Polytechnic University for their hospitality during her visit to Hong Kong.The second author acknowledges the support of RGC Earmarked grant 500909 and research fund of HongKong Polytechnic University (A-PL14).

References

1. Mohammed, S.E.A.: Stochastic Functional Differential Equations. Pitman Advanced Publishing Pro-gram, Boston (1984)

2. Mohammed, S.E.A.: Stochastic Differential Equations with Memory: Theory, Examples and Appli-cations. Stochastic Analysis and Related Topics 6, Progress in Probability. Birkhäuser, Basel (1996)

3. Peng, S., Yang, Z.: Anticipated backward stochastic differential equation. Ann. Probab. 37, 877–902(2009)

4. Delong, Ł., Imkeller, P.: Backward stochastic differential equations with time delayed generators-results and counterexamples. Ann. Appl. Probab. 20, 1512–1536 (2010)

5. Delong, Ł.: BSDEs with time-delayed generators of a moving average type with applications to non-monotone preferences. Stoch. Models 28, 281–315 (2012)

6. Delong, Ł.: Applications of time-delayed backward stochastic differential equations to pricing, hedg-ing and portfolio management. Working paper (2012)

7. Øksendal, B., Sulem, A.: A maximum principle for optimal control of stochastic systems with delay,with applications to finance. In: Menaldi, J.L., Rofman, E., Sulem, A. (eds.) Optimal Control andPartial Differential Equations, pp. 64–79. IOS Press, Amsterdam (2001)

8. Øksendal, B., Sulem, A., Zhang, T.: Optimal control of stochastic delay equations and time-advancedbackward stochastic differential equations. Adv. Appl. Probab. 43, 572–596 (2011)

9. Antonellia, F., Baruccib, E., Mancinoc, M.E.: Asset pricing with a forward-backward stochastic dif-ferential utility. Econ. Lett. 72, 151–157 (2001)

10. Federico, S.: A stochastic control problem with delay arising in a pension fund model. Finance Stoch.15, 421–459 (2011)

11. Chen, L., Wu, Z.: Maximum principle for the stochastic optimal control problem with delay andapplication. Automatica 46, 1074–1080 (2010)

J Optim Theory Appl

12. Larssen, B.: Dynamic programming in stochastic control of systems with delay. Stoch. Stoch. Rep.74, 651–673 (2002)

13. Kohlmann, M., Zhou, X.Y.: Relationship between backward stochastic differential equations andstochastic controls: a linear-quadratic approach. SIAM J. Control Optim. 38, 1392–1407 (2000)

14. Ma, J., Yong, J.: Forward–Backward Stochastic Differential Equations and Their Applications. Lec-ture Notes in Math., vol. 1702. Springer, Berlin (1999)

15. Dokuchaev, N., Zhou, X.Y.: Stochastic control with terminal contingent conditions. J. Math. Anal.Appl. 238, 143–165 (1999)

16. El Karoui, N., Peng, S., Quenez, M.C.: A dynamic maximum principle for the optimization of recur-sive utilities under constrains. Ann. Appl. Probab. 11, 664–693 (2001)

17. Lim, A., Zhou, X.Y.: Linear-quadratic control of backward stochastic differential equations. SIAM J.Control Optim. 40, 450–474 (2001)

18. Augustynowicz, A., Leszczynski, H., Walter, W.: On some nonlinear ordinary differential equationswith advanced arguments. Nonlinear Anal. 53, 495–505 (2003)

19. Cooke, K.L., Wiener, J.: An equation alternately to retarded and advanced type. Proc. Am. Math. Soc.99, 726–732 (1987)

20. Hall, A.J., Wake, G.C., Gandar, P.W.: Steady size distributions for cells in one dimensional plantissues. J. Math. Biol. 30, 101–123 (1991)

21. Kato, T., McLeod, J.B.: The functional-differential equation. Bull. Am. Math. Soc. 77, 891–937(1971)

22. Papavassilopoulos, G.P., Olsder, G.J.: On a linear differential equation of the advanced type. J. Math.Anal. Appl. 103, 74–82 (1984)

23. Yoneda, T.: On the functional-differential equation of advanced type. J. Math. Anal. Appl. 332, 487–496 (2006)

24. Cadenillas, A., Karatzas, I.: The stochastic maximum principle for linear convex optimal control withrandom coefficients. SIAM J. Control Optim. 33, 590–624 (1995)