MEAN FIELD GAMES WITH COMMON NOISE A …vt075xr1988/Thesis_main-augmented.pdfWe characterize the ﬁrst order approximation terms as the solution to a linear FBSDE of mean-ﬁeld type

MEAN FIELD GAMES WITH COMMON NOISE

A DISSERTATION

SUBMITTED TO THE DEPARTMENT OF MATHEMATICS

AND THE COMMITTEE ON GRADUATE STUDIES

OF STANFORD UNIVERSITY

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS

FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

Saran Ahuja

July 2015

http://creativecommons.org/licenses/by-nc/3.0/us/

This dissertation is online at: http://purl.stanford.edu/vt075xr1988

© 2015 by Saran Ahuja. All Rights Reserved.

Re-distributed by Stanford University under license with the author.

This work is licensed under a Creative Commons Attribution-Noncommercial 3.0 United States License.

ii



http://purl.stanford.edu/vt075xr1988

I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.

George Papanicolaou, Primary Adviser


Georg Menz


Leonid Ryzhik

Approved for the Stanford University Committee on Graduate Studies.

Patricia J. Gumport, Vice Provost for Graduate Education

This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file inUniversity Archives.

iii

Abstract

Mean field games (MFG) are a limit of stochastic differential games with a large

number of identical players. They were proposed and first studied by Lasry and Lions

and independently by Caines, Huang, and Malhame in 2006. They have attracted a

lot of interest in the past decades due to their application in many fields. By assuming

independence among each agent, taking the limit as N → ∞ reduces a problem to a

fully-coupled system of forward-backward partial differential equations (PDE). The

backward one is a Hamilton-Jacobi-Bellman (HJB) equation for the value function of

each player while the forward one is the Fokker-Planck (FP) equation for the evolution

of the players distribution. This limiting system is more tractable and one can use

its solution to approximate the Nash equilibrium strategy of N -player games.

In this thesis, we consider the MFG model in the presence of common noise,

relaxing the usual independence assumption of individual random noise. The presence

of common noise clearly adds an extra layer of complexity to the problem as the

distribution of players now evolves stochastically. Our first task is proving existence

and uniqueness of a Nash equilibrium strategy for this game, showing wellposedness

of MFG with common noise. We use a probabilistic approach, namely the Stochastic

Maximum Principle (SMP), instead of a PDE approach. This approach gives us a

forward-backward stochastic differential equation (FBSDE) of McKean-Vlasov type

instead of coupled HJB-FP equations. This was first done by Carmona and Delarue

in the case of no common noise and we extend their results to MFG with common

noise. We are able to extend their results under a linear-convexity framework and a

weak monotonicity assumption on the cost functions. In addition to wellposedness

results, we also prove the Markov property of McKean-Vlasov FBSDE by proving the

iv

existence of a decoupling function.

In the second part of this thesis, we consider ε-MFG models when the common

noise is small. For simplicity, we assume a quadratic running cost function while

keeping a general terminal cost function satisfying the same assumptions as in the

first part. Our goal is to give an approximation of Nash equilibrium of this game

using the solution from the original MFG with no common noise, which could be

described through a finite-dimensional system of PDEs. We characterize the first

order approximation terms as the solution to a linear FBSDE of mean-field type.

We then show that the solution to this FBSDE is a centered Gaussian process with

respect to the common noise. By assuming regularity of the decoupling function of

the 0-MFG problem, we can find an explicit solution showing that they are in the

form of a stochastic integral with respect to the common noise with the integrands

adapted to the information from the 0-MFG only. We then are able to compute the

covariance function explicitly.

v

Acknowledgments

I would like to express my deepest gratitude to Professor George Papanicolaou, my

principal advisor, for his continuous supports and thoughtful guidances towards my

research. His insights and advices have been extremely helpful and his enthusiasm

for mathematics has been inspiring. I am very grateful to have learned and grown so

much as a researcher from working closely with him in the past five years. I would

like to thank the other members of my committee; Professor Lenya Ryzhik, Professor

Georg Menz, Professor Lexing Ying, and Professor Kay Giesecke for taking time out

from their busy schedules to serve in my committee.

I would also like to acknowledge Dr.Tzu-Wei Yang for his tremendous help and

encouragement. He is always available to talk whenever I seek advice and always

gives helpful feedbacks. I also thank David Ren for all the discussions we had and for

carefully reading this thesis and providing useful comments on my work.

I am very thankful for all the friends and colleagues I have met during my time at

Stanford. I am particularly thankful for being a part of the strong Thai community

here which always makes me feel like home. My special thanks goes to Eve, my

wonderful girlfriend and my best friend, whose love and support has been invaluable.

I cannot express how fortunate I am to have her by my side throughout this eventful

journey. Lastly and most importantly, I am very grateful for my family in Thailand;

my mom and dad, my brother and sister, who always believe in me and continuously

send their love, supports, and encouragement from afar.

vi

Contents

Abstract iv

Acknowledgments vi

1 Introduction 1

1.1 Model setup, notations, and main assumptions . . . . . . . . . . . . . 4

1.1.1 N -player stochastic differential game . . . . . . . . . . . . . . 5

1.1.2 Definition of mean field games . . . . . . . . . . . . . . . . . . 6

1.1.3 Derivative with respect to a probability measure . . . . . . . . 9

1.1.4 Main assumptions . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.2 Summary of the main results . . . . . . . . . . . . . . . . . . . . . . . 19

1.2.1 Two approaches to mean field games . . . . . . . . . . . . . . 20

1.2.2 Mean field games with common noise . . . . . . . . . . . . . . 27

1.2.3 Asymptotic analysis of mean field games . . . . . . . . . . . . 33

1.2.4 Linear quadratic mean field games with common noise . . . . 39

1.3 Summary of contributions . . . . . . . . . . . . . . . . . . . . . . . . 42

2 Two approaches to mean field games 44

2.1 Dynamic Programming Principle (DPP) . . . . . . . . . . . . . . . . 44

2.2 Stochastic Maximum Principle (SMP) . . . . . . . . . . . . . . . . . 52

2.3 Comparison between the two approaches . . . . . . . . . . . . . . . . 60

3 Mean field games with common noise 64

3.1 Wellposedness of MFG with common noise . . . . . . . . . . . . . . . 64

vii

3.1.1 FBSDE with monotone functionals . . . . . . . . . . . . . . . 65

3.1.2 A priori estimate . . . . . . . . . . . . . . . . . . . . . . . . . 67

3.1.3 Wellposedness result . . . . . . . . . . . . . . . . . . . . . . . 68

3.2 Markov property and a decoupling function . . . . . . . . . . . . . . 74

3.3 Master equation and connection to DPP approach . . . . . . . . . . . 82

3.4 Proof of lemmas, propositions, and theorems . . . . . . . . . . . . . . 86

3.4.1 Proof of Theorem 3.1.1 . . . . . . . . . . . . . . . . . . . . . . 86

3.4.2 Proof of Theorem 3.1.2 . . . . . . . . . . . . . . . . . . . . . . 91

3.4.3 Proof of Theorem 3.2.8 . . . . . . . . . . . . . . . . . . . . . . 95

4 Asymptotic analysis of mean field games 97

4.1 Linear variational FBSDE . . . . . . . . . . . . . . . . . . . . . . . . 98

4.1.1 Wellposedness result . . . . . . . . . . . . . . . . . . . . . . . 100

4.1.2 Convergence result . . . . . . . . . . . . . . . . . . . . . . . . 101

4.2 Approximate Nash equilibrium . . . . . . . . . . . . . . . . . . . . . . 102

4.3 Gaussian property and decoupling function . . . . . . . . . . . . . . . 104

4.3.1 Centered Gaussian process . . . . . . . . . . . . . . . . . . . . 104

4.3.2 Decoupling function . . . . . . . . . . . . . . . . . . . . . . . 105

4.4 Explicit solution and covariance function . . . . . . . . . . . . . . . . 108

4.4.1 Sensitivity functional . . . . . . . . . . . . . . . . . . . . . . . 110

4.4.2 Explicit solution . . . . . . . . . . . . . . . . . . . . . . . . . 114

4.4.3 Covariance function . . . . . . . . . . . . . . . . . . . . . . . . 114

4.5 Proof of lemmas, propositions, and theorems . . . . . . . . . . . . . . 116

4.5.1 Proof of Theorem 4.1.2 . . . . . . . . . . . . . . . . . . . . . . 116

4.5.2 Proof of Theorem 4.2.5 . . . . . . . . . . . . . . . . . . . . . . 118

4.5.3 Proof of Theorem 4.3.1 . . . . . . . . . . . . . . . . . . . . . . 119

4.5.4 Proof of Theorem 4.3.2 . . . . . . . . . . . . . . . . . . . . . . 120

5 Linear-quadratic MFG with common noise 123

5.1 DPP approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

5.1.1 Master Equation . . . . . . . . . . . . . . . . . . . . . . . . . 125

5.1.2 Optimal controlled process and stochastic value function . . . 127

viii

5.2 SMP approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

5.3 Asymptotic analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Bibliography 131

ix

List of Tables

2.1 Summary of the types of equations from applying DPP and SMP ap-

proaches to MFG problems . . . . . . . . . . . . . . . . . . . . . . . . 62

x

List of Figures

xi

Chapter 1

Introduction

Mean field games, or MFG for short, were recently proposed by Lasry and Lions in

a series of papers [13, 42, 43, 44] and independently by Caines, Huang, and Mal-

hame [37], who called them Nash Certainty Equivalence. An MFG is a limit of a

stochastic differential game with a large number of players, symmetric cost, and weak

interaction. Specifically, each player executes a stochastic control problem whose cost

and/or dynamics depend not only on their own state and control but also on other

players’ states. However, this interaction is weak in a sense that a player feels the

effect of other players only through their empirical distribution. Searching for a Nash

equilibrium strategy for an N -player games is known to be intractable when N is

large as the dimensionality grows with the number of players. However, by assuming

independence of the random noise in the players’ state processes, symmetry of the

cost functions, and a mean-field interaction, taking the limit as N → ∞ reduces

a problem to solving a fully-coupled system of forward-backward partial differential

equations (FBPDEs). The backward one is a Hamilton-Jacobi-Bellman (HJB) equa-

tion for the value function for each player while the forward one is the Fokker-Planck

(FP) equation for the evolution of the player’s probability distribution. This limit

system is more tractable and one can use its solution to approximate Nash equilibrium

strategies of the N -player games. Lasry and Lions studied this model extensively and

rigorously analyzed the limit in some cases [42, 43, 44].

Alternatively, one can apply the Stochastic Maximum Principle (SMP) to the

1

CHAPTER 1. INTRODUCTION 2

control problem as opposed to the Dynamic Programming Principle (DPP). Instead

of the HJB equation, we have a forward backward stochastic differential equations

(FBSDE). In the MFG setting, this FBSDE is of McKean-Vlasov type as it also

involves the law of the process. This probabilistic analysis of MFG was first done by

Carmona and Delarue in [17] and is the framework that we follows in this thesis.

In the past decade, active research has been done in this area and tremendous

progress has been made in several directions. See [32] for a brief survey and [9]

for a more extensive reference. These include a finite state model [29, 30], a model

with major/minor players [8, 23, 36, 50, 51], and a study of convergence from finite

player games to MFG [7, 27, 30, 40]. One important assumption in most of the prior

work is independence of the random factors in each player’s state processes. From

this assumption and the Law of Large Numbers, the distribution of players’ state

evolves deterministically in the limit. It is this property that plays a key role in

reducing the dimension of this complex problem and making it tractable. However,

many models in applications do not satisfy this assumption. For instance, in financial

applications, any reasonable model which attempts to understand the interactions of

a large number of market participants will have to assume that they are exposed to

some type of overall market randomness. This common random factor is applied to

all the players, and, as a result, the independence assumption does not hold. See

[2, 21, 33, 45] for examples of mean field game models with common noise in finance

and economics.

The presence of common noise clearly adds an extra layer of complexity to the

problem as the empirical distribution of players in the limit now evolves stochastically.

One approach would be to abandon the forward backward coupling and simply add the

players’ distribution as an argument in the value function. Using the Dynamic Pro-

gramming Principle, this generalized value function will satisfy an infinite-dimensional

HJB equation referred to as the master equation. This equation requires new theo-

ries and tools as it is a second order infinite-dimensional HJB involving derivatives

with respect to probability measures. See [10, 18, 31] for some discussion on the

master equation. Alternatively, we could revisit the forward-backward equations in

the presence of common noise. Following the PDE approach of Lasry and Lions, the


common noise then turns HJB-FP equations to stochastic HJB-FP equations which

are of forward backward stochastic partial differential equation (FBSPDE for short)

type. This FBSPDE is clearly more complicated than the FBPDE counterpart in

the original mean field game. On the other hands, as we shall see, the probabilistic

approach introduced by Carmona and Delarue in [17] can be extended more naturally

to accommodate the common noise. The law of the state process which occurs in the

McKean-Vlasov FBSDE from the SMP will simply be replaced by its conditional law

given the common Brownian motion.

In the first part of this thesis, we are interested in a general MFG model with

common noise. By using SMP, we turn the problem to analyzing a McKean-Vlasov

FBSDE. Our main result is establishing existence and uniqueness of a strong solution

to this FBSDE under a linear-convexity framework and a weak monotonicity condition

on the cost functions. We also prove a Markov property of the solution by showing

the existence of a deterministic function that decouples the FBSDE. The latter gives

a clear connection to the DPP approach. The material in this part is mostly from

our work in [3].

Recently, in addition to [3], there has been progress in this direction concerning

mostly well-posedness results. Carmona et. al. [20] give a general existence and

uniqueness result of a weak solution. In [10, 18], the master equation was discussed

from the perspective of both HJB and probabilistic approaches. Despite these results,

a general common noise model is still difficult and impractical to solve numerically

or explicitly as it does not enjoy the dimension reduction property as in the case of

MFG without common noise. In the original MFG model of Lasry and Lions, we deal

with a system of PDEs, hence numerical schemes can be developed to approximate

the solution. See [1] for analysis and applications. Intuitively, when there is no

common noise, the law m0,αt is expected to be deterministic, so we only need to find

the optimal strategy along the equilibrium distribution. A common noise model,

on the other hand, is much more complex as the flow of the players’ distribution is

stochastic. This means that we need to specify the optimal action for all possible

trajectories of the players’ distribution which is infinite-dimensional. We would like

to point out that while a common noise model will most likely be intractable, under


special circumstances, it might be explicitly solvable through a certain transformation

that turns the problem to the original MFG. See [21, 33] for some examples of this

sort and [41] for a more general treatment of what is called a translation-invariant

MFG with common noise.

In the second part of this thesis, we consider MFG problems when the common

noise is small as shall be indicated by the parameter ε. In this set up, it is reasonable

to seek an approximate solution using only finite-dimensional information from the

ε = 0 problem, i.e. the original MFG with no common noise. We prove that the

first order approximation can be characterized as the solution to a linear FBSDE of

mean-field type. We then show that this FBSDE is uniquely solvable and its solution

is a mean-zero Gaussian process with respect to a common Brownian motion. Our as-

sumptions are similar to those in the first part with additional regularity assumptions.

Furthermore, by assuming that the decoupling function of McKean-Vlasov FBSDE is

sufficiently regular, the linear FBSDE described above can be solved explicitly. The

solution is Wt-path dependent in the form of a stochastic integral of dWt with the

integrand adapted to the information from 0-MFG only. We then are able to compute

the covariance function explicitly.

This chapter is organized as follows; In Section 1.1.1-1.1.2, we introduce a model

for an N -player stochastic differential game and formulate its limit, an MFG problem

with common noise. We then discuss a notion of differentiating with respect to a

probability measure in Section 1.1.3, and list all the main assumptions that will be

used throughout the thesis in Section 1.1.4. Lastly, we summarize all the results in

Section 1.2 and summarize our contributions in Section 1.3.

1.1 Model setup, notations, and main assumptions

In this section, we describe a stochastic differential game model with N players, then

formulate an MFG model by formally taking the limit as N → ∞.


1.1.1 N-player stochastic differential game

Let T be a fixed terminal time, W i = (W it )0≤t≤T , i = 1, 2, . . . , N, W = (Wt)0≤t≤T

are one dimensional independent Brownian motions. Consider a stochastic dynamic

game with N players, where each player i ∈ 1, 2, . . . , N controls a state process

(X it)0≤t≤T in R given by

dX it = αi

tdt+ σdW it + εdWt, X i

0 = ξi0

by selecting a control αi = (αit)0≤t≤T in H2([0, T ];R), the set of progressively measur-

able processes β = (βt)0≤t≤T such that

E

[∫ T

0

β2t dt

]

<∞

where ξi0 is an initial state of player i. We assume that (ξi0)1≤i≤N are independent

identically distributed, independent of all Brownian motions, and satisfy E[(ξi0)2] <∞

for all 1 ≤ i ≤ N . We will refer to W i as an individual noise or idiosyncratic noise

and W as a common noise.

Given the other players’ strategies, player i selects a control αi ∈ H2([0, T ];R) to

minimize his/her expected total cost given by

J i(αi|(αj)j 6=i) , E

[∫ T

0

f(t, X it , mt, α

it)dt+ g(X i

T , mT )

]

where (αj)j 6=i denotes a strategy profile of other players excluding i, f : [0, T ]× R×

P(R)× R, g : R× P(R) → R are the running and terminal cost functions which are

identical for all players, P(R) denotes the space of Borel probability measure on R,

and mt denote the empirical distribution of (X it)1≤i≤N , i.e.

mt =1

N

N∑

i=1

δXit(dx)

Note that the strategies of the other players have an effect on the cost of player i


through mt. This interaction is the main feature that makes this set up a game. We

are seeking a type of equilibrium solution widely used in game theory settings called

a Nash equilibrium.

Definition 1.1.1. A set of strategies (αi)1≤i≤N is a Nash Equilibrium if for every

player i, αi is an optimal control given the other players’ strategies are (αj)j 6=i. In

other words,

J i(αi|(αj)j 6=i) = minα∈H2([0,T ];R)

J i(α|(αj)j 6=i), ∀i ∈ 1, 2, . . . , N

Since the cost function of each player is identical, if the control problem for each

player has a unique solution, then clearly the Nash equilibrium strategy must be

symmetric. However, solving this problem is impractical when N is large due to

high dimensionality, so we formally take the limit as N → ∞ and consider the limit

problem, called a mean field game, instead. Solving a mean field game problem yields

a control that can be used to approximate the exact Nash equilibrium for an N -player

game. See [9, 17] for discussions and results on an approximate Nash equilibrium for

N -player games.

1.1.2 Definition of mean field games

We now formulate an MFG problem in the presence of common noise by formally

taking a limit as N → ∞. By considering the limit problem and assuming that each

player adopts the same strategy, one can represent the distribution of players mt by

the law of a single representative player. However, since the common noise is applied

to all the players, this law is a conditional law given a common noise path. In other

words, we formulate the MFG with common noise as a stochastic control problem for

a single (representative) agent with an equilibrium condition involving a conditional

law of the state process.

Fix a terminal time T > 0. Let (Wt)0≤t≤T , (Wt)0≤t≤T be two independent Brown-

ian motions defined on a complete filtered probability space (Ω,F ,F = Ft0≤t≤T ,P),

augmented by all the P-null sets. (Wt)0≤t≤T is called individual or idiosyncratic noise


and (Wt)0≤t≤T is called common noise. Let P2(R) denote the space of Borel proba-

bility measures on R with finite second moment, i.e. all Borel probability measure µ

such that∫

R

x2dµ(x) <∞

It is a complete separable metric space equipped with a Wasserstein metric defined

as

W2(m1, m2) =

(

infγ∈Γ(m1,m2)

∫

R2

|x− y|2γ(dx, dy)

)12

(1.1)

where Γ(m1, m2) denotes the collection of all probability measures on R2 with marginals

m1 andm2. Let F st denote the filtration generated by Wr−Ws, s ≤ r ≤ t and Ft = F0

t .

Suppose G is a sub σ-algebra of F and G = Gt0≤t≤T is a sub filtration of F, then

let L2G denote the set of G-measurable real-valued square integrable random vari-

ables, L2G(P2(R)) denote the set of G-measurable random probability measures µ on

R with finite second moment, and H2G([0, T ];R) denote the set of all Gt-progressively-

measurable process β = (βt)0≤t≤T such that

E

[∫ T

0

β2t dt

]

<∞

We define similarly the space H2G([s, t];R) for any 0 ≤ s < t ≤ T and we will often

omit the subscript and writeH2([0, T ];R) forH2F([0, T ];R). We also letM([0, T ];P2(R))

denote the space of continuous Ft-adapted stochastic flow of probability measures

µ = (µt)0≤t≤T with values in P2(R) such that

E

[

sup0≤t≤T

∫

R

y2dµt(y)

]

<∞

and define similarly M([s, t],R).

Let ξ0 ∈ L2F0

be an initial state. For any control α ∈ H2([0, T ];R), we denote by

Xα = (Xαt )0≤t≤T the corresponding state process, i.e.

Xαt = ξ0 +

∫ t

0

αtdt+ σWt + εWt, (1.2)


and mαt is the law of Xα

t conditional on Ft, i.e.

mαt , L(Xα

t |Ft), ∀t ∈ [0, T ] (1.3)

where L(·|Ft) denotes the conditional law given Ft. It is easy to check that when

α ∈ H2([0, T ];R) and E[ξ20 ] < ∞, mα = (mαt )0≤t≤T ∈ M([0, T ];P2(R)), i.e. it has a

continuous trajectory with finite second moment for all t ∈ [0, T ] a.s. We first define

a control problem for a single player given a stochastic flow of probability measures.

Let f : [0, T ] × R × P2(R) × R → R, g : R × P2(R) → R be measurable functions

denoting a running and terminal cost function.

Definition 1.1.2. Given a stochastic flow of probability measure m = (mt)0≤t≤T ∈

M([0, T ];P2(R)), a single player stochastic control problem with state process

dXαt = αtdt+ σdWt + εdWt, X0 = ξ0

and cost

J ε(α|m) = E

[∫ T

0

f(t, Xαt , mt, αt)dt+ g(Xα

T , mT )

]

is called an individual control problem given m.

Remark 1.1.3. Note that because m = (mt)0≤t≤T is stochastic, the individual control

problem given m is a control problem with random cost.

We are now ready to state the definition of an MFG with common noise.

Definition 1.1.4. a control α = (αt)0≤t≤T ∈ H2([0, T ];R) is called a solution to MFG

with common noise if an optimal control for an individual player given (mαt )0≤t≤T is

α. In other words, α satisfies

J ε(α|mα) ≤ J ε(α|mα), ∀α ∈ H2([0, T ];R)

We will often refer to the MFG problem described above as ε-MFG to emphasize

the existence and magnitude of the common noise and call α a solution to ε-MFG


with initial ξ0. Note that 0-MFG is simply the original MFG with no common noise

arising from an N -player game with independent Brownian motions.

We would like to emphasize that in the control problem above, mα is exogenous

and is not affected by a player’s control. Thus, MFG is a standard control problem

with an additional equilibrium condition. A type of problem where a player’s control

can affect the law is referred to as Mean Field Type Control Problem. In that setting,

we are searching for a control α such that

J ε(α|mα) ≤ J ε(α|mα), ∀α ∈ H2([0, T ];R)

Notice the difference in the RHS of the inequality. See [9, 16, 19] for analysis on this

different model and some discussion of the differences between the two problems.

The MFG problem described above can also be viewed as a fixed point problem

as follows. Given a strategy α ∈ H2([0, T ];R), then mα is determined as defined in

(1.3). We then solve an individual control problem given mα (see Definition 1.1.2).

This step yields a new optimal control α. The following diagram summarizes the

process

α = (αt)0≤t≤T → mα = (mαt )0≤t≤T → α = (αt)0≤t≤T (1.4)

By the definition of ε-MFG, α is an ε-MFG solution if and only if it is a fixed point

of this map.

1.1.3 Derivative with respect to a probability measure

From the set up of MFG problems, we see that the distribution of a player evolves

stochastically and, as a result, some notion of optimization, hence differentiation, over

a probability measure is unavoidable. In this section, we discuss a notion of derivative

for a function with a probability measure as its argument.

A notion of derivative of a function on the space of probability measures was first

defined using a geometric approach. See [4, 57] for extensive treatments on the subject

in this direction. In this work, however, it is more convenient to use an alternative

approach which is more probabilistic in nature. This method was introduced by Lions


in his lecture at the College de France[13]. Since then, many works on MFG have

employed this notion of derivative. While we will only discuss the results that are

necessary for our work here, we refer interested readers to [16] or to [24] for more

details on this framework.

The idea is based on lifting up a function on a space of probability measures to a

function on a space of random variables. When the space of probability measures we

are working on is P2(R), this method is extremely useful because it allows us to work

on a Hilbert space of square integrables random variable instead of a metric space

P2(R). Consequently, we are able to use a notion of Frechet derivative in Hilbert

space to help define a derivative.

Given a functional V : P2(R) → R, we define the lifting function V : L2(Ω, F , P) →

R by

V (X) = V (L(X))

where (Ω, F , P) is an arbitrary probability space with Ω being a Polish space and

P being atomless. This is sufficient to ensure that given any m ∈ P2(R), there

exist a random variable X such that L(X) = m, where L(X) denotes the law of X .

Suppose there exists a random variable X0 ∈ L2(Ω, F , P) such that L(X0) = m0 and

V is Frechet differentiable at X0, denote its derivative by V ′(X0)(·), then by Reiz

representation theorem, there exist an element DV (X0) in L2(Ω, F , P) such that

V ′(X0)(Y ) = E[DV (X0) · Y ]

It then can be shown that if V is differentiable at X0, then it is differentiable at

any X ∈ L2(Ω, F , P) with L(X) = m0 and the law of DV (X) is independent of the

choice of X . See Theorem 6.2 in [13]. Furthermore, it can be shown that there exist

a function h ∈ L2m0

(R;R) uniquely defined m0-a.s. such that

DV (X) = h(X)

See Theorem 6.5 in [13]. As a result, it is natural to call this function h the derivative

of V with respect to m and we shall denote it by ∂mV . Thus, the following relation


holds, for any X, Y ∈ L2(Ω, F , P)

V (L(X + Y )) = V (L(X)) + E[∂mV (L(X))(X)Y ] + o(‖Y ‖2)

From the definition, ∂mV (m0)(·) is only uniquely-defined m0-a.s.. Thus, to consider

jointly the map (m, z) → ∂mV (m)(z), we need to specify a canonical version to be

used. When DV is Lipschitz, then it can be shown (see Lemma 3.2 in [16]) that

there exist a version of ∂mV such that z → ∂mV (m)(z) is Lipshitz with the same

constant for all m ∈ P2(R). We shall take this version whenever possible. The higher

order derivative can be defined similarly by using the Frechet derivative on the lifting

function to get the second derivative of the form ∂mmV (m0)(·, ·) ∈ L2m0

(R2;R).

Example 1.1.5. Consider a general class of functionals V : P2(R) → R of the form

V (m) = F

(∫

φ(z)dm(z)

)

where φ, F : R → R are continuously differentiable. Then the lifting function V :

L2(Ω, F , P) → R is

V (X) = F (E [φ(X)])

Thus,

DV (X)(Y ) = F ′(E [φ(X)])(E [φ′(X)Y ]

so that

∂mV (m)(z) = F ′

(∫

φ(u)dm(u)

)

φ′(z)

In the context of MFG with common noise, we will be dealing with a stochastic

flow of probability measures m = (mt)0≤t≤T ∈ M([0, T ];P2(R)) which arises from the

law conditional on Ft of a state process

Xt = ξ0 +

∫ t

0

αsds+ σWt + εWt, ∀t ∈ [0, T ]

To take the derivative using this notion at mt = L(Xt|Ft), we need to find a random

variable to represent it. An obvious choice is simply the state process X = (Xt)0≤t≤T


itself. To do this in an explicit manner, we first need to separate the path space

for individual noise and common noise. From now and throughout the rest of the

thesis, we assume that (Ω,F ,P) is in the form (Ω0 × Ω,F0 ⊗ F ,P0 ⊗ P) where the

individual noise Wt and common noise Wt are supported in the space (Ω0,F0,P0)

and (Ω, F , P) respectively. We will also assume that (Ω, F , P) is the canonical sample

space of the Brownian motion (Wt)0≤t≤T , so that F0 is trivial. To avoid confusion

between a lifting space and the original space, we let (Ω0, F0, P0) denote a copy of

(Ω0,F0,P0) and write

(Ω, F , P) = (Ω0 × Ω, F0 ⊗ F , P0 ⊗ P)

to denote a copy of (Ω,F ,P) sharing a common noise space. For any X ∈ L2(Ω,F ,P),

we denote by X , a copy of X in L2(Ω, F , P). Also, we let E0[·] denote the expectation

with respect to P0 only, or equivalently, the expectation with respect to P conditional

on FT .

In addition to introducing a “copy” space sharing a common noise, it will also

be convenient to have a random variable independent of all Brownian motion to

represent an arbitrary law m0 ∈ P2(R). So we further assume that (Ω,F ,P), and

hence (Ω, F , P) as well, is sufficiently rich that for any m0 ∈ P2(R), there exist an

F0-measurable random variable ξ such that L(ξ) = m0. We end this section with the

following lemma which will be useful in the later chapters. The result is standard

in the optimal transport theory and we refer the readers to [55] (see, in particular,

Theorem 3.2.9).

Lemma 1.1.6. Let (Ω, F , P) be a probability space with Ω being a Polish space and

P being an atomless measure. Given m0 ∈ P2(R) and ξ0 ∈ L2(Ω, F , P) with law m0,

then for any m ∈ P2(R), there exists ξ ∈ L2(Ω, F , P) with law m such that

E[(ξ0 − ξ)2] = W22 (m0, m)


1.1.4 Main assumptions

In this section, we list all the main assumptions on the cost functions that will be

used in the thesis.

The first set of assumptions, assumption A, is essential for ensuring that given any

stochastic flow of probability measure m = (mt)0≤t≤T ∈ M([0, T ],R), the stochastic

control for an individual player given m (see Definition 1.1.2) is uniquely solvable.

Assumption A consists of standard Lipschitz, linear and quadratic growth, and con-

vexity conditions. For notational convenience, we will use the same constant K for

all the conditions below.

(A1). (Lipschitz) ∂xf, ∂αf, ∂xg exist and are K-Lipschitz continuous in (x, α) uni-

formly in (t,m); for any t ∈ [0, T ], x, x′, α, α′ ∈ R, m ∈ P2(R).

|∂xg(x,m)− ∂xg(x′, m)| ≤ K|x− x′|

|∂xf(t, x,m, α)− ∂xf(t, x′, m, α′)| ≤ K (|x− x′|+ |α− α′|)

|∂αf(t, x,m, α)− ∂αf(t, x′, m, α′)| ≤ K (|x− x′|+ |α− α′|)

(1.5)

(A2). (Growth) ∂xf, ∂αf, ∂xg satisfy a linear growth condition; for any t ∈ [0, T ], x, α ∈

R, m ∈ P2(R),

|∂xg(x,m)| ≤ K

(

1 + |x|+

(∫

R

y2dm(y)

)12

)

|∂xf(t, x,m, α)| ≤ K

(

1 + |x|+ |α|+

(∫

R

y2dm(y)

)12

)

|∂αf(t, x,m, α)| ≤ K

(

1 + |x|+ |α|+

(∫

R

y2dm(y)

)12

)

(1.6)

In addition, f, g satisfy a quadratic growth condition in m.

|g(0, m)| ≤ K

(

1 +

∫

R

y2dm(y)

)

|f(t, 0, m, 0)| ≤ K

(

1 +

∫

R

y2dm(y)

) (1.7)


(A3). (Convexity) g is convex in x and f is convex jointly in (x, α) with strict con-

vexity in α. That is, for any x, x′ ∈ R, m ∈ P2(R),

(∂xg(x,m)− ∂xg(x′, m))(x− x′) ≥ 0 (1.8)

and there exist a constant cf > 0 such that for any t ∈ [0, T ], x, x′, α, α′ ∈ R, m ∈

P2(R),

f(t, x′, α′, m) ≥ f(t, x, α,m)+∂xf(t, x, α,m)(x′−x)+∂αf(t, x, α,m)(α′−α)+cf |α′−α|2

(1.9)

The Lipschitz and linear growth conditions (A1),(A2) are standard in SDE theory

to ensure the existence of a strong solution. The convexity assumption (A3) is essen-

tial to our setup in various ways. First, it ensures that the Hamiltonian is strictly

convex, so that there is a unique minimizer in a feedback form. In addition, it satisfies

sufficient conditions for the SMP so that solving an optimal control problem can be

translated to solving the corresponding FBSDE. See Section 6.4.2 in [53] for instance.

Lastly, it gives a monotonicity property for the FBSDE corresponding to an individual

player control problem so that it is uniquely solvable. We refer to [34, 52] for general

solvability properties of standard FBSDEs related to convex control problems.

The second set of assumptions, assumption B, is specific to MFG and are condi-

tions on the m-argument in the cost functions. These assumptions are essential in

showing the wellposed-ness of ε-MFG.

(B1). (Lipschitz in m) ∂xg, ∂xf is Lipschitz continuous in m uniformly in (t, x), i.e.

there exist a constant K such that

|∂xg(x,m)− ∂xg(x,m′)| ≤ KW2(m,m

′)

|∂xf(t, x,m, α)− ∂xf(t, x,m′, α)| ≤ KW2(m,m

′)(1.10)

for all t ∈ [0, T ], x, α ∈ R, m,m′ ∈ P2(R), where W2(m,m′) is the second order

Wasserstein metric defined by (1.1). This is equivalent to the following; for any


t ∈ [0, T ], x, α ∈ R, ξ, ξ′ ∈ L2(Ω, F , P;R) where (Ω, F , P) is arbitrary,

|∂xg(x,L(ξ))− ∂xg(x,L(ξ′))| ≤ K‖ξ − ξ′‖2

|∂xf(t, x,L(ξ), α)− ∂xf(t, x,L(ξ′), α)| ≤ K‖ξ − ξ′‖2

(1.11)

where ‖ · ‖2 denote the L2-norm

(B2). (Separable in α,m) f is of the form

f(t, x,m, α) = f 0(t, x, α) + f 1(t, x,m) (1.12)

where f 0 is assumed to be convex in (x, α) strictly in α, f 1 is assumed to be convex

in x.

(B3). (Weak monotonicity) For all t ∈ [0, T ], m,m′ ∈ P2(R) and γ ∈ P2(R2) with

marginals m,m′ respectively,

∫

R2

[(∂xg(x,m)− ∂xg(y,m′))(x− y)] γ(dx, dy) ≥ 0

∫

R2

[(∂xf(t, x,m, α)− ∂xf(t, y,m′, α))(x− y)]γ(dx, dy) ≥ 0

(1.13)

Equivalently, for any x ∈ R, ξ, ξ′ ∈ L2(Ω, F , P;R) where (Ω, F , P) is arbitrary,

E[∂xg(ξ,L(ξ))− ∂xg(ξ′,L(ξ′))(ξ − ξ′)] ≥ 0

E[∂xf(t, ξ,L(ξ), α)− ∂xf(t, ξ′,L(ξ′), α)(ξ − ξ′)] ≥ 0

(1.14)

Lipschitz (in m) condition is similar to that used in other works for 0-MFG model.

Separability condition (B2) is not necessary in establishing existence for 0-MFG, but

is needed for the uniqueness result. See Proposition 3.7 and 3.8 in [17] for instance.

In our case, we rely on the monotonicity property of the McKean-Vlasov FBSDE

and this separability condition is necessary for establishing this property. Our last

assumption (B3) is new and can be viewed as a stronger version of the weak mean-

reverting assumption used in [17] to show the existence result of 0-MFG and a weaker

version of Lasry and Lions’ monotonicity assumption used in [13, 17, 31] to show the


uniqueness result. We discuss this point in more detail below.

The next set of assumptions, assumption C, concerns the second order derivative

of the cost functions. This is necessary for our asymptotic analysis in Chapter 4.

(C1). ∂xf, ∂xg are differentiable in (x,m) with Lipschitz continuous and bounded

derivative. Denote their bound and Lipschitz constant by the same K. Specifically for

∂mxf, ∂mxg, they satisfy, for all t ∈ [0, T ], x, α ∈ R, m,m′ ∈ P2(R), and ξ, ξ′ ∈ L2

F

with law m,m′,

E[∂mxf(t, x,m, α)(ξ)2]

12 ≤ K

E[∂mxg(x,m)(ξ)2]12 ≤ K

E[(∂mxf(t, x,m, α)(ξ)− ∂mxf(t, x,m′, α)(ξ′))2]

12 ≤ K‖ξ − ξ′‖2

E[(∂mxg(x,m)(ξ)− ∂mxg(x,m′)(ξ′))2]

12 ≤ K‖ξ − ξ′‖2

(1.15)

Remark 1.1.7. The function ∂mxf, ∂mxg involve the derivative with respect to a

probability measure. We follow the framework introduced by Lasry and Lions in [13]

which is based on a Frechet derivative of a lifting function defined on a space of

random variables. See Section 1.1.3 for more detail.

Our last assumption, assumption D, concerns the second derivative with respect

to m. This assumption was used in [24] to prove differentiability in m-argument of

the decoupling function and we will use this result in Chapter 4 to find the decoupling

function of the linear variational FBSDE.

(D1). For all m ∈ P2(R), the map (x, z) 7→ ∂mxf(t, x,m, α)(z), ∂mxg(x,m)(z), are

continuously differentiable and satisfy; for all t ∈ [0, T ], x, x′, α ∈ R,m,m′ ∈ P2(R),

and ξ, ξ′ ∈ L2F with law m,m′,

E

[

(∂z∂mxf(t, x,m, α)(ξ)− ∂z∂mxf(t, x′, m′, α)(ξ′))

2]

12≤ K (|x− x′|+ ‖ξ − ξ′‖2)

E

[

(∂z∂mxg(x,m)(ξ)− ∂z∂mxg(x′, m′)(ξ′))

2]

12≤ K (|x− x′|+ ‖ξ − ξ′‖2)

(1.16)


Comparison to assumptions for 0-MFG

Assumptions A and (B1) are similar to those used in [17] where they establishes

existence of 0-MFG solution under standard Lipschitz and linear-convexity model.

In addition, they also assume the following weak mean reverting assumption which

states that there exist a constant C > 0 such that for all t ∈ [0, T ], x ∈ R

x∂xf(t, 0, δx, 0) ≥ −C(1 + |x|)

x∂xg(0, δx) ≥ −C(1 + |x|)(1.17)

where δx denote the Dirac measure at x. Our weak monotonicity assumption (B3)

can be viewed as a stronger version of this assumption. This can be seen easily by

plugging in deterministic ξ = x, ξ′ = 0 in (1.14).

From the uniqueness result, the main assumption in the literature [13, 17, 31] is

the separability assumption and the Lasry and Lions’ monotonicity assumption which

states that∫

(h(x,m1)− h(x,m2))d(m1 −m2)(x) ≥ 0

for anym1, m2 ∈ P2(R). This condition can be expressed in terms of random variables

as follows; For any ξ, ξ′ ∈ L2 (over an arbitrary probability space).

E [h(ξ′,L(ξ′)) + h(ξ,L(ξ))− h(ξ,L(ξ′))− h(ξ′,L(ξ))] ≥ 0 (1.18)

Our weak monotonicity assumption (B3) is, as the name suggests, a weaker version

of (1.18) when the cost functions are convex. This is shown in the proposition below

Proposition 1.1.8. Let h : R × P2(R) → R be a convex continuously differentiable

function satisfying the monotonicity condition (1.18) stated above, then h satisfies the

weak monotonicity condition (1.14).

Proof. Suppose (1.18) holds. Let ξ, ξ′ ∈ L2, then by convexity in x of h, we get

h(ξ′,L(ξ′))− h(ξ,L(ξ′)) ≤ ∂xh(ξ′,L(ξ′))(ξ′ − ξ)


and

h(ξ,L(ξ))− h(ξ′,L(ξ)) ≤ −∂xh(ξ,L(ξ))(ξ′ − ξ)

Summing up, taking expectation, and using (1.18) yields (1.14).

The converse of the proposition above does not hold. Example (1.20) below gives

a cost function that is convex in x, satisfies (1.14), but does not satisfy (1.18). Thus,

our result gives a more general uniqueness theorem for 0-MFG.

Examples of cost functions

We consider a rather general class of f, g of the form

f(t, x,m, α) = F0(t, x, α) + F1

(

t, x,

∫

φf(z)dm(z)

)

+ F2

(∫

ψf (z)dm(z)

)

g(x,m) = G1

(

x,

∫

φg(z)dm(z)

)

+G2

(∫

ψg(z)dm(z)

) (1.19)

where F0, F1 : [0, T ] × R2 → R, G1 : R2 → R, φf , ψf , φg, ψg, F2, G2 : R → R are

all smooth with bounded second derivatives. Then the assumptions A,B,C,D are

satisfied if

∂xxF0, ∂xxF1, ∂xxG0 ≥ 0, ∂ααF0 ≥ c > 0, ∂xxF0∂ααF0 − (∂xαF0)2 ≥ 0

∂xxF1 + ∂xyF1‖φ′f‖∞ ≥ 0, ∂xxG1 + ∂xyG1‖φ

′g‖∞ ≥ 0

The first line is the convexity assumption in x, α strictly in α. The second line is

sufficient to guarantee that the weak monotonicity condition holds. This form of f, g

includes the following examples

f(t, x,m, α) = Aα2 +B

(

x−

∫

zdm(z)

)2

,

g(x,m) = C

(

x−

∫

zdm(z)

)2(1.20)


or

f(t, x,m, α) = Aα2 +B

∫

(x− z)2dm(z),

g(x,m) = C

∫

(x− z)2dm(z),

where A,B,C > 0, which occur frequently in applications (see [21, 33] for instance).

It also includes the cost functions in the general linear-quadratic mean field games

(LQMFG) discussed in [11] where f, g takes the form

f(t, x,m, α) =1

2

(

qx2 + α2 + q(x− sm)2)

g(x,m) =1

2

(

qTx2 + (x− sT m)2qT

)

(1.21)

where

m =

∫

R

zdm(z)

and q, q, s, qT , qT , sT are constant satisfying

q + q − qs ≥ 0, qT + qT − qT sT ≥ 0

We will solve this LQMFG explicitly in Chapter 5.

1.2 Summary of the main results

We now summarize all the main results of this thesis. We begin by discussing two ap-

proaches to MFG, namely the Dynamic Programming Principle (DPP) and Stochastic

Maximum Principle (SMP), which are two different approaches to tackle a stochastic

optimal control problem. We then summarize our results on the wellposed-ness of

a general MFG with common noise and an asymptotic analysis when the common

noise is small. Lastly, we work out explicitly the linear-quadratic mean field games

(LQMFG) model.


1.2.1 Two approaches to mean field games

There are two general approaches to stochastic optimal control problems. We discuss

here briefly how to apply both of them to an MFG problem.

Dynamic Programming Principle

We begin by assuming that there exists an ε-MFG solution αε = (αεt )0≤t≤T ∈ H2([0, T ];R)

which is given in the feedback form

αεt = αε(t, Xε

t , mεt )

where Xεt = X αε

t , mεt = L(Xε

t |Ft), αε : [0, T ]× R× P2(R) is deterministic Lipschitz

function. Using this function αε, we know that mε = (mεt )0≤t≤T is a solution to the

stochastic Fokker-Planck equation

dmε(t, x) =

(

−∂x(αε(t, x,mε)mε) +

σ2 + ε2

2∂xxm

ε

)

dt− ε∂xmε dWs (1.22)

Given the stochastic flow of probability measures mε, we have a standard control

problem parametrized by mε, so we define the value function Vε : [0, T ]×R×P2(R)

by

Vε(t, x,m) = inf(αs)t≤s≤T

E

[∫ T

t

f(s,X t,x,αs , mε

s, αs)ds+ g(X t,x,αT , mε

T )∣

∣

∣X

t,x,αt = x,mε

t = m

]

(1.23)

Suppose that Vε is sufficiently regular, then we can use standard Principal of Opti-

mality and Ito’s formula to show that Vε satisfies

∂tVε(t, x,m) + H(t, x,m, ∂xV

ε(t, x,m)) +σ2 + ε2

2∂xxV

ε(t, x,m)

+ E0[

∂mVε(t, x,m)(X)(α(t, X,m))

]

+σ2

2∂mmV

ε(t, x,m)(X)[ζ, ζ ]

+ε2

2∂mmV

ε(t, x,m)(X)[1, 1] + ε2E0[

∂xmVε(t, x,m)(X)1

]

= 0

(1.24)


with the terminal condition

Vε(T, x,m) = g(x,m)

where

H(t, x,m, y) , infα∈R

(αy + f(t, x,m, α)) (1.25)

is the Hamiltonian, X is a lifting random variable with law m, and ζ is a N (0, 1)-

random variable independent of X .

Remark 1.2.1. The Ito’s lemma used to derive (1.26) is non-standard as it involves

the derivative with respect to m. We refer to Proposition 6.5 in [18] for a full state-

ment and a proof of this Ito’s formula.

It will be useful to note here that under assumption A, there exist a unique

minimizer α : [0, T ]×R×P2(R)×R to (1.25). Thus, by a verification theorem for a

standard control problem, it can be shown that if Vε is a classical solution to (1.26),

then the optimal control is given by

αε(t, x,m) = α(t, x,m, ∂xVε(t, x,m))

But by definition of ε-MFG, this must be the same as αε. That is, we must have

αε(t, x,m) = αε(t, x,m) = α(t, x,m, ∂xVε(t, x,m))

Plugging this back in (1.24) yields


ε(t, x,m)) +σ2 + ε2

2∂xxV

ε(t, x,m)

+ E0[

∂mVε(t, x,m)(X)(α(t, X,m, ∂xV

ε(t, X,m)))]

+σ2

2∂mmV

ε(t, x,m)(X)[ζ, ζ ]

+ε2

2∂mmV

ε(t, x,m)(X)[1, 1] + ε2E0[


]

= 0

(1.26)

The equation (1.26) contains all the information about the ε-MFG, in a sense that


having a classical solution to (1.26) yields an ε-MFG solution in feedback form by

α(t, x,m) = α(t, x,m, ∂xVε(t, x,m))

See Proposition 4.1 in [18] for this verification-type result. For this reason, it is called

the master equation for MFG. However, solving this equation even numerically is

not feasible since it is a second-order non-linear infinite-dimensional HJB equation

involving derivatives with respect to a probability measure. We refer to [10, 18, 24]

for more discussion on the master equation.

Next, we proceed in a similar way as done by Lasry, Lions [13] for 0-MFG by

considering the value function along the trajectory of the equilibrium distribution

mε. We define a random function uε : [0, T ]× R× Ω by

uε(t, x) = Vε(t, x,mεt )

When ε > 0, uε is a stochastic value function which is Wt-path dependent. By Ito-

Kunita lemma, it can be shown that uε(t, x) satisfies the backward stochastic partial

differential equation (BSPDE)

duε(t, x) =

(

−H(t, x,mεt , ∂xu

ε(t, x))−σ2

2∂xxu

ε(t, x)−ε2

2

(

∂xxuε(t, x)− 2∂xv

ε(t, x))

)

dt

− εvε(t, x)dWt

where

vε(t, x) , E0[

∂mVε(t, x,mε

t )(X)1]

with terminal condition

uε(T, x) = g(x,mεT )

More details on the derivation of this BSPDE is provided in Section 2.1. Similarly,

solving this BSPDE yields an optimal feedback control;

αε(t, x) = α(t, x,mεt , ∂xu

ε(t, x))


Plugging this back in (1.22) yields a system of forward backward stochastic partial

differential equation (FBSPDE),

duε(t, x) =

(

−H(t, x,mε, ∂xuε)−

σ2

2∂xxu

ε −ε2

2∂xxu

ε − ε2∂xvε

)

dt− εvεdWt

dmε(t, x) =

(

−∂x(α(t, x,mε, ∂xu

ε)mε) +σ2 + ε2

2∂xxm

ε

)

dt− ε∂xmε dWt

(1.27)

with boundary conditions

mε(0, x) = m0(x) = L(ξ0), uε(T, x) = g(x,mεT )

Similar to the master equation above, there is a verification-type theorem (see Section

4.2 in [10]) which states that a sufficiently regular solution (uε, vε, mε) of (1.27) yields

ε-MFG solution in a feedback form by

αεt = α(t, Xε

t , mεt , ∂xu

ε(t, Xεt ))

Remark 1.2.2. As in the standard BSDE or BSPDE, vε is not specified and is part

of the solution to ensure adaptivity of uε. See Ch.1 in [49] for a brief introduction to

BSDE.

While this FBSPDE seems simpler than the master equation due to an absence

of the terms involving the derivative with respect to m, it is still difficult to solve

and to the best of our knowledge, there is no wellposedness result for FBSPDE on

unbounded domains. See [58] for a recent work on the case of bounded domains.

One can also view FBSPDE (1.27) simply as a different way to represent the

value function. As opposed to Vε which gives the minimum expected cost given

(t, x,m), uε describes the same quantity as a function of (t, x, ω), where ω is a common

Brownian motion path. Both of these equations are infinite-dimensional due to their

last arguments and are difficult to solve. While we have a verification-type theorem

for both equations, there is no existence theory for either of them and to solve these

equations numerically is difficult.

However, when ε = 0, we can reduce (1.27) to a system of fully-coupled finite


dimensional PDE. This can be seen by plugging ε = 0 in (1.27) which results in

∂tu0 = −H(t, x,m0, ∂xu

0)−σ2

2∂xxu

0, u0(T, x) = g(x,m0T )

∂tm0 = −∂x(α(t, x,m

0, ∂xu0)m0) +

σ2

2∂xxm

0, m0(0, x) = m0(x) = L(ξ0)

(1.28)

This is precisely the system of FBPDE that was introduced and studied by Lasry and

Lions [13, 42, 43, 44].

Stochastic Maximum Principle

SMP is an alternative approach to a stochastic control problem which studies opti-

mality conditions satisfied by an optimal control. It gives sufficient and necessary

conditions for the existence of an optimal control in terms of a backward stochastic

differential equation (BSDE) of an adjoint process. We summarize here how the SMP

can be applied to ε-MFG model. For a general control problem, we refer to [53, 61].

To state the SMP, we first introduce an adjoint process corresponding to an indi-

vidual control problem given m ∈ M([0, T ];P2(R)). Given a control α ∈ H2([0, T ];R)

and its corresponding controlled process X (see (1.2)), then the adjoint equation is

defined by

dYt = −∂xH(t, αt, Xt, mt, Yt)dt+ ZtdWt + ZtdWt, YT = ∂xg(XT ) (1.29)

where H : [0, T ]× Rd+3 → R is a generalized Hamiltonian defined as

H(t, a, x, y,m) , αy + f(t, x,m, a) (1.30)

Equation (1.29) is a backward stochastic differential equation (BSDE). We would

like to emphasize that the solution to (1.29) is a pair (Yt, Zt, Zt)0≤t≤T . The processes

(Zt, Zt)0≤t≤T arise naturally from the martingale representation theorem and are nec-

essarily part of a solution to ensure that an adapted solution (Yt)0≤t≤T can be found.

We refer the readers to Ch.1 in [49] for a brief introduction to BSDE.

We can now state a necessary and sufficient condition for an optimal control in


terms of the adjoint process. In short, the necessary condition (see Theorem 2.2.2)

states that if α is an optimal control, then the corresponding adjoint process is solvable

and the following maximum condition (or minimum in this case) holds

H(t, αt, Xt, mt, Yt) = mina∈R

H(t, a, Xt, mt, Yt), 0 ≤ t ≤ T, a.s. (1.31)

where (Yt)0≤t≤T is a solution to (1.29). The sufficient condition (see Theorem 2.2.4)

requires extra convexity assumptions on H, g, which hold under A, and the state-

ment is similar. It says that, given an optimal control α and its corresponding state

process X , if the adjoint BSDE is solvable and its solution (Yt)0≤t≤T satisfies the max-

imum condition (1.31), then α is an optimal control. In fact, under strict convexity

assumption (see (A3)), we have an estimate

J (β) ≥ J (α) + C

∫ T

0

|βt − αt|2dt, ∀β ∈ H2([0, T ];R)

This estimate implies that an optimal control, if exists, must be unique.

Under a strict convexity assumption on f , the generalized Hamiltonian has a

unique minimizer α : [0, T ]× R× P2(R)× R → R. It is then easy to check that

∂xH(t, α(t, x,m, y), x,m, y) = ∂xH(t, x,m, y)

where H is the Hamiltonian defined in (1.25). Consequently, we can plug this min-

imizer function into both the forward controlled process and the backward adjoint

process; as a result, we have a system of forward backward stochastic differential

equation (FBSDE)

dXt = α(t, Xt, mt, Yt)dt+ σdWt + εdWt

dYt = −∂xH(t, Xt, mt, Yt)dt+ ZtdWt + ZtdWt

X0 = ξ0, YT = ∂xg(XT , mT )

(1.32)

We can now state the SMP for an individual control problem givenm ∈ M([0, T ];P2(R))

(recall Definition 1.1.2) in term of FBSDE (1.32) as follows;


Theorem 2.2.7. Assume A holds, then the individual control problem given (mt)0≤t≤T ∈

M([0, T ];P2(R)) has an optimal control if and only if FBSDE (2.17) is solvable. In

that case, the optimal control is given by

αt = α(t, Xt, mt, Yt)

for all t ∈ [0, T ], where (Xt, Yt, Zt, Zt)0≤t≤T is a solution to FBSDE (2.17).

Recall the definition of ε-MFG solution which says that given the stochastic flow

of probability measure mα corresponding to a control α, the optimal control of an

individual control problem given mα is again α. This definition is equivalent to the

following consistency condition

mαt = L(Xα

t |Ft)

Plugging this to (1.32), we have our main result for this section which establishes the

SMP for ε-MFG.

Theorem 2.2.8. Assume that A holds, then ε-MFG is solvable if and only if the

FBSDEdXt = α(t, Xt,L(Xt|Ft), Yt)dt+ σdWt + εdWt

dYt = −∂xH(t, Xt,L(Xt|Ft), Yt)dt+ ZtdWt + ZtdWt

X0 = ξ0, YT = ∂xg(XT ,L(XT |FT ))

(1.33)

is solvable. In that case, ε-MFG solution is given by

αt = α(t, Xt,L(Xt|Ft), Yt), ∀t ∈ [0, T ]

Equation (1.33) is a McKean-Vlasov FBSDE. It was first introduced in [17] from

the 0-MFG problem. When there is no common noise, the conditional law L(Xt|Ft)

is simply the law L(Xt). In [17], Carmona and Delarue show that the McKean-Vlasov

FBSDE corresponding to a 0-MFG is solvable under assumptions similar to A, (B1),

plus what they call a weak mean reverting assumptions (see (1.17)). The proof is

based on Schauder fixed point theorem similar to Lasry and Lions’s existence proof


for the system (1.28). In the next section (Section 1.2.2), we discuss the existence

and uniqueness of the solution to both (1.32) and (1.33). The latter requires extra

weak monotonicity conditions which can be viewed as a stronger version of the weak

mean-reverting condition (1.17).

Connection to the DPP approach

The connection between Vε or (uε, vε, mε) from the DPP approach and a solution

(Xεt , Y

εt , Z

εt , Z

εt )0≤t≤T of McKean-Vlasov FBSDE (1.33) from the SMP approach can

be stated simply through the relations

mεt = L(Xε

t |Ft)

Y εt = ∂xV

ε(t, Xεt , m

εt) = ∂xu

ε(t, Xεt )

Zεt = σ∂xxV

ε(t, Xεt , m

εt )

Zεt = ε

(

∂xxVε(t, Xε

t , mεt ) + E

0[

∂xmVε(t, Xε

t , mεt)(Xt)

])

(1.34)

To prove this relation, extra regularity for Vε(t, x,m) (C1,3,3) is needed and the proof

is based on applying Ito’s formula to the process (∂xVε(t, Xεt ,L(X

εt |Ft)))0≤t≤T . For

the full statement, we refer to Theorem 2.3.1.

1.2.2 Mean field games with common noise

In Section 1.2.1, we have shown that under assumption A, the ε-MFG problem is

equivalent to solving the McKean-Vlasov FBSDE (1.33). In this section, we summa-

rize our results on a wellposed-ness theory of this FBSDE, hence for a general MFG

with common noise. We will discuss the existence and uniqueness of a solution to

ε-MFG, a Markov property, and a connection to the HJB approach. Our main result

for this section is the following

Theorem 3.1.6 (Wellposedness of McKean-Vlasov FBSDE). Assume that A and B


hold, then there exist a unique solution (Xt, Yt, Zt, Zt)0≤t≤T to FBSDE (1.33) satisfy-

ing

E

[

sups≤t≤T

[X2t + Y 2

t ] +

∫ T

s

[Z2t + Z2

t ]dt

]

≤ C

(

E[ξ2] + (∂xg(0, δ0))2 +

∫ T

s

(α(t, 0, δ0, 0))2 + (∂xH(t, 0, δ0, 0))

2dt+ σ2 + ε2)

(1.35)

where δx denote the Dirac measure at x. Moreover, two solutions (X it , Y

it , Z

it , Z

it)s≤t≤T , i =

1, 2 to FBSDE (2.2.8) with initial ξi satisfies the estimate

E

[

sups≤t≤T

1A∆X2t + sup

s≤t≤T1A∆Y

2t +

∫ T

s

[1A∆Z2t + 1A∆Z

2t ]dt

]

≤ CK,TE[1A∆ξ2]

(1.36)

where ∆Xt = X1t − X2

t , ∆Yt,∆Zt,∆Zt,∆ξ are defined similarly, CK,T is a constant

depends only on K, T , A is an Fs-measurable set, and 1A denotes the indicator func-

tion.

In the existence proof of both Lasry and Lions PDE approach and Carmona

and Delarue probabilistic approach, they applied Schauder fixed point theorem to

establish the existence of a fixed point. However, this strategy cannot be extended

to the common noise case as the flow of probability measures given by the law of

the optimal state process is no longer deterministic. Instead, we have to deal with

a random flow of probability measures from a conditional law. Working with this

larger space, we cannot establish compactness which is necessary to apply Schauder

fixed point theorem in the same way.

As a result, we will use an alternative approach, namely the Banach fixed point

theorem. In the same way as in the proof of wellposedness of FBSDE, the Banach

fixed point theorem can be used to establish the existence of a solution when the

time duration T is sufficiently small. This method can usually be applied in that

case because the solution estimate depends on T , thus we can get a contraction map

when T is sufficiently small. See [5, 49] for a proof of existence and uniqueness of

a solution to FBSDE for a small time duration. However, the small time restriction

is not a desirable assumption for obvious reasons, so we wish to extend the solution


to an arbitrary time duration. To do so, we need an extra condition to be able to

control the Lipschitz constant of a new terminal condition as we move backwards in

time. This is precisely the weak monotonicity condition (B3).

In our subsequent analysis, we will see a similar type of FBSDE which requires a

similar technique involving a monotonicity property, so we state and prove the result

in a slightly more general framework. That is, we will consider the following FBSDE

over [s, T ]

dXt = b(t, Xt, Yt)dt+ σdWt + εdWt

dYt = F (t, Xt, Yt)dt+ ZtdWt + ZtdWt

Xs = ξ, YT = G(XT )

(1.37)

where ξ ∈ L2Fs

and b, F,G are measurable maps

b : [0, T ]×L2F × L2

F × Ω → R

F : [0, T ]× L2F ×L2

F × Ω → R

G : L2F × Ω → R

(1.38)

Notice that the map b, F,G are “functional” in a sense that their inputs are random

variables. In addition to standard measurability, Lipschitz, and linear growth condi-

tion, we need the following monotonicity condition which reads; For any t ∈ [0, T ],

X,X ′, Y, Y ′ ∈ L2FT

, and A ∈ Gt,

E [1A(F (t, X, Y )− F (t, X ′, Y ′))(X −X ′) + 1A(b(t, X, Y )− b(t, X ′, Y ′))(Y − Y ′)] ≤

−β1E[

1A(b(t, X, Y )− b(t, X ′, Y ′))2]

− β2E[

1A(Y − Y ′)2]

E [1A(G(X)−G(X ′))(X −X ′)] ≥ 0

(1.39)

for some constant β1, β2 ≥ 0 with β1 + β2 > 0. For the full statement of all the

assumptions, we refer to assumption H in Section 3.1.1. Then we have the following

result

Theorem 3.1.2. Let ξ ∈ L2Fs

and b, F,G be functionals satisfying assumption H, then

there exist a unique adapted solution (Xt, Yt, Zt, Zt)s≤t≤T to FBSDE (1.37) satisfying


the estimate

E

[

1A sups≤t≤T

X2t + 1A sup

s≤t≤TY 2t + 1A

∫ T

s

[Z2t + Z2

t ]dt

]

≤ CK,T

(

E

[

1Aξ2 + 1AG(0)

2 + 1A

∫ T

s

(

b(t, 0, 0)2 + F (t, 0, 0)2)

dt

]

+ σ2 + ε2)

(1.40)

for some constant CK,T depends only on K, T and for all A ∈ Gt.

Using assumption A,B, we can show that (see Theorem 3.1.6) b, F,G defined by

b(t, X, Y ) = α(t, X,L(X|Ft), Y ),

F (t, X, Y ) = −∂xH(t, X,L(X|Ft), Y ),

G(X) = ∂xg(X,L(X|FT ))

Gt0≤t≤T = Ft0≤t≤T

(1.41)

satisfies all the assumption in Theorem 3.1.2. As a result, we can apply Theorem

3.1.2 to the McKean-Vlasov FBSDE (1.33) which, combining with Theorem 2.2.8,

yields the existence and uniqueness result for an ε-MFG solution.

Markov property and a decoupling function

In this section, we discuss a Markov property of McKean-Vlasov FBSDE (1.33). A

classical FBSDE is said to have a Markov property if there exist a deterministic

function θ : [0, T ]× R → R such that

Yt = θ(t, Xt)

This function is often called a decoupling function of an FBSDE. By plugging it into

the FBSDE, it can be seen that θ is a solution to a quasilinear PDE. Solving this PDE,

one can decouple the system and reduce the problem to solving just the forward SDE.

This method of solving FBSDE is called Four-step scheme [46]. Markov property of

a classical FBSDE holds when the coefficients are deterministic.


Even though all the coefficients of McKean-Vlasov FBSDE (1.33) are determinis-

tic, it is not obvious that this Markovian property holds particularly in the case of

common noise. For a fixed m ∈ M([0, T ];P2(R)), we are in fact dealing with FBSDE

with non-deterministic coefficients. Specifically, we have a path dependent functions

∂xH(t, x,mt(ω), y), g(x,mT (ω))

However, as f, g are still deterministic functions of m, it is reasonable to expect a

Markov property if we include the current distribution of players as an additional

input, or in FBSDE context, the conditional distribution of the state process. Our

main result for this section is the following theorem

Theorem 3.2.1. Let (Xεt , Y

εt , Z

εt , Z

εt )0≤t≤T denote the solution to McKean-Vlasov

FBSDE (1.33), then there exist a deterministic function Uε : [0, T ]×R×P2(R) such

that

Y εt = Uε(t, Xε

t ,L(Xεt |Ft)) (1.42)

Moreover, Uε satisfies the estimates

1. |Uε(t, x,m)− Uε(t, x′, m′)| ≤ CK,T (|x− x′|+W2(m,m′))

2. (Uε(t, x,m)− Uε(t, x′, m)) (x− x′) ≥ 0

for all t ∈ [0, T ], x, x′ ∈ R, m,m′ ∈ P2(R) where CK,T depends only on K, T .

We explain briefly here how Uε(s, x,m) is defined; Fix (s, x,m) ∈ [0, T ] × R ×

P2(R). First, we solve McKean-Vlasov FBSDE over [s, T ] using initial Xs = ξ with

L(ξ) = m to get a solution (Xs,mt , Y

s,mt , Z

s,mt , Z

s,mt )s≤t≤T , then we define a stochastic

flow of probability measure (ms,mt )s≤t≤T ∈ M([s, T ];P2(R)) by ms,m

t = L(Xs,mt |F s

t )

where F st = σ(Wr − Ws; s ≤ r ≤ t). Intuitively, ms,m

t represents the distribution of

players at time t under ε-MFG solution strategy given that the distribution at time

s is m. Then we consider an individual control problem over [s, T ] given (ms,mt )s≤t≤T

with the initial position Xs = x by solving the corresponding FBSDE (1.32) to get a

solution (Xs,x,mt , Y

s,x,mt , Z

s,x,mt , Z

s,x,mt )s≤t≤T . We then define Uε(s, x,m) to be Y s,x,m

s

which is F ss -measurable, hence deterministic. See Section 3.2 for more details.


Having defined Uε, we are left to show that it is indeed a decoupling function, i.e.

(1.42) holds. The proof is based on a discretizing argument and a priori estimate of

Uε. See Theorem 3.2.12 for the full statement and its proof.

Connection to DPP approach

In the previous section, we discussed the relation between Vε, uε and the solution

(Xεt , Y

εt , Z

εt , Z

εt )0≤t≤T of FBSDE (1.33). Having introduced the deterministic decou-

pling function Uε, we can restate the connection between the two approaches through

this function. Comparing (1.34) and (1.42), we have

Uε(t, x,m) = ∂xVε(t, x,m) (1.43)

Consequently, we can deduce other relations which are summarized below

mεt = L(Xε

t |Ft)

Y εt = Uε(t, Xε

t , mεt ) = ∂xV

ε(t, Xεt , m

εt ) = ∂xu

ε(t, Xεt )

Uε(t, x,mεt ) = ∂xV

ε(t, x,mεt ) = ∂xu

ε(t, x)

Uε(t, x,m) = ∂xVε(t, x,m)

E0[

∂mUε(t, x,mε

t )(X)1]

= E0[

∂xmVε(t, x,mε

t )(X)1]

= ∂xvε(t, x)

(1.44)

Using the relation (1.43) and the master equation (1.26) for Vε, the equation for Uε

can be derived easily and is given by

∂tUε(t, x,m) + ∂xH(t, x,m,Uε(t, x,m)) + ∂yH(t, x,m,Uε(t, x,m))∂xU

ε(t, x,m)

+σ2 + ε2

2∂xxU

ε(t, x,m)− E0[

∂mUε(t, x,m)(X)α(t, X,m,Uε(t, X,m))

]

+σ2

2∂mmU

ε(t, x,m)(X)[ζ, ζ ] +ε2

2∂mmU

ε(t, x,m)(X)[1, 1]

+ ε2E0[

∂xmUε(t, x,m)(X)1

]

= 0

(1.45)

Similar to Vε and its master equation, we have a verification theorem which says

that a classical solution to (1.45) is a decoupling function to McKean-Vlasov FBSDE


(1.33). See Theorem 3.3.2 for the full statement.

1.2.3 Asymptotic analysis of mean field games

Recently, there has been progress in studying the MFG with common noise concerned

mostly with the general well-posedness theory. See [3, 20, 10, 18]. Despite this

progress, a general common noise model is still difficult and impractical to solve

numerically or analytically as it does not enjoy the dimension reduction property

as in the case of MFG without common noise. When the common noise is small,

however, it is reasonable to seek an approximate solution using only finite-dimensional

information from the ε = 0 problem.

Let (αεt , X

εt )0≤t≤T be an ε-MFG solution, and its corresponding state process, we

are interested in the following ε-expansion

αεt = α0

t + εδαt + o(ε), Xεt = X0

t + εδXt + o(ε)

Through the SMP, this is equivalent to studying the limit as ε→ 0 of

Xεt −X0

t

ε,

Y εt − Y 0

t

ε(1.46)

where (Xεt , Y

εt , Z

εt , Z

εt )0≤t≤T denote the solution to McKean-Vlasov FBSDE (1.33).

We now state additional regularity assumption; From now and throughout the rest

of the section, we assume, in addition to A,B, that C holds.

(C1) ∂xxf, ∂xmf, ∂xxg, ∂xmg exist and are continuous and bounded. Denote their

bounds by the same constant K.

To reduce notations, particularly those from the Hamiltonian H and its deriva-

tives, we will assume in this section that

f(t, x,m, α) =α2

2


while keeping a general terminal cost function g. The same result still holds for a

general running cost f satisfying A,B,C. Note that with this running cost, it follows

that

α(t, x,m, y) = −y, H(t, x,m, y) = −y2

2

so that the McKean-Vlasov FBSDE (1.33) reads

dXt = −Ytdt+ σdWt + εdWt

dYt = ZtdWt + ZtdWt

X0 = ξ0, YT = ∂xg(XT ,L(XT |FT ))

(1.47)

By formally taking the limit (1.46) through (1.47), we get the following linear varia-

tional mean-field FBSDE

dUt = −Vtdt+ dWt

dVt = QtdWt + QtdWt

U0 = 0, VT = ∂xxg(X0T , m

0T )UT + E[∂xmg(X

0T , m

0T )(X

0T )UT ]

(1.48)

where

m0t = L(X0

t |Ft) = L(X0t )

and X0 and U are identical copies ofX0 and U in (Ω, F , P) and E0[·] is the expectation

with respect to ω0 only. We can write the terminal function explicitly as

E0[∂xmg(X

0T , m

0T )(X

0T )UT ] =

∫

Ω0

∂xmg(X0T (ω

0), m0T )(X

0T (ω

0))UT (ω0, ω)dP0(ω0)

where we suppress the ω in X0T , X

0T as they do not depend on it. We can see that the

term E0[∂xmg(X0T , m

0T )(X

0T )UT ] is a mean-field term that couples UT (ω, ·) for different

ω ∈ Ω0 together. Intuitively, each different path ω ∈ Ω0 represents different players

and the mean-field term represents the effect on a single player from all other players.

First, by applying Theorem 3.1.2, we can show that this FBSDE is uniquely solvable

(see Theorem 4.1.1). We are now ready to state our first main theorem which gives

a convergence result of the limit formally taken above.


Theorem 4.1.2. Assume A,B,C hold, for all ε > 0, let (Xεt , Y

εt , Z

εt , Z

εt )0≤t≤T de-

note the solution to McKean-Vlasov FBSDE (1.33) corresponding to ε-MFG and

(Ut, Vt, Qt, Qt)0≤t≤T denote the solution to (1.48), then there exist a constant CK,T

dependent only on K, T such that

E sup0≤t≤T

[

(

Xεt −X0

t

ε− Ut

)2

+

(

Y εt − Y 0

t

ε− Vt

)2]

≤ CK,Tε2 (1.49)

Approximate Nash equilibrium

Using the result above, we can construct a first order approximation for ε-MFG by

taking

βεt , α0

t − εVt (1.50)

for all t ∈ [0, T ]. Being a game, an appropriate notion must be used to check that

β is a good approximate strategy. This notion is what is called δ-approximate Nash

equilibrium. An admissible strategy α = (αt)0≤t≤T ∈ H2([0, T ];R) is called a δ-Nash

equilibrium for ε-MFG if

J ε(α|mα) ≤ J ε(β|mα) + δ

for all β = (βt)0≤t≤T where J ε(·) denote the cost function and (mαt )0≤t≤T denote

the law conditional on the common noise of a player adopting strategy α. From this

definition, a 0-approximate Nash equilibrium is an ε-MFG solution.

This notion of approximate Nash equilibrium has applications in many areas in-

cluding infinite horizon stochastic games or algorithmic game theory. In those cases,

an exact Nash equilibrium either does not exist or is computationally expensive. In

MFG, recall that the motivation for considering this model is its application for find-

ing a good approximate strategy for an N -player game when N is large. This notion

is used mainly in the study of this approximation. See [17, 22, 37, 39]. In these

works, however, we are only concerned with the model at the continuum limit. That

is, we are more interested in the approximate solution for ε-MFG using information

available from the 0-MFG solution. Our main result for this section is the following;


Theorem 4.2.5. Assume A,B,C hold. For ε > 0, let αε = (αεt )0≤t≤T denote the

solution to ε-MFG and (Ut, Vt, Qt, Qt)0≤t≤T denote the solution to the linear variation

FBSDE (4.5). Define an approximate strategy βε = (βεt )0≤t≤T by

βεt , α0

t − εVt (1.51)

Then βε is an ε2-Nash equilibrium for ε-MFG.

Gaussian properties of (Ut, Vt) and decoupling function

In this section, we would like to discuss some properties of (Ut, Vt)0≤t≤. The first

property is that (Ut, Vt)0≤t≤ is a Gaussian process. This follows from the fact that

the FBSDE (1.48) is linear with respect to common noise. Furthermore, using the

weak monotonicity condition and the fact that the initial U0 = 0, it can be shown

that their means are zero (see Theorem 4.3.1).

To compute the covariance function, however, the standard approach for a linear

SDE involving taking Ito’s lemma is no longer applicable due to the fully-coupled for-

ward backward structure, particularly due to the presence of the terms (Qt, Qt)0≤t≤T .

As a result, we need to resort to the decoupling function of (1.48). Recall that we

have the decoupling function Uε for McKean-Vlasov FBSDE (1.33) which satisfies

the relation

Y εt = Uε(t, Xε

t ,L(Xεt |Ft))

The following theorem is crucial in obtaining the decoupling function for FBSDE

(1.48)

Theorem 4.3.2. Let Uε denote the decoupling function of FBSDE (1.33) as defined

in (1.42), then the following holds;

limε→0

Uε(t, x,m)− U0(t, x,m)

ε= 0 (1.52)

uniformly in (t, x,m) ∈ [0, T ]× R× P2(R).


Remark 1.2.3. The theorem above implies that to approximate the ε-MFG solu-

tion at the first order, we simply need to use 0-MFG solution along the trajectory

(t, Xεt ,L(X

εt |Ft)). However, we would like to emphasize that we do not usually know

U0(t, x,m) for all (t, x,m) since that would require us to solve the master equation

(2.8) or (3.48) which is an infinite-dimensional problem and clearly non-trivial to do

so.

While there is no result on the regularity of Uε and all the discussions so far has

been formal [10, 18, 13], there has been a recent progress for the original no common

noise model. In a recent paper by Chassagneux et al.[24], they showed the existence

and uniqueness of a classical solution to the master equation (3.48) when ε = 0. As

a by-product, we have that U0 is continuously differentiable in (x,m). To apply this

result, we need assumption D which provides additional regularity assumption on the

cost functions involving second derivative with respect to m-argument. As a result,

we have the decoupling function for linear variational FBSDE (4.5).

Proposition 4.3.3. Assume that A,B,C,D holds. Let (Ut, Vt, Qt, Qt)0≤t≤T denote

the unique solution to FBSDE (4.5), then

Vt = ∂xU0(t, X0

t , m0t )Ut + E

0[∂mU0(t, X0

t , m0t )(X

0t )Ut] (1.53)

Explicit solution and covariance function

From (1.48) and (1.53) above, we see that the functions U0(t, x,m0t ), ∂xU

0(t, x,m0t )

and E0[∂mU0(t, X0t , m

0t )(X

0t )Ut] are essential terms in our asymptotic analysis. The

first two terms are simply ∂xu0(t, x), ∂xxu

0(t, x) where (u0, m0) is the solution to Lasry

and Lions’ FBPDE (2.10). We let

η(t, x, ω) , E0[∂mU

0(t, x,m0t )(X

0t )Ut]

To proceed and further analyze the linear mean-field FBSDE (4.5), particularly

the random function η(t, x), an extra regularity assumption on the decoupling func-

tion of 0-MFG U0 is required to derive the equations for η(t, x) and compute the


covariance functions. While there are some results that give sufficient conditions for

the differentiability with respect to (t, x) when m = m0t , since it is directly related

to u0(t, x) from Lasry and Lions’s FBSDE, the only work that provides similar result

for the derivative with respect to m-argument for U0 is [24]. However, their result

is still not sufficient for our application below. So going forwards, we will proceed

formally assuming that U0 is sufficiently regular with bounded derivatives. Our goal

is to derive the equation for η(t, x) and to compute the covariance function of the

process (Ut, Vt)0≤t≤T .

By using Ito’s lemma, we can show that η(t, x) satisfies the SPDE

dη(t, x) =

[

η(t, x)∂xU0(t, x,m0

t ) + ∂xη(t, x)U0(t, x,m0

t )−σ2

2∂xxη(t, x)

]

dt− w(t, x)dWt

(1.54)

with initial condition η(0, x) = 0 and w : [0, T ]× R → R is given by

w(t, x) , E0[

∂mU0(t, x,m0

t )(X0t )]

To fully describe the system, we are left to find w(t, x), which requires us to compute

∂mU0(t, x,m0t )(·). Again, by using Ito’s lemma and the master equation for U0 (3.48),

it can be shown that the resulting derivative satisfies an infinite-dimensional Riccati-

type equation. More precisely, w(t, x) is given by

w(t, x) = E0[

∂mU0(t, x,m0

t )(X)]

=

∫

R

h(t, x, z)m0(t, z)dz

for all (t, x) ∈ [0, T ]× R, where h(t, x, z) satisfies

∂th(t, x, z) = h(t, x, z)∂xxu0(t, x) + ∂xh(t, x, z)∂xu

0 −σ2

2∂xxh(t, x, z)

+

∫

R

h(t, x, u)h(t, u, z)m0(t, u)du+ ∂z(h(t, x, z)∂zu0(t, z))−

σ2

2∂zzh(t, x, z)

(1.55)

where the equation is to be interpreted weakly in the z-argument. We provide more

details of all these calculation in Section 4.3.2.

We have described the decoupling function for linear variational FBSDE (1.48)


explicitly in term of η(t, x) and w(t, x) defined above. In addition, we know that

η(t, x) satisfies a linear SPDE and w(t, x) can be expressed in term of h(t, x, z), a

solution to a deterministic partial integro-differential equation (1.55). Being a linear

SPDE, we have an explicit solution of (1.54) by mean of Duhamel’s principle. That

is,

η(t, x) =

∫ t

0

ψs(t, x)dWs

where ψs is a solution to the PDE

∂tφ(t, x) = ∂xφ(t, x)U0(t, x,m0

t ) + φ(t, x)∂xU0(t, x,m0

t )−σ2

2∂xxφ(t, x), (t, x) ∈ [s, T ]× R

φ(s, x) = w(s, x), x ∈ R

(1.56)

Using this and the relation (1.53) above, we can derive an explicit solution to FBSDE

(1.48) as follows; First, we find w by solving the PDE (1.55), then solve of η by

solving for ψ satisfying (1.56). Next, we plug in η in the forward SDE of Ut and we

can easily solve for Ut by using integrating factor. We then use the relation (1.53) to

get Vt explicitly.

Similarly, by using Ito’s lemma, we can fully describe the covariance function for

Ut, Vt in terms of ∂xxu0, a solution from 0-MFG system (2.10), and ψs, a solution to

PDE (4.22). We omit the detail here and refer the readers to Section 4.4.2 for all the

equations.

1.2.4 Linear quadratic mean field games with common noise

We consider a linear quadratic model with controlled process

dXt = αtdt + σdWt + εdWt, X0 = ξ0


and cost functions

f(t, a, x,m) =1

2

(

qx2 + α2 + q(x− sm)2)

g(x,m) =1

2

(


)

,

Under this LQ model, we can solve for the generalized value function Vε by assuming

that it takes a quadratic form. As a result, we get that

Vε(t, x,m) =1

2p(t)x2 + q(t)xm+

1

2r(t)m2 + s(t)

where p(t), q(t), r(t), s(t) : [0, T ] → R satisfies

p′(t) = p2(t)− q − q, p(T ) = qT + qT

q′(t) = 2p(t)q(t) + q2(t) + qs, q(T ) = −sT qT

r′(t) = 2(p(t) + q(t))r(t) + q(t)2 − qs2, r(T ) = s2T qT

s′(t) = −1

2(σ2 + ε2)p(t)−

1

2ε2r(t)− ε2q(t), s(T ) = 0

(1.57)

Consequently, all other functions including the solutions to FBSPDE, McKean-

Vlasov FBSDE, and linear variational FBSDE can be found explicitly as they are

directly related to Vε. The resulting functions are summarized below, while more

detail on the calculation are provided in Chapter 5. Let

a(t) =σ2

2

∫ T

t

p(s)ds, b(t) =

∫ T

t

(

1

2(p(s) + r(s)) + q(s)

)

ds (1.58)


then from the DPP approach, we have

Vε(t, x,m) =1

2p(t)x2 + q(t)xm+

1

2r(t)m2 + a(t) + ε2b(t)

uε(t, x) =1

2p(t)x2 + q(t)x

(

β0,tmε0 + ε

∫ t

0

βs,tdWs

)

+1

2r(t)

(

β0,tmε0 + ε

∫ t

0

βs,tdWs

)2

+ a(t) + ε2b(t)

vε(t, x) = q(t)x+ r(t)

(

β0,tmε0 + ε

∫ t

0

βs,tdWs

)

From the SMP approach, we have

Uε(t, x,m) = p(t)x+ q(t)m

Y εt = p(t)Xε

t + q(t)E[Xεt |Ft]

Zεt = σp(t)

Zεt = −εq(t)(p(t) + q(t))E[Xε

t |Ft]

where (Xεt )0≤t≤T solves

dXεt = (−p(t)Xε

t − q(t)E[Xεt |Ft])dt+ σdWt + εWt, Xε

0 = ξ0

For the asymptotic analysis, we have

Vt = (p(t) + q(t))Ut

where (Ut)0≤t≤T solves

dUt = −(p(t) + q(t))Utdt+ dWt, U0 = 0

Consequently, (Ut, Vt)0≤t≤T are Gaussian processes with mean zero and the following


covariance functions

dE[U2t ]

dt= −2(p(t) + q(t))E[U2

t ] + 1, E[U20 ] = 0

E[UtVt] = (p(t) + q(t))E[U2t ]

E[V 2t ] = (p(t) + q(t))2E[U2

t ]

Lastly, η, w, h are given by

η(t, x) = q(t)Ut = q(t)

∫ t

0

e−∫ t

s(p(r)+q(r))drdWs

w(t, x) = h(t, x, z) = ∂mU0(t, x,m0

t )(z) = q(t)

Note that the functions from the asymptotic analysis of LQMFG are trivial. This

result is mainly due to the fact that the second derivative of f, g in LQ model are

constant which results in the second derivative of the value function and related

functions being constant.

1.3 Summary of contributions

Motivated by applications in finance and economics, we are interested in an MFG

model with common noise which extends the original MFG model proposed by Lasry

and Lions. We operate under a linear-convexity framework and what we call the weak

monotonicity assumption (see (B3)) on the cost functions. Our contributions can be

summarized as follows;

• Wellposed theory. We show the existence and uniqueness of a strong solu-

tion to an MFG model with common noise using the SMP approach. That is,

we prove that the McKean-Vlasov FBSDE corresponding to ε-MFG is uniquely

solvable (Theorem 3.1.6) and its solution yields an ε-MFG solution (Theorem

2.2.8). Our main assumption, in addition to standard assumptions, is the weak

monotonicity condition which is stronger than the weak-mean reverting assump-

tion used to show the existence of a 0-MFG solution and is weaker than Lasry


and Lions’ monotonicity assumption used to show the uniqueness (Proposition

1.1.8). As a by-product, we give a more general uniqueness result for 0-MFG.

• Markov property. We show that the ε-MFG solution is in the feedback form.

This is done by proving the existence of a deterministic decoupling function of

the corresponding McKean-Vlasov FBSDE. Our main result is Theorem 3.2.1.

• Asymptotic analysis. We obtain the first order expansion of the ε-MFG

solution which is characterized as the solution to a linear FBSDE of mean-field

type (Theorem 4.1.2). We then show that the first order approximate solution is

ε2-approximate Nash equilibrium (Theorem 4.2.5). Furthermore, we show that

the solution is a Gaussian process with mean zero (Theorem 4.3.1) and find

the decoupling function for this FBSDE (Proposition 4.3.3) in terms of U0, the

decoupling function for 0-MFG. We then formally derive, assuming regularity

on U0, the explicit solution and compute the covariance functions (Section 4.4).

All the results in this chapter are specialized to the case of quadratic running

cost for simplicity, although the same method can be applied to general running

cost functions satisfying convexity and weak monotonicity assumptions.

Chapter 2

Two approaches to mean field

games

There are two main approaches to a stochastic optimal control problem, namely

Dynamic Programming Principle (DPP) and Stochastic Maximum Principle (SMP).

The DPP is a PDE approach which involves the study of an HJB equation for a

value function. The SMP is a probabilistic approach which involves the backward

stochastic differential equation (BSDE) of an adjoint process.

In this section, we describe how to apply these two approaches to an MFG model

in the presence of common noise. We begin with the DPP approach in Section 2.1

and the SMP approach in Section 2.2.4. We then discuss the relation between the

two approaches in Section 2.3.

2.1 Dynamic Programming Principle (DPP)

The DPP approach to a continuous-time stochastic control problem is the study of

its value function which describes the optimal cost that can be obtained as a function

of the current time and state. By exploiting the Markovian structure of the problem,

we can derive formally the PDE satisfied by the value function. For a stochastic

control problem where the controlled process is a diffusion process, this method gives

a second-order non-linear PDE called Hamilton-Jacobi-Bellman (HJB) equation. We

44

CHAPTER 2. TWO APPROACHES TO MEAN FIELD GAMES 45

can show that a classical solution of this equation gives an optimal cost and, as a by-

product, the optimal policy for each state. This type of result is called a verification

theorem. We refer to Ch.3 of [28] for more details on the DPP approach to a stochastic

control problem.

Applying this approach to an MFG problem is non-trivial since the cost depends

on the distribution (mt)0≤t≤T which is governed by other players’ decisions. How-

ever, if each player has the same control problem, it is reasonable to expect that, at

equilibrium, the optimal control of each player is identical and is given in a feedback

form, i.e. as a function of his/her own state Xt and the distribution of the players mt.

Let us suppose that there exist a unique solution αε ∈ H2([0, T ];R) to the ε-MFG

problem which is of the form

αεt = αε(t, Xε

t ,L(Xεt |Ft))

whereXεt := X αε

t and αε : [0, T ]×R×P2(R) → R is a deterministic Lipschitz function.

The feedback form and Lipschitz property of an optimal control does indeed hold

under assumption A, a fact that will be proven in Section 2.2.4 through the SMP

approach.

Next, let mεt := mαε

t denote the corresponding conditional law, that is,

mεt = L(Xε

t |Ft)

then we have that mε = (mεt )0≤t≤T is a weak solution of the following stochastic

Fokker-Planck equation

dmε(t, x) =

(

−∂x(αε(t, x,mε)mε) +

σ2 + ε2

2∂xxm

ε

)

dt− ε∂xmε dWs (2.1)


Now we can define the generalized value function Vε : [0, T ]× R× P2(R) by

Vε(t, x,m) = inf(αs)t≤s≤T∈H2([t,T ];R)

E

[∫ T

t

f(s,X t,x,αs , mε

s, αs)ds+ g(X t,x,αT , mε

T )∣

∣

∣X

t,x,αt = x,mε

t = m

]

= E

[∫ T

t

f(s,Xεs , m

εs, α

ε(s,Xεs , m

εs))ds+ g(Xε

T , mεT )∣

∣

∣Xε

t = x,mεt = m

]

(2.2)

where

dX t,x,αs = αsds+ σdWs + εdWs, X

t,x,αt = x, s ∈ [t, T ]

Given that the αε is a solution to ε-MFG, the value function above represents the

minimum expected cost from t to T given the state of the game at time t. By the

Principal of Optimality, we expect to have

Vε(t, x,m) = infα∈H2([t,t+h];R)

E

[∫ t+h

t

f(s,X t,x,αs , mt,m

s , αs)ds+ Vε(t + h,Xt,x,αt+h , m

t,mt+h)

]

(2.3)

where

dmt,ms =

(

−∂x(αε(s, x,mt,m

s )mt,ms ) +

σ2 + ε2

2∂xxm

t,ms

)

dt−ε∂xmt,ms dWs, m

t,mt = m

or equivalently,

mt,ms = L(X t,m

s |Fs)

dX t,ms = αε(s, X t,m

s , mt,ms )ds+ σdWs + εdWs, X

t,mt = ξ

Here ξ is independent of Ft with law m, (Ws)t≤s≤T is a Brownian motion independent

of all other Brownian motion, (X t,ms )t≤s≤T is the dynamics of a typical or representa-

tive player under the ε-MFG solution, and (mt,ms )t≤s≤T represents the corresponding

distribution of players with distribution m at time t.

Suppose that Vε is sufficiently regular, then we can use Ito’s formula in (2.3) to


show that Vε satisfies


ε(t, x,m)) +σ2 + ε2

2∂xxV

ε(t, x,m)

+ E0[

∂mVε(t, x,m)(X)(α(t, X,m))

]

+σ2

2∂mmV

ε(t, x,m)(X)[ζ, ζ ]

+ε2

2∂mmV

ε(t, x,m)(X)[1, 1] + ε2E0[


]

= 0

(2.4)

with the terminal condition


where

H(t, x,m, y) , infα∈R

(αy + f(t, x,m, α)) (2.5)

is the Hamiltonian, X is a lifting random variable, i.e. L(X) = m, and ζ is a N (0, 1)-

random variable independent of X .

Remark 2.1.1. The derivative with respect to the m-argument is based on the prob-

abilistic framework proposed by Lasry and Lions in [13]. See Section 1.1.3 for more

detail. Also, an extended version of Ito’s lemma is necessary to incorporate this notion

of derivative. This result is shown in [18], see specifically Proposition 6.5.

When f is strictly convex in α (see (1.9)), then there is a unique minimizer to

(2.5) which we will denote by

α(t, x,m, y) = argminα∈R

(αy + f(t, x,m, α)) (2.6)

From the equation (2.4) above, the optimal control in feedback form is given by

αε(t, x,m) = α(t, x,m, ∂xVε(t, x,m)) (2.7)

By the definition of ε-MFG, we are searching for an optimal control such that given

a control (αεt )0≤t≤T and its corresponding stochastic flow of distribution (mαε

t )0≤t≤T ,

the optimal control for an individual problem given (mαε

t )0≤t≤T is again (αεt )0≤t≤T .


Equivalently, we would like to find αε such that the following holds

αε(t, x,m) = αε(t, x,m) = α(t, x,m, ∂xVε(t, x,m))

Plugging in this relation back in (2.4), it follows that Vε(t, x,m) must satisfy


ε(t, x,m)) +σ2 + ε2

2∂xxV

ε(t, x,m)

+ E0[


ε(t, X,m)))]

+σ2

2∂mmV

ε(t, x,m)(X)[ζ, ζ ]

+ε2

2∂mmV

ε(t, x,m)(X)[1, 1] + ε2E0[


]

= 0

(2.8)



Equation (2.8) is often referred to as the master equation. It is an infinite-

dimensional non-linear second order HJB equation involving derivative with respect

to a probability measure. It was first introduced by Lasry and Lions in a heuristic

fashion and was discussed more extensively in [10, 18, 24]. In [10], Bensoussan de-

scribed the master equation through the HJB approach, while in [18], Carmona et

al. viewed the master equation through the decoupling function of McKean-Vlasov

FBSDE associated with the ε-MFG. This is related to Section 2.3 where we discuss

the relation between DPP and SMP. Recently, Delrarue et al. in [24] showed the ex-

istence of a classical solution to the master equation. The result was obtained under

a no common noise model and with monotonicity-type assumptions.

We have derived this equation by assuming the existence of an ε-MFG solution,

its Markovian property, and the regularity of the value function. We now state the

verification theorem which says that if there exist a classical solution to this master

equation satisfying an integrability condition, then the ε-MFG solution is indeed given

in the feedback form as

αεt = α

(

t, Xεt ,L(X

εt |Ft), ∂xV

ε(t, Xεt , m

εt ))


Let C1,2,2([0, T )× R× P2(R)) denote the space of real-valued function h on [0, T )×

R× P2(R) whose partial derivatives ∂th, ∂xxh, ∂mmh, ∂xmh exist and are continuous,

C([0, T ]×R×P2(R)) denote the space of continuous real-valued function on [0, T ]×

R×P2(R), and L2m(R) denote the space of measurable function h such that

∫

R

h2(x)dm(x) <∞

Theorem 2.1.2. Suppose Vε is a function in C1,2,2([0, T )×R×P2(R))∩C([0, T ]×

R×P2(R)) satisfying the master equation (2.8) with terminal condition Vε(T, ·) = g.

Suppose further that Vε satisfies

|∂xVε(t, x,m)|+ ‖∂mV

ε(t, x,m)(·)‖L2m(R) ≤ C

(

1 + |x|+

∫

R

y2dm(y)

)

for any (t, x,m) ∈ [0, T ]× R× P2(R) and there exist a unique solution to the SDE

dXt = α(t, Xt,L(Xt|Ft), ∂xVε(t, Xt,L(Xt|Ft)))dt+ σdWt + εdWt, X0 = ξ0

then

(αεt )0≤t≤T = (α(t, Xε

t , mεt , ∂xV

ε(t, Xεt , m

εt )))0≤t≤T

is a solution to ε-MFG with initial ξ0.

For the proof, we refer to Proposition 4.1 in [18]. We would like to note that

while this master equation contains all the information about the ε-MFG in the

same way that the classical HJB equation contains all the information about the

control problem, unlike the classical HJB equation, solving the master equation even

numerically is not feasible as we need to discretize the P2(R) space.

Next, we proceed in a similar way as done by Lasry and Lions in [13] for the

case of no common noise. That is, we consider the value function along the path of

the optimal distribution. Recall that we assume that there exists a unique solution

(αεt )0≤t≤T to the ε-MFG and let (mε

t )0≤t≤T ∈ M([0, T ];P2(R)) be the corresponding

conditional law given by the stochastic Fokker-Plank equation (2.1). We consider the


value function along (mεt )0≤t≤T by letting

uε(t, x) , Vε(t, x,mεt )

Using Ito-Kunita formula and the master equation (2.8), it follows that

duε(t, x) =

(

∂tVε(t, x,mε

t )− E0[

∂mVε(t, x,mε

t )(X)(α(t, X,mεt , ∂xV

ε(t, X,mεt )))]

+σ2

2∂mmV

ε(t, x,mεt )(X)[ζ, ζ ] +

ε2

2∂mmV

ε(t, x,mεt )(X)[1, 1]

)

dt

− εE0[

∂mVε(t, x,mε

t )(X)1]

dWt

=

(

− H(t, x,mεt , ∂xV

ε(t, x,mεt ))−

σ2

2∂xxV

ε(t, x,mεt )−

ε2

2∂xxV

ε(t, x,mεt )

− ε2E0[

∂xmVε(t, x,mε

t )(X)1]

)

dt− εE0[

∂mVε(t, x,mε

t )(X)1]

dWt

=

(

− H(t, x,mεt , ∂xu

ε(t, x))−σ2

2∂xxu

ε(t, x)−ε2

2∂xxu

ε(t, x)

− ε2E0[

∂xmVε(t, x,mε

t )(X)1]

)

dt− εE0[

∂mVε(t, x,mε

t )(X)1]

dWt

=

(

−H(t, x,mεt , ∂xu

ε(t, x))−σ2

2∂xxu

ε(t, x)−ε2

2

(

∂xxuε(t, x)− 2∂xv

ε(t, x))

)

dt

− εvε(t, x)dWt

where

vε(t, x) , E0[

∂mVε(t, x,mε

t )(X)1]

Combining with (2.1) and (2.7), we have arrived at a system of forward backward


stochastic partial differential equations (FBSPDE)

duε(t, x) =

(


σ2

2∂xxu

ε −ε2

2∂xxu

ε − ε2∂xvε

)

dt− εvεdWt

dmε(t, x) =

(


ε)mε) +σ2 + ε2

2∂xxm

ε

)

dt− ε∂xmε dWt

(2.9)

with boundary conditions

mε(0, x) = m0(x) = L(ξ0), uε(T, x) = g(x,mεT )

Recall that H denotes the Hamiltonian (see (2.11)) and α denotes the minimizer of

H (see (2.6)).

Similarly to Theorem 2.1.2, we have a verification theorem for (2.9) which states

that if we have a sufficiently regular solution (uε, mε, vε) to the FBSPDE (2.9) above,

then the ε-MFG solution is given in a feedback form as

αεt = α(t, Xε

t , mεt , ∂xu

ε(t, Xεt )),

We refer the reader to Section 4.2 in [10] for this result. The triple (uε, mε, vε) then

gives, respectively, the value function, distribution of the optimal state process, and

the sensitivity of the value function with respect to a spatial shift of the distribution

process. Consequently, despite the fact that (2.9) above is derived from a solution of

the master equation, it actually contains the same information as the master equation.

The function Vε represents the value function at time t as a function of (x,m) while

uε represents the value function at time t as a function of (x, ω) where ω is a common

Brownian motion path.

Remark 2.1.3. As mentioned earlier, the FBSPDE above is simply a different rep-

resentation of a value function and is not necessarily easier to deal with than the

master equation. Nevertheless, by considering (uε, mε), we can avoid working with a

derivative with respect to a probability measure. Also, it might be useful in the case

of a small ε since we can attempt to analyze the approximation of (uε, mε) around

(u0, m0).


When ε = 0, the system of SPDEs above reduces to a system of PDEs

du0(t, x) =

(

−H(t, x,m0, ∂xu0)−

σ2

2∂xxu

0

)

dt

dm0(t, x) =

(

−∂x(α(t, x,m0, ∂xu

0)m0) +σ2

2∂xxm

0

)

dt

or equivalently,

∂tu0 = −H(t, x,m0, ∂xu

0)−σ2

2∂xxu

0, u0(T, x) = g(x,m0T )

∂tm0 = −∂x(α(t, x,m

0, ∂xu0)m0) +

σ2

2∂xxm

0, m0(0, x) = m0(x) = L(ξ0)

(2.10)

Recall that H denotes the Hamiltonian (see (2.11)) and α denotes the minimizer of

H (see (2.6)).

The system of PDEs above is precisely the 0-MFG model originally derived and

studied by Larsy and Lions in a series of papers [42, 43, 44, 13]. The system (2.10)

says that when there is no common noise, we can turn the infinite-dimensional HJB

equation into a system of finite-dimensional fully-coupled forward backward PDEs.

Solving this system gives an optimal control and the distribution of the optimal state

process.

However, this dimension reduction cannot be obtained when there is common noise

since we need to specify an optimal control for each trajectory of the common Brow-

nian motion. As we shall see in Chapter 4, some finite dimensional approximation

can be done when the common noise is small.

2.2 Stochastic Maximum Principle (SMP)

Stochastic Maximum Principle (SMP) or Pontryagin Maximimum Principle is an

approach to a control problem which studies optimality conditions satisfied by an

optimal control. It gives sufficient and necessary conditions for the existence of an

optimal control in terms of a backward stochastic differential equation (BSDE) of an

adjoint process. In this section, we review general SMP results and apply them to


the MFG problem. For more details about SMP in general, see [53, 61] for instance.

Consider a general continuous time stochastic control problem with state process

in R given by the following dynamic

dXs = b(t, Xs, αs)dt+ σ(t, Xs)dBs, X0 = ξ0

where (Bt)0≤t≤T denotes a d-dimensional Brownian motion over a filtered probability

space (Ω,F , Ft0≤t≤T ,P), (αt)0≤t≤T denotes a control, b, σ are given functions taking

values in R,Rd respectively. Our goal is to select α = (αt)0≤t≤T ∈ H2([0, T ];R) to

minimize the expect cost

J (α) = E

[∫ T

0

r(t, Xt, αt)dt+ h(XT )

]

where r, h are running and terminal cost functions taking values in R. We assume

that b, σ, r, h are C1 in x with bounded derivative, r, h are also continuous in (t, x) for

all α and satisfy a quadratic growth condition. Define the generalized Hamiltonian

H : [0, T ]× Rd+3 → R by

H(t, a, x, y, z) , b(t, x, a)y + σ(t, x)T z + r(t, x, a) (2.11)

Given a control and the corresponding state process, we consider a pair of Ft-adapted

processes (Yt, Zt)0≤t≤T which satisfies

Yt = ∂xh(XT ) +

∫ T

t

∂xH(t, αt, Xt, Yt, Zt)dt−

∫ T

t

ZtdBt (2.12)

The equation above is called an adjoint equation and (Yt)0≤t≤T , if exist, is called

the adjoint process. Equation (2.12) is a backward stochastic differential equation

(BSDE) and can be written in a differential form as

dYt = −∂xH(t, αt, Xt, Yt, Zt)dt+ ZtdBt, YT = ∂xh(XT ) (2.13)

Remark 2.2.1. The process (Zt)0≤t≤T is part of a solution to ensure Ft-adaptivity


of Yt. For a quick introduction to BSDE, we refer to Ch.1 of [49].

The following theorem gives necessary conditions for an optimal control

Theorem 2.2.2. Suppose (αt)0≤t≤T is an optimal control with (Xt)0≤t≤T being a

corresponding state process, then the adjoint equation

dYt = −∂xH(t, αt, Xt, Yt, Zt)dt+ ZtdBt, YT = ∂xh(XT )

has an Ft-adapted solution (Yt, Zt)0≤t≤T and the following maximum condition holds

αt = argmaxa∈R

H(t, a, Xt, Yt, Zt), 0 ≤ t ≤ T, a.s.

Remark 2.2.3. There are more general statements involving the second-order varia-

tional process and a more general Hamiltonian when the diffusion coefficient depends

on a control and/or when the control space is non-convex. Proof for Theorem 2.2.2 is

based on the Taylor expansion of the cost functional J (·) around an optimal control

which is perturbed by a “spike” or “needle” variation. See Ch.3 in [61] for details.

When the cost functions are convex, then we have sufficient conditions which gives

an optimal control in term of a solution to the adjoint process.

Theorem 2.2.4. Let α = (αt)0≤t≤T ∈ H2([0, T ];R) be a control and X = (Xt)0≤t≤T

be the corresponding state process. Suppose that there exist a solution (Yt, Zt)0≤t≤T to

the corresponding adjoint equation such that

H(t, αt, Xt, Yt, Zt) = mina∈R

H(t, a, Xt, Yt, Zt), 0 ≤ t ≤ T, a.s.

and both h and (a, x) → H(t, a, x, Yt, Zt) are convex for all t ∈ [0, T ], then αt is an

optimal control, i.e.

J (α) = infα∈H2([0,T ];R)

J (α)

If, in addition, H(t, a, x, Yt, Zt) is strongly convex in a uniformly in t ∈ [0, T ] and


x ∈ R, i.e. there exist a constant C such that for all t ∈ [0, T ] and x, a, a′ ∈ R,

H(t, a, x, Yt, Zt) ≥ H(t, a′, x, Yt, Zt) + ∂aH(t, a′, x, Yt, Zt)(a− a′) + C|a− a′|2 (2.14)

then for any control β = (βt)0≤t≤T , the following estimate holds;

J (β) ≥ J (α) + C

∫ T

0

|βt − αt|2dt (2.15)

Proof. Let β ∈ H2([0, T ];R) be an arbitrary admissible control and Xt be the cor-

responding state process. Let ∆rt = r(t, Xt, αt) − r(t, Xt, βt) and define similarly

∆Xt,∆h,∆bt,∆σt, then using convexity of h and Ito’s lemma, it follows that

J (α)− J (β) = E

[∫ T

0

∆rtdt+∆h

]

≤ E

[∫ T

0

∆rtdt+ ∂xh(XT )∆XT

]

= E

[∫ T

0

∆rtdt+ YT∆XT

]

= E

[∫ T

0

(

∆rtdt+ Yt∆bt + Zt∆σt − ∂xH(t, αt, Xt, Yt, Zt)∆Xt

)

dt

]

= E

[∫ T

0

(

H(t, αt, Xt, Yt, Zt)−H(t, βt, Xt, Yt, Zt)− ∂xH(t, αt, Xt, Yt, Zt)∆Xt

)

dt

]

≤ 0

where the last inequality is from the fact that ∂aH(t, αt, Xt, Yt, Zt) = 0 and (a, x) →

H(t, a, x, Yt, Zt) is convex. Thus,

J (α) = minβ∈H2([0,T ];R)

J (β)

If H(t, a, x, Yt, Zt) is strongly convex in a (see (2.14)), then the last inequality can be


strengthened,

J (α)− J (β) = E

[∫ T

0

(

H(t, αt, Xt, Yt, Zt)−H(t, βt, Xt, Yt, Zt)− ∂xH(t, αt, Xt, Yt, Zt)∆Xt

)

dt

]

≤ −CE

[∫ T

0

|βt − αt|2dt

]

and we get the estimate (2.15) as desired.

Remark 2.2.5. The estimate (2.15) implies that an optimal control, if it exists, is

unique.

Remark 2.2.6. Note that for our particular ε-MFG model as defined in Section 1.1,

we have b(t, x, a) = a, σ(t, x, a) = (σ, ε), and Bt = (Wt, Wt).

When applying SMP to solve a control problem, if possible, one usually begins by

solving for a minimizer of the Hamiltonian as a function of (t, x, y, z), i.e.

α(t, x, y, z) = arg infa∈R

H(t, a, x, y, z)

The minimizer is unique when the Hamiltonian is (strictly) convex. We plug this

minimizer function back to the SDEs which results in a coupled system of SDE where

one is a forward SDE of the state process and the other is a backward SDE of the

adjoint process,

dXt = b(t, Xt, α(t, Xt, Yt, Zt))dt+ σ(t, Xt)dBt

dYt = −∂xH(t, α(t, Xt, Yt, Zt), Xt, Yt, Zt)dt+ ZtdBt

X0 = ξ0, YT = ∂xh(XT )

(2.16)

This type of system is called a forward backward stochastic differential equation

(FBSDE). FBSDE was first introduced by Bismut in [12] from a stochastic control

problem similar to what is presented here. Since then, the wellposed-ness theory of

FBSDE has expanded significantly. See [49] for a general reference on FBSDE and

[47] for a more recent development on a wellposed-ness theory.


If we can solve the FBSDE (2.16) above to get a solution (Xt, Yt, Zt)0≤t≤T and

show that

αt , α(t, Xt, Yt, Zt) ∈ H2([0, T ];R),

it then follows from Theorem 2.2.4 that αt is an optimal control. In fact, from both

necessary (Theorem 2.2.2) and sufficient (Theorem 2.2.4) conditions, when operating

under convexity assumption as we are, solving for an optimal control is equivalent to

solving the corresponding FBSDE.

Next, we would like to apply the SMP to our ε-MFG problem. We begin by consid-

ering the individual control problem given a stochastic continuous flow of probability

measure as defined in (1.1.2). Given (mt)0≤t≤T ∈ M([0, T ];P2(R)), then we have a

standard control problem with running and terminal cost functions given by

r(t, x, α, ω) = f(t, x,mt(ω), α), h(x, ω) = g(x,mT (ω))

Note that with (mt)0≤t≤T being random, r, h is now a random function. A represen-

tative player attempts to control his/her state process

dXαt = αtdt+ σdWt + εdWt

to minimize the cost

J (α) = E

[∫ T

0

(f(t, Xαt , mt, αt)dt+ g(Xα

T , mT )

]

With m as an argument in the cost functions, the generalized Hamiltonian now reads

H(t, a, x,m, y, z) = ay + f(t, x,m, a) + σz

Consequently,

∂xH(t, a, x,m, y, z) = ∂xf(t, x,m, a)


and the minimizer is given by

α(t, x, y, z) = argmina∈R

(ay + f(t, x,m, a) + σz)

Note that both ∂xH and α is independent of z, so we can reduce our notation and

exclude the z-argument in both H and α. In addition, it is more convenient to work

with the Hamiltonian (as opposed to the generalized Hamiltonian)

H(t, x,m, y) = mina∈R

(ay + f(t, x,m, a)) = H(t, α(t, x, y,m), x,m, y)

Using optimality of α, it is easy to check that

∂xH(t, x,m, y) = ∂xH(t, α(t, x, y,m), x,m, y)

Thus, the corresponding FBSDE for an individual control problem given m is given

by

dXt = α(t, Xt, Yt, mt)dt+ σdWt + εdWt


X0 = ξ0, YT = ∂xg(XT , mT )

(2.17)

By applying the necessary and sufficient conditions above, we have the following

theorem which turns the control problem of each individual player to an FBSDE with

random coefficient

Theorem 2.2.7. Assume A holds, then the individual control problem given (mt)0≤t≤T ∈

M([0, T ];P2(R)) has an optimal control if and only if the FBSDE (2.17) is solvable.

In that case, the optimal control is given by

αt = α(t, Xt, mt, Yt)

for all t ∈ [0, T ], where (Xt, Yt, Zt, Zt)0≤t≤T is a solution to FBSDE (2.17).

In fact, as we shall see in Chapter 3 (see particularly Theorem 3.1.3), under the

same set of assumptions, the FBSDE (2.17) is uniquely solvable.


Next, we proceed from a single player control problem to a mean field game. From

the definition of ε-MFG (see Definition 1.1.4), it simply requires that the following

equilibrium or consistency condition holds

mt = L(Xt|Ft)

where Xt is an optimal controlled process for an individual control problem given

m = (mt)0≤t≤T . Adding this condition into FBSDE (2.17) yields the SMP for ε-

MFG.

Theorem 2.2.8. Assume that A holds, then ε-MFG is solvable if and only if the

FBSDEdXt = α(t, Xt,L(Xt|Ft), Yt)dt+ σdWt + εdWt

dYt = −∂xH(t, Xt,L(Xt|Ft), Yt)dt+ ZtdWt + ZtdWt

X0 = ξ0, YT = ∂xg(XT ,L(XT |FT ))

(2.18)

is solvable. In that case, ε-MFG solution is given by

αt = α(t, Xt,L(Xt|Ft), Yt), ∀t ∈ [0, T ]

Proof. Suppose (αt)0≤t≤T is an ε-MFG solution, then let (mt)0≤t≤T be the conditional

law of the corresponding state process, i.e.

mt = L(Xt|Ft)

where (Xt)0≤t≤T follows the dynamic

dXt = αtdt + σdWt + εdWt, X0 = ξ0

By definition of an ε-MFG solution, α must be an optimal control for an individual


control problem given m as well. Thus by Theorem 2.2.2, the FBSDE

dXt = α(t, Xt, mt, Yt)dt+ σdWt + εdWt


X0 = ξ0, YT = ∂xg(XT , mT )

(2.19)

is solvable and αt = α(t, Xt, mt, Yt). Thus, the forward solution (Xt)0≤t≤T to (2.19)

has the same law as (Xt)0≤t≤T , so that mt = L(Xt|Ft) = L(Xt|Ft). Plug this back to

(2.19) and we get (2.18) as desired.

Now, suppose (2.18) is solvable and denote a solution by (Xt, Yt, Zt, Zt)0≤t≤T . Let

αt = α(t, Xt,L(Xt|Ft), Yt), then the result follows directly from the definition of ε-

MFG and Theorem 2.2.4 using the fact that f, g are convex under assumption A.

The FBSDE (2.18) above is a mean-field type or more specifically McKean-Vlasov

type FBSDE as it involves the law of the process. The McKean-Vlasov forward SDE

was first introduced by McKean as a limit of interacting particles system following

diffusion processes. We refer to [25, 56] for classical references on a general McKean-

Vlasov theory. The FBSDE version of McKean-Vlasov equation was first introduced

by Carmona and Delarue in [17] as a result of SMP applied to MFG with no common

noise or 0-MFG. In that case, the conditional law L(Xt|Ft) is simply L(Xt). To the

best of our knowledge, McKean-Vlasov FBSDE with conditional law has not been

considered. We will show existence and uniqueness result for this FBSDE in Chapter

3 under an additional monotonicity type condition.

2.3 Comparison between the two approaches

We now describe the connection between the two approaches in the MFG context.

In a general stochastic control problem, it is known that the adjoint process is a

gradient of a value function along the optimal path provided that the value function

is sufficiently regular. See Theorem 6.4.2 in [53] for instance. The following theorem

gives a similar result in the MFG context with the generalize value function Vε.


Theorem 2.3.1. Suppose that Vε ∈ C1,3,3([0, T )×R×P2(R))∩C([0, T ]×R×P2(R))

is a classical solution to the master equation (2.8). Suppose that ∂xxVε, ∂xmVε, and

∂xVε(t, 0, δ0) are bounded then

(Xt, Yt, Zt, Zt)0≤t≤T =(

Xεt , ∂xV

ε(t, Xεt , m

εt), σ∂xxV

ε(t, Xεt , m

εt), ε

(

∂xxVε(t, Xε

t , mεt ) + E

0[

∂xmVε(t, Xε

t , mεt)(Xt)

]))

(2.20)

is a solution to FBSDE (2.18) satisfying

E

[

sup0≤t≤T

(Xεt )

2 + sup0≤t≤T

(Y εt )

2 +

∫ T

0

(Zεt )

2 +

∫ T

0

(Zεt )

2

]

<∞ (2.21)

where (Xεt )0≤t≤T is a solution to the SDE


mεt = L(Xε

t |Ft), and (Xt)0≤t≤T is a lifting random variable of (Xεt )0≤t≤T sharing a

common noise space. In particular,

αεt , α(t, Xt,L(Xt|Ft), ∂xV

ε(t, Xt,L(Xt|Ft))) ∈ H2([0, T ];R)

is a solution to ε-MFG.

Proof. The bounded second derivative condition of Vε implies that the SDE


admits a unique solution. Combining this with the fact that ∂xVε(t, 0, δ0) is bounded,

we have that ∂xVε satisfies the linear growth condition. By standard SDE estimate, we

have that (Xεt , Y

εt , Z

εt , Zt)0≤t≤T satisfying (2.21). Then we can apply generalized Ito’s

lemma to ∂xVε(t, Xε

t ,L(Xεt |Ft)) (see Proposition 6.5 in [18]) to verify directly that

(Xεt , Y

εt , Z

εt , Zt)0≤t≤T satisfies FBSDE (2.18). The last remark follows from Theorem

2.2.8


Remark 2.3.2. Observe that extra regularity on a value function is required to estab-

lish the connection between DPP and SMP. For a classical stochastic control problem,

there is a more general result involving a viscosity solution and a sub/super gradient

that does not require as much regularity. We refer to [61] for more details on the

relation between the two approaches in a classical stochastic control set up.

We now summarize the discussion of the two approaches to MFG in Table 2.1 by

describing the types of equations that results from applying both approaches to MFG

model in both cases: with and without common noise.

Approachε = 0

(no common noise)

ε > 0

(with common noise)

Dynamic Programming Principle FBPDEInfinite-dimensional HJB

or FBSPDE

Stochastic Maximum Principle McKean-Vlasov FBSDEMcKean-Vlasov FBSDE

with conditional law

Table 2.1: Summary of the types of equations from applying DPP and SMP ap-

proaches to MFG problems

Most of the prior works has been on the upper-left corner of the table, i.e. the

study of FBPDE (2.10) and some of its variations. However, extending those results

to the ε > 0 case is non-trivial because we now either have to work with an infinite-

dimensional non-linear PDE or a system of FBSPDE. In either cases, they are much

more complicated than a system of finite-dimensional PDEs and are much less studied.

For the SMP, the main difference between a model with and without common noise

is in the mean-field terms appearing in the FBSDE. Changing from a deterministic

unconditional law to a law conditional on a common brownian motion filtration, the

FBSDE for a given m is changed from having deterministic coefficients to the one

with random coefficients from a stochastic flow of probability measure. These two

kinds of FBSDE are very different as the former can be solved via PDE techniques


while the tools to tackle the latter are much more limited. In addition, the spaces

of deterministic flows and stochastic flows of probability measures are considerably

different. As a result, the proof based on compactness and the Schauder fixed point

theorem previously used in the 0-MFG by Carmona and Delarue in [17] cannot be

extended to the ε-MFG case. Despite this fact, there have been active researches on

FBSDE with random coefficients in the past decades. See [5, 35, 47, 48, 52, 59, 60],

and these results are helpful in dealing with Mckean-Vlasov FBSDE with conditional

law.

Chapter 3

Mean field games with common

noise

In this chapter, we discuss a general MFG with common noise. In Section 3.1, we show

the existence and uniqueness of ε-MFG solutions for a linear-convexity model with a

weak monotonicity condition using SMP approach discussed in the previous chapter.

In Section 3.2, we discuss the Markov property of ε-MFG by showing existence of a

deterministic decoupling function. We then discuss the connection to HJB through

this decoupling function in Section 3.3.

3.1 Wellposedness of MFG with common noise

Recall that by applying the SMP to ε-MFG, we turn the problem to solving the

following McKean-Vlasov FBSDE (see Theorem 2.2.8)

dXt = α(t, Xt,L(Xt|Ft), Yt)dt+ σdWt + εdWt

dYt = ∂xH(t, Xt,L(Xt|Ft), Yt)dt+ ZtdWt + ZtdWt

X0 = ξ, YT = ∂xg(XT ,L(XT |FT ))

(3.1)

Our goal is to show that the system above is uniquely solvable under assumption

A and B outlined in Section 1.1.4. We will in fact prove a slightly more general

result, namely the existence and uniqueness of an FBSDE with monotone functional,

64

CHAPTER 3. MEAN FIELD GAMES WITH COMMON NOISE 65

since this type of FBSDE will appear frequently in our subsequent analysis.

3.1.1 FBSDE with monotone functionals

We are interested in the following FBSDE over [s, T ]

dXt = b(t, Xt, Yt)dt+ σdWt + εdWt

dYt = F (t, Xt, Yt)dt+ ZtdWt + ZtdWt

Xs = ξ, YT = G(XT )

(3.2)

where ξ ∈ L2Fs

and b, F,G are measurable maps

b : [0, T ]×L2F × L2

F × Ω → R

F : [0, T ]× L2F ×L2

F × Ω → R

G : L2F × Ω → R

(3.3)

Notice that the maps b, F,G are “functionals”. An example of b, F,G to keep in mind

is precisely that from McKean-Vlasov FBSDE (3.1) of ε-MFG. That is,

b(t, X, Y ) = α(t, X,L(X|Ft), Y )

F (t, X, Y ) = −∂xH(t, X,L(X|Ft), Y )


(3.4)

In this example, they are functionals only with respect to X . We now list a set of

assumptions (denoted by H) on b, F,G. Being a functional, we first need to ensure

that the stochastic integral is well-defined, so the following measurability assumption

is necessary

(H1). (b(t, Xt, Yt))0≤t≤T , (F (t, Xt, Yt))0≤t≤T are Ft-progressively measurable for any

Ft-progressively measurable (Xt)0≤t≤T ,(Yt)0≤t≤T .

The second assumption is a standard SDE assumption, namely Lipschitz and

linear growth condition. In a functional form, it is stated under expectation, or


for our application to ε-MFG, under a conditional expectation. Fix a sub-filtration

G = Gt0≤t≤T of F = Ft0≤t≤T , then the assumption reads

(H2). There exist a constant K such that for any t ∈ [0, T ], X,X ′, Y, Y ′ ∈ L2FT

,

A ∈ Gt.

E[

1A(b(t, X, Y )− b(t, X ′, Y ′))2]

≤ KE[

1A((X −X ′)2 + (Y − Y ′)2)]

E[

1A(F (t, X, Y )− F (t, X ′, Y ′))2]

≤ KE[

1A((X −X ′)2 + (Y − Y ′)2)]

E[

1A(G(X)−G(X ′))2]

≤ KE[

1A(X −X ′)2]

(3.5)

and

E

[∫ T

0

b(t, 0, 0)2 + F (t, 0, 0)2dt

]

<∞ (3.6)

In addition to the standard assumptions above, the following monotonicity con-

dition on b, F,G is needed.

(H3). For any t ∈ [0, T ], X,X ′, Y, Y ′ ∈ L2FT

, and A ∈ Gs,

E [1A(F (t, X, Y )− F (t, X ′, Y ′))(X −X ′) + 1A(b(t, X, Y )− b(t, X ′, Y ′))(Y − Y ′)] ≤

−β1E[

1A(b(t, X, Y )− b(t, X ′, Y ′))2]

− β2E[

1A(Y − Y ′)2]

E [1A(G(X)−G(X ′))(X −X ′)] ≥ 0

(3.7)

for some constant β1, β2 ≥ 0 with β1 + β2 > 0.

This monotonicity condition is motivated by results in standard FBSDE theory

that have successfully dealt with the case of random coefficients. See [52, 49] for in-

stance. This type of equation, with these three assumptions on the drift and terminal

functionals, covers most, if not all, of the FBSDEs that we will encounter in this

thesis including those in Chapter 4 where we analyze asymptotic behavior when the

common noise is small. For instance, given m = (mt)0≤t≤T ∈ M([0, T ];P2(R)), then

b(t, X, Y ) = α(t, X,mt, Y )

F (t, X, Y ) = −∂xH(t, X,mt, Y )

G(X) = ∂xg(X,mT )

(3.8)


would correspond to FBSDE (2.17) arising from an individual control problem given

m with f, g satisfying assumption A. As we shall see in Theorem 3.1.3, the convexity

on f, g gives a monotonicity condition on b, F,G defined above. Another case and

perhaps a more important one is (3.4) as it corresponds to the McKean-Vlasov FBSDE

(2.2.8) arising from the ε-MFG as discussed above.

3.1.2 A priori estimate

We first present a priori estimates necessary for our proof of existence. Working with

FBSDE, this type of estimate can usually be obtained only when the time duration

is sufficiently small. In this case, we can achieve this estimate for arbitrary time

s ∈ [0, T ] by using the monotonicity condition (H3)

Theorem 3.1.1. Let ξ1, ξ2 ∈ L2Fs

and, for i = 1, 2, let (X it , Y

it , Z

it , Z

it)s≤t≤T denote

a solution to FBSDE (3.2) with initial condition ξi and coefficients (σi, εi, bi, Fi, Gi).

That is, they satisfy

dX it = bi(t, X

it , Y

it )dt+ σidWt + εidWt

dY it = Fi(t, X

it , Y

it ) + Z i

tdWt + Z itdWt

X is = ξi, Y i

T = Gi(XiT )

(3.9)

Assume that (bi, Fi, Gi) satisfies assumptions H. Then there exists a constant CK,T

that depends only on K, T such that the following estimate holds for any A ∈ Gs:

E

[

sups≤t≤T

1A(∆Xt)2 + sup

s≤t≤T1A(∆Yt)

2 +

∫ T

s

[1A(∆Zt)2 + 1A(∆Zt)

2]dt

]

≤ CK,T

(

E

[

1A(∆ξ)2 + 1AG(X

1T )

2 + 1A

∫ T

s

(

F (θ1t )2 + b(θ1t )

2)

dt

]

+ (∆σ)2 + (∆ε)2)

(3.10)

where∆Xt = X1t −X

2t , b = b1−b2, θ1t = (t, X1

t , Y1t ) and∆Yt,∆Zt,∆Zt,∆ξ,∆σ,∆ε,F , G

are defined similarly.

Proof. See Section 3.4.1.


3.1.3 Wellposedness result

Using estimate (3.10), we are ready to prove our main theorem which shows that

FBSDE (3.2) is wellposed. We then apply this result to establish unique solvability

of two FBSDEs: FBSDE (2.17) arising from an individual control problem given m,

and McKean-Vlasov FBSDE (3.1) from ε-MFG.

Theorem 3.1.2. Let ξ ∈ L2Fs

and b, F,G be functionals satisfying assumption H, then

there exists a unique adapted solution (Xt, Yt, Zt, Zt)s≤t≤T to FBSDE (3.2) satisfying

the estimate

E

[

1A sups≤t≤T

X2t + 1A sup

s≤t≤TY 2t + 1A

∫ T

s

[Z2t + Z2

t ]dt

]

≤ CK,T

(

E

[

1Aξ2 + 1AG(0)

2 + 1A

∫ T

s

(

b(t, 0, 0)2 + F (t, 0, 0)2)

dt

]

+ σ2 + ε2)

(3.11)

for some constant CK,T depends only on K, T and for all A ∈ Gs.

Proof. The proof can be summarized as follows; we begin by showing existence of a

solution over a small time duration by means of Banach fixed point theorem. This

small time depends on the Lipschitz constants of the functionals. We then define

a new terminal functional and repeat the process backward in time. The estimate

(3.10) above allows us to control the Lipschitz constant of the terminal functional

as we move backward to construct a solution. See Section 3.4.2 for details of the

proof.

We now apply Theorem 3.1.2 to FBSDE (2.17) (over [s, T ]) by setting

b(t, X, Y ) = α(t, X,mt, Y ),

F (t, X, Y ) = −∂xH(t, X,mt, Y ),

G(X) = ∂xg(X,mT )

Gts≤t≤T = Fts≤t≤T

(3.12)

for a given (mt)s≤t≤T ∈ M([s, T ];P2(R)), where H is the Hamiltonian, α is its min-

imizer function, and g is the terminal cost function. See (2.5) and (2.6) for their


definitions. This FBSDE corresponds to an individual stochastic control given m.

See Theorem 2.2.7.

Theorem 3.1.3. Assume thatA holds, there exist a unique solution (Xt, Yt, Zt, Zt)s≤t≤T

to the FBSDE (2.17) over [s, T ] satisfying

E

[

sups≤t≤T

[X2t + Y 2

t ] +

∫ T

s

[Z2t + Z2

t ]dt

]

≤ CK,TE

[

ξ2 + (∂xg(0, mT ))2 +

∫ T

s

(α(t, 0, mt, 0))2 + (∂xH(t, 0, mt, 0))

2dt+ σ2 + ε2]

(3.13)

Moreover, two solutions (X it , Y

it , Z

it , Z

it)s≤t≤T , i = 1, 2 to FBSDE (2.17) with initial ξi

satisfies the estimate

E

[

sups≤t≤T

1A∆X2t + sup

s≤t≤T1A∆Y

2t +

∫ T

s

[1A∆Z2t + 1A∆Z

2t ]dt

]

≤ CK,TE[1A∆ξ2]

(3.14)



depends only on K, T , and A ∈ Fs.

Proof. We simply need to check that under assumption A on f, g, the correspond-

ing b, F,G as defined in (3.12) satisfy assumptions H with filtration Gt0≤t≤T =

Ft0≤t≤T . Then we can apply Theorem 3.1.2 to get the desired result.

First, by measurability of f, g, hence for α, ∂xH , and the assumption that (mt)0≤t≤T

is Ft-progressively measurable, (H1) follows. Next, by definition, α satisfies

∂αf(t, x,m, α(t, x,m, y)) + y = 0 (3.15)

for any x, y ∈ R, m ∈ P2(R). Consequently, by the Lipschitz property of ∂αf in

(x, α,m), it follows that α is Lipschitz in x, y. In addition, using the optimality

condition on α, we have

∂xH(t, x,m, y) = ∂xf(t, x,m, α(t, x,m, y)) (3.16)

Combining with Lipschitz assumption (A1) on ∂xf, ∂αf, ∂xg, (H2) follows easily. We


now check the monotonicity condition (H3). By strict convexity assumption on f (see

(1.9) in (A3)), we have

f(t, x,m, α)+∂xf(t, x,m, α)(x′−x)+∂α(t, x,m, α)(α

′−α)+cf |α′−α|2 ≤ f(t,m, x′, α′)

where

α = α(t, x,m, y), α′ = α(t, x′, m, y′)

Interchanging (x, α) and (x′, α′), we also have

f(t, x′, m, α′)+∂xf(t, x′, m, α′)(x−x′)+∂αf(t, x

′, m, α′)(α−α′)+cf |α′−α|2 ≤ f(t, x,m, α)

Summing both equations and using (3.15), it follows that

−(∂xf(t, x,m, α)−∂xf(t, x′, m, α′))(x−x′)+(y−y′)(α− α′)+cf (α− α

′)2 ≤ 0 (3.17)

for any x, x′, y, y′ ∈ R. The condition (H3) then follows by (3.16),(3.17) with β1 =

cf , β2 = 0. The monotonicity of G is obvious from the convexity of g. The estimate

(3.14) follows immediately from estimate (3.10) in Theorem 3.1.2

In addition to estimate (3.14), we are also interested in an estimate between two

solutions from two different stochastic flows of probability measure.

Proposition 3.1.4. In addition to A, we assume that ∂xf, ∂αf, ∂xg are Lipschitz

in m with the same Lipschitz constant K. Let ξ1, ξ2 ∈ L2Fs, (m1

t )s≤t≤T , (m2t )s≤t≤T ∈

M([s, T ];P2(R)), and (Xit , Y

it , Z

it , Z

it)s≤t≤T denote the solution to FBSDE (2.17) given

mi and initial ξi, then the following estimate holds

E

[

sups≤t≤T

1A∆X2t + sup

s≤t≤T1A∆Y

2t +

∫ T

s

[1A∆Z2t + 1A∆Z

2t ]dt

]

≤ CK,TE[1A∆ξ2 + 1A

∫ T

s

(∆mt)2dt]

(3.18)

where∆Xt = X1t −X

2t , ∆Yt,∆Zt,∆Zt,∆ξ are defined similarly, and∆mt = W2(m

1t , m

2t ).


Proof. Let

bi(t, X, Y ) = α(t, X,mit, Y ),

F i(t, X, Y ) = −∂xH(t, X,mit, Y ),

Gi(X) = ∂xg(X,miT )

Gt0≤t≤T = Ft0≤t≤T

(3.19)

for i = 1, 2. Then from the Lipschitz in m of ∂xf, ∂αf, ∂xg, it follows that α, ∂xH, ∂xg

are Lipschitz in m and the result follows by applying estimate (3.10) in Theorem 3.1.2

with (ξi, bi, F i, Gi), i = 1, 2 above.

Remark 3.1.5. Additional assumption in Proposition 3.1.4 holds under (B2),(B1).

Going back to McKean-Vlasov FBSDE (2.2.8), we now apply Theorem 3.1.2 by

setting

b(t, X, Y ) = α(t, X,L(X|Ft), Y ),

F (t, X, Y ) = −∂xH(t, X,L(X|Ft), Y ),


Gts≤t≤T = Fts≤t≤T

(3.20)

Notice that the filtration used here is not as general as that in the previous theorem.

The result is the following;

Theorem 3.1.6. [Wellposedness of McKean-Vlasov FBSDE] Assume that A,B hold,

then there exist a unique solution (Xt, Yt, Zt, Zt)s≤t≤T to FBSDE (2.2.8) satisfying

E

[

sups≤t≤T

[X2t + Y 2

t ] +

∫ T

s

[Z2t + Z2

t ]dt

]

≤ C

(

E[ξ2] + (∂xg(0, δ0))2 +

∫ T

s

(α(t, 0, δ0, 0))2 + (∂xH(t, 0, δ0, 0))

2dt+ σ2 + ε2)

(3.21)

where δa denote the dirac measure at a. Moreover, given two solutions (X it , Y

it , Z

it , Z

it)s≤t≤T , i =

1, 2 to FBSDE (2.2.8) with initial ξi, the following estimate holds:

E

[

sups≤t≤T

1A∆X2t + sup

s≤t≤T1A∆Y

2t +

∫ T

s

[1A∆Z2t + 1A∆Z

2t ]dt

]

≤ CK,TE[1Aξ2] (3.22)




depends only on K, T , and A ∈ Fs.

Proof. Similar to Theorem 3.1.3, we simply need to check that under assumption A,

B on f, g, the corresponding b, F,G as defined in (3.20) satisfy assumption H with

filtration Gt0≤t≤T = Ft0≤t≤T .

First, (H1) follows from the measurability of f, g and the fact that (Xt)0≤t≤T 7→

(L(Xt|Ft))0≤t≤T is measurable. Next, by separable assumption (B2), α is independent

of m, that is, α := α(t, x, y) satisfies

∂αf0(t, x, α(t, x, y)) + y = 0 (3.23)

Consequently, by Lipschitz property of ∂αf0 in (x, α), it follows that α is Lipschitz in

x, y. In addition, using the optimality condition on α, we have

∂xH(t, x,m, y) = ∂xf0(t, x, α(t, x, y)) + ∂xf

1(t, x,m) (3.24)

Combining with Lipschitz assumption (A1),(B1) on ∂xf, ∂αf, ∂xg, (H2) follows easily.

We now check the monotonicity condition (H3). By strict convexity assumption on

f 0, (see (1.9) in (A3)), we follow the same argument as in Theorem 3.1.3 to get

−(∂xf0(t, x, α)− ∂xf

0(t, x′, α′))(x− x′) + (y − y′)(α− α′) + cf(α− α′)2 ≤ 0 (3.25)

for any x, x′, y, y′ ∈ R. where

α = α(t, x, y), α′ = α(t, x′, y′)


Now, by weak monotonicity condition (B3) on f, g, we have

E

[

1A(∂xf1(t, X,L(X|Ft))− ∂xf

1(t, X ′,L(X ′|Ft))(X −X ′)]

= E

[

1AE

[

(∂xf1(t, X,L(X|Ft))− ∂xf

1(t, X ′,L(X ′|Ft))(X −X ′)|Ft

]]

≥ 0

E

[

1A(∂xg(X,L(X|Ft))− ∂xg(X′,L(X ′|Ft))(X −X ′)

]

= E

[

1AE

[

(∂xg(X,L(X|Ft))− ∂xg(X′,L(X ′|Ft))(X −X ′)|Ft

]]

≥ 0

(3.26)

for any X ∈ L2F , A ∈ Ft. The condition (H3) then follows by (3.24),(3.25),(3.26) with

β1 = cf , β2 = 0.

Combining with Theorem 2.2.8, we have the wellposedness result for ε-MFG with

common noise.

Corollary 3.1.7 (Wellposedness of ε-MFG). Under assumption A and B, there exist

a unique ε-MFG solution for any initial ξ0 ∈ L2F0.

Remarks on the functional framework

The existence and uniqueness proof of (3.2) is based on the standard techniques in

FBSDE involving Lipschitz and monotonicity condition. The essential difference is the

m-argument in the coefficients. However, working under the second order Wasserstein

metric, we have the inequality

W2(m,m′) ≤ E[(ξ − ξ′)2]

12

where ξ, ξ′ are square integrable random variables over an arbitrary probability space

with law m,m′. As a result, we have an estimate of the form

E[ϕ(ξ,L(ξ))− ϕ(ξ′,L(ξ′))] ≤ KE[(ξ − ξ′)2]12


whenever ϕ isK-Lipschitz in (x,m)-arguments. Using this estimate, we can deal with

the m-argument in the same way as we do with the x-argument. Under this general

framework, these two arguments are then grouped into the same argument in the

functional and their Lipschitz property are treated together. By brining the mono-

tonicity condition, which is widely used in the standard FBSDE, to this functional

framework, we arrive at the weak monotonicity condition.

However, it is worth noting that while this set of assumptions is rather general,

it does not include many interesting coefficient functions. A main example includes

local coupling function on m where the output depends on a local information of a

probability measure such that the density function. An example of such functions is

g(x,m) = ϕm(x)

where ϕm denotes the density function of m. See [14, 15, 54] for some works on the

MFG with local coupling.

3.2 Markov property and a decoupling function

Generally, to solve an FBSDE, we need to find what is called a decoupling field, a pos-

sibly random function describing the relation of the backward process Yt as a function

of the forward process Xt. When the coefficients are deterministic, this function is

deterministic and is a solution to a quasilinear PDE. In that case, the FBSDE is said

to be Markovian and we call the function a decoupling function. Using this function,

we can decouple an FBSDE and reduce the problem to solving a standard forward

SDE. This method of solving FBSDE is called Four-steps scheme and was first pro-

posed by Ma et al. in [46]. See [26] for more detail on a decoupling function of a

classical FBSDE in a deterministic case and [47] for a decoupling field in a general

case.

While we should expect a similar result in ε-MFG problem when the running

and terminal cost functions are deterministic, it is not obvious a priori that this

Markovian property holds particularly in the case of common noise. For a fixed


m ∈ M([0, T ];P2(R)), we are dealing with FBSDE with non-deterministic coefficients

from the fact that m is stochastic. Specifically, we have a path dependent functions

∂xH(t, x,mt(ω), y), g(x,mT (ω))

However, as f, g are still deterministic functions of m, it is reasonable to expect a

Markov property if we include an additional input, the current distribution of players,

or in FBSDE context, the conditional law ofXt. Our main result for this section is the

following theorem which shows the existence of a deterministic decoupling function

for ε-MFG thereby proving the Markov property of MFG with common noise.

Theorem 3.2.1. There exist a deterministic function Uε : [0, T ] × R × P2(R) such

that

Y εt = Uε(t, Xε

t ,L(Xεt |Ft)) (3.27)

Moreover, Uε satisfies the estimates


2. (Uε(t, x,m)− Uε(t, x′, m)) (x− x′) ≥ 0

for all t ∈ [0, T ], x, x′ ∈ R, m,m′ ∈ P2(R) where CK,T depends only on K, T .

We will state and prove a slightly more general result in Theorem 3.2.12 towards

the end of this section. For the estimates, see Lemma 3.2.11.

Remark 3.2.2. We will focus here only on Yt as it directly relates to the ε-MFG

solution in our model, but the existence of a decoupling function indeed holds for

Zt, Zt as well.

Remark 3.2.3. Similar to a classical FBSDE, it is natural to ask if Uε is a solution

to a certain PDE, and this question brings us to the connection between the SMP and

HJB approach in Section 3.3

As we shall see in Chapter 4, the decoupling function Uε and its PDE are essential

in our asymptotical analysis. The main idea for proving Theorem 3.2.1 is to define


Uε(t, x,m) through a solution of a certain FBSDE, then show that this solution is

well-defined and deterministic, and the resulting function satisfies the relation

Y εt = Uε(t, Xε

t ,L(Xεt |Ft)) (3.28)

where (Xεt , Y

εt , Z

εt , Z

εt )0≤t≤T is the solution to McKean-Vlasov FBSDE (2.2.8).

From now and throughout the rest of this section, we will operate under assump-

tion A,B. Under these assumptions, α := α(t, x, y) is independent of m and

∂xH(t, x,m, y) = ∂xf0(t, x, α(t, x, y)) + ∂xf

1(t, x,m)

We begin with the definition of the FBSDE corresponding to a subgame over [s, T ].

This FBSDE will be used for defining Uε.

Definition 3.2.4. Let s ∈ [0, T ], ξ ∈ L2Fs

and Gts≤t≤T be a sub-filtration of Fts≤t≤T .

We define FBSDE with data (s, ξ, Gts≤t≤T ) or simply FBSDE (s, ξ, Gts≤t≤T ) to

be the following FBSDE

dXt = α(t, Xt, Yt)dt+ σdWt + εdWt

dYt = −∂xH(t, Xt,L(Xt|Gt), Yt)dt+ ZtdWt + ZtdWt

Xs = ξ, YT = ∂xg(XT ,L(XT |GT ))

(3.29)

(Xt, Yt, Zt, Zt)s≤t≤T is called a solution to FBSDE (s, ξ, Gts≤t≤T ) if they are Ft-

adapted and satisfy FBSDE (3.29).

From this definition, the McKean-Vlasov FBSDE corresponding to the ε-MFG

with initial ξ ∈ L2F0

is simply FBSDE (0, ξ, Ft0≤t≤T ). Under assumption A,B,

existence and uniqueness of the solution to FBSDE above is guaranteed by Theorem

3.1.6.

Recall that F st is a σ-algebra generated by the common brownian motion start-

ing at time s, i.e. F st = σ

(

Wr − Ws; s ≤ r ≤ t)

. For ξ ∈ L2Fs, we denote by

(Xs,ξt , Y

s,ξt , Z

s,ξt , Z

s,ξt )s≤t≤T the unique solution to FBSDE (s, ξ, F s

t s≤t≤T ), i.e. they


satisfies

dXs,ξt = α(t, Xs,ξ

t , Ys,ξt )dt+ σdWt + εdWt

dYs,ξt = −∂xH(t, Xs,ξ,L(Xs,ξ|F s

t ), Ys,ξt ) + Z

s,ξt dWt + Z

s,ξt dWt

Xs,ξs = ξ, Y

s,ξT = ∂xg(X

s,ξT ,L(Xs,ξ

T |F sT ))

(3.30)

The FBSDE above corresponds to the ε-MFG over [s, T ] with initial distribution at

time s being L(ξ). We define, for 0 ≤ s ≤ t ≤ T , the following map

Θs,t :P2(R) → L2Fs

t(P2(R))

m→ L(Xs,ξt |F s

t )(3.31)

where ξ ∈ L2Fs

with L(ξ) = m. Intuitively, Θs,t(m) gives the unique distribution of

players’ state at time t corresponding to the ε-MFG solution over [s, T ] with initial

law at time s being m. We will sometimes use the following notation

ms,mt , Θs,t(m) (3.32)

First, we check that this map is well-defined, that is, L(Xs,ξt |F s

t ) is independent of the

choice of ξ ∈ L2Fs

provided that L(ξ) = m. This is equivalent to a conditional weak

uniqueness for FBSDE (3.30). This property is implied from a pathwise uniqueness

of FBSDE (3.30). See Proposition 3.2.6 for a more general statement.

Corollary 3.2.5. Suppose that ξ1, ξ2 ∈ L2Fs

such that L(ξ1) = L(ξ2) = m ∈ P2(R),

then L(Xs,ξ1t |F s

t ) = L(Xs,ξ2t |F s

t ) for all t ∈ [s, T ] where Xs,ξ1t , X

s,ξ2t are as defined

above.

For a slightly more general result, we consider the FBSDE (s, ξ, F rt s≤t≤T ) where

0 ≤ r ≤ s ≤ T , i.e.

dXt = α(t, Xt, Yt)dt+ σdWt + εdWt

dYt = −∂xH(t, Xt,L(Xt|Frt ), Yt)dt+ ZtdWt + ZtdWt

Xs = ξ, YT = ∂xg(XT ,L(XT |FrT ))

(3.33)


Observe that there is a subtle difference between (3.33) and (3.30) in the filtration

used in the equations. Now, we state a more general weak uniqueness result which

implies Corollary 3.2.5 above by setting r = s.

Proposition 3.2.6. Suppose that ξ1, ξ2 ∈ L2Fs

such that L(ξ1|F rs ) = L(ξ2|F r

s ) ∈

L2Fs(P2(R)), then L(Xs,ξ1

t |F rt ) = L(Xs,ξ2

t |F rt ) for all t ∈ [s, T ] where Xs,ξ1

t , Xs,ξ2t are

as defined above.

Proof. Fix a path of the common Brownian motion ω ∈ Ω, then follow the same

argument as in Theorem 5.1 in [6] which shows that pathwise uniqueness implies

weak uniqueness for an FBSDE.

Next, we show the following estimate

Proposition 3.2.7. For 0 ≤ s ≤ t ≤ T , m,m′ ∈ P2(R), there exists a constant CK,T

that depends only on K, T such that

E[

W2(Θs,t(m),Θs,t(m′))

]

≤ CK,TW2(m,m′) (3.34)

Proof. Let ξ, ξ′ be arbitrary elements of L2Fs

with law m,m′. Let (Xt, Yt, Zt, Zt)0≤t≤T

and (X ′t, Y

′t , Z

′t, Z

′t)0≤t≤T denote the solutions of FBSDE (2.18) with initial Xs =

ξ,X ′s = ξ′, then by the estimate (3.22), it follows that

E[W2(Θs,t(m),Θs,t(m′))] ≤ E[(Xt −X ′

t)2] ≤ CK,TE[(ξ − ξ′)2]

for a constant CK,T depends only on K, T . Since ξ, ξ′ are arbitrary, we conclude that

E[W2(Θs,t(m),Θs,t(m′))] ≤ CW2(m,m

′)

With Proposition 3.2.6 and 3.2.7, we can now proceed to show the following

property of Θ. The result below is essentially the Markov property of the stochastic

flow of probability measure corresponding to ε-MFG.


Theorem 3.2.8. For any m ∈ P2(R) and 0 ≤ s ≤ t ≤ u ≤ T

Θt,u(Θs,t(m)) = Θs,u(m) (3.35)

where Θ is as defined in (3.31).

Proof. The proof is based on the localization argument and a prior global Lipschitz

estimate (3.34). Given an arbitrary random variable ξ ∈ L2Fs

with lawm ∈ P2(R), the

values Θs,t(m) and Θs,u(m) are known immediately from its definition (see (3.31)).

However, since Θs,t(m) is random, Θt,u(Θs,t(m)) cannot be computed easily from its

definition. So we construct an approximaton of Θs,t(m) and use the estimate (3.34)

to bound the difference. We refer to Section 3.4.3 for the details of the proof.

Remark 3.2.9. Using notation (3.32), this is equivalent to ms,mu = m

t,ms,mt

u for any

m ∈ P2(R) and 0 ≤ s ≤ t ≤ u ≤ T .

Next, we let ξ ∈ L2Fs

and define (Xs,ξ,mt , Y

s,ξ,mt , Z

s,ξ,mt , Z

s,ξ,mt )s≤t≤T to be the Ft-

adapted solution to the following FBSDE

dXs,ξ,mt = α(t, Xs,ξ,m

t , Ys,ξ,mt )dt+ σdWt + εdWt

dYs,ξ,mt = −∂xH(t, Xs,ξ,m

t , ms,mt , Y

s,ξ,mt )dt+ Z

s,ξ,mt dWt + Z

s,ξ,mt dWt

Xs,ξ,ms = ξ, Y

s,ξ,mT = ∂xg(X

s,ξ,mT , m

s,mT )

(3.36)

Remark 3.2.10. The FBSDE (3.36) is a classical FBSDE with random coefficients

and not a McKean-Vlasov FBSDE since the stochastic law (ms,mt )0≤t≤T in the system

is given exogenously.

The existence and uniqueness of the FBSDE above is guaranteed under assumption

A by setting.

b(t, X, Y ) = α(t, X, Y )

F (t, X, Y ) = −∂xH(t, X,ms,mt , Y )

G(X) = ∂xg(X,ms,mT )

(3.37)

See Theorem 3.1.3. Note that the initial ξ here is arbitrary and does not necessarily

have law m. When ξ = x is a constant, (3.36) corresponds to the FBSDE from an


individual control problem over [s, T ] given (ms,mt )s≤t≤T with initial state Xs = x.

In that case, Y s,x,ms is deterministic since it is F s

s -measurable. This fact allows us to

define the following map

Uε :[0, T ]× R× P2(R) → R

(s, x,m) 7→ Y s,x,ms

(3.38)

To complete the proof of the Markov property of FBSDE (2.2.8), we are left to show

(3.27). We first state a necessary estimate for Uε.

Lemma 3.2.11. Let Uε : [0, T ]×R×P2(R) → R be as defined above, then it satisfies


2. (Uε(t, x,m)− Uε(t, x′, m)) (x− x′) ≥ 0

for all t ∈ [0, T ], x, x′ ∈ R, m,m′ ∈ P2(R), where CK,T depends only on K, T .

Proof. The first estimate directly follows from estimate (3.14) and (3.34). Next,

Let (Xt, Yt, Zt, Zt)s≤t≤T and (X ′t, Y

′t , Z

′t, Z

′t)s≤t≤T denote the solutions to the FBSDE

corresponding to the definition of Uε(s, x,m) and Uε(s, x′, m) respectively. Let ∆Xt =

Xt−X ′t and define similarly ∆Yt,∆∂xf,∆∂xg. Applying Ito’s lemma to ∆Xt∆Yt and

using convexity assumption (A3), particularly inequality (3.17), gives

E[∆Ys∆Xs] = E[∆∂xg∆XT ] + E

[∫ T

s

∆∂xft∆Xt −∆α∆Ytdt

]

≥ 0

By definition of Uε and the fact that it is deterministic, we deduce that

Uε(s, x,m)− Uε(s, x′, m)(x− x′) ≥ 0

Now we are ready to state and prove the existence of a deterministic decoupling

function thereby establishing the Markov result. Using Theorem 3.2.8 above, we can

show (3.27) using a similar argument as was done for a classical FBSDE (see Corollary

1.5 in [26] for instance).


Theorem 3.2.12. Let s ∈ [0, T ], m ∈ P2(R), ξ ∈ L2Fs, consider (ms,m

t )s≤t≤T and

(Xs,ξ,mt , Y

s,ξ,mt , Z

s,ξ,mt , Z

s,ξ,mt )s≤t≤T as defined above, then it follows that for all t ∈

[s, T ]

Ys,ξ,mt = Uε(t, Xs,ξ,m

t , ms,mt ) (3.39)

Remark 3.2.13. (3.27) in Theorem 3.2.1 follows from (3.39) by setting s = 0 and

L(ξ) = m.

Proof. We will use a similar argument as in the proof of Theorem 3.2.8 which is based

on the localization argument and global Lipschitz property. Note that R× P2(R) is

separable, hence there exist a countable disjoint set Ann∈N such that⋃∞

n=1An =

P2(R) and diag(An) < δ. Let (xn, mn) ∈ R × P2(R) be a fixed element of An, then

let

Bn = ω ∈ Ω; (Xs,ξ,mt , m

s,mt ) ∈ An (3.40)

Then by Lemma 3.2.11, we have

∑

n∈N

|Uε(t, Xs,ξ,mt , m

s,mt )− Uε(t, xn, mn)|

21Bn

≤ C1δ2 (3.41)

On the other hands, using (3.35), it follows that (Xs,ξ,mr , Y s,ξ,m

r , Zs,ξ,mr , Zs,ξ,m

r )t≤r≤T

satisfies the FBSDE

dXr = α(t, Xr, Yr)dr + σdWr + εdWr

dYr = −∂xH(r,Xr,Θt,r(ms,m

t ), Yr)dt+ ZrdWr + ZrdWr

Xt = Xs,ξ,mt , YT = gx(XT ,Θ

t,T (ms,mt ))

(3.42)

Thus, we get by (3.18), (3.34), and (3.40) that

∑

n∈N

E

[

(Y s,ξ,mt − Y

t,xn,mn

t )21Bn

]

≤ C2δ2 (3.43)

Combining (3.41) and (3.43), it follows that

E

[

(

Uε(t, Xs,ξ,mt , m

s,mt )− Y

s,ξ,mt

)2]

≤ C3δ2


Since δ is arbitrary, (3.39) holds as desired.

3.3 Master equation and connection to DPP ap-

proach

In this section, we state the connection between the decoupling function and the

generalized value function. Consequently, we obtain the master equation for Uε and

provide a verification-type theorem. We then discuss the connection to the solution

to FBSPDE of ε-MFG and FBPDE of 0-MFG.

One of the consequences of this Markov property is the fact that the ε-MFG

solution is in the feedback form. That is,

αεt = α(t, Xε

t , Yεt ) = α(t, Xε

t ,Uε(t, Xε

t ,L(Xεt |Ft))) (3.44)

With this fact, we can proceed as we did in Section 2.3 and define the value function

Vε for ε-MFG as a minimum expected cost given the current state and the current

distribution of players. We have shown that it satisfies the master equation


ε(t, x,m)) +σ2 + ε2

2∂xxV

ε(t, x,m)

+ E0[


ε(t, X,m)))]

+σ2

2∂mmV

ε(t, x,m)(X)[ζ, ζ ]

+ε2

2∂mmV

ε(t, x,m)(X)[1, 1] + ε2E0[


]

= 0

(3.45)



where X is a lifting random variable, i.e. L(X) = m, and ζ is a N (0, 1)-random

variable independent of X . Furthermore, in Theorem 2.3.1, we have established the


connection between the SMP and DPP approach through the relation

Y εt = ∂xV

ε(t, Xεt ,L(X

εt |Ft)) (3.46)

Now we are ready to restate this relation through a decoupling function Uε by com-

bining (3.46) and (3.44).

Corollary 3.3.1. Assume that A,B hold and suppose that Vε ∈ C1,3,3([0, T )× R ×

P2(R)) ∩ C([0, T ] × R × P2(R)) is a classical solution to the master equation (2.8),

then the following holds.

Uε(t, x,m) = ∂xVε(t, x,m) (3.47)

Using (3.47) and the master equation for Vε, we can now get the PDE for Uε. We

state the result through a verification theorem for Uε.

Theorem 3.3.2. Suppose that Uε : [0, T ]×R×P2(R) → R is a classical solution to

∂tUε(t, x,m) + ∂xH(t, x,m,Uε(t, x,m)) + ∂yH(t, x,m,Uε(t, x,m))∂xU

ε(t, x,m)

+σ2 + ε2

2∂xxU

ε(t, x,m)− E0[

∂mUε(t, x,m)(X)α(t, X,m,Uε(t, X,m))

]

+σ2

2∂mmU

ε(t, x,m)(X)[ζ, ζ ] +ε2

2∂mmU

ε(t, x,m)(X)[1, 1]

+ ε2E0[

∂xmUε(t, x,m)(X)1

]

= 0

(3.48)


Uε(T, x,m) = ∂xg(x,m)

where X is a lifting random variable, i.e. L(X) = m, and ζ is a N (0, 1)-random

variable independent of X. Suppose further that ∂xUε, ∂mUε, and Uε(t, 0, δ0) are

bounded then

(Xt, Yt, Zt, Zt)0≤t≤T =(

Xεt ,U

ε(t, Xεt , m

εt ), σ∂xU

ε(t, Xεt , m

εt ), ε

(

∂xUε(t, Xε

t , mεt ) + E

0[

∂mUε(t, Xε

t , mεt )(X

εt )]))

(3.49)


is a solution to FBSDE (2.18) satisfying

E

[

sup0≤t≤T

(Xεt )

2 + sup0≤t≤T

(Y εt )

2 +

∫ T

0

(Zεt )

2 +

∫ T

0

(Zεt )

2

]

<∞ (3.50)

where (Xεt )0≤t≤T is a solution to the SDE

dXt = α(

t, Xt,L(Xt|Ft),Uε(t, Xt,L(Xt|Ft))

)

dt+ σdWt + εdWt, X0 = ξ0

mεt = L(Xε

t |Ft), and (Xεt )0≤t≤T is a lifting random variable of (Xε

t )0≤t≤T sharing a

common noise space. As a result,

αεt , α(t, Xε

t ,L(Xεt |Ft), Y

εt ) = α(t, Xε

t ,L(Xεt |Ft),U

ε(t, Xεt ,L(X

εt |Ft)))

is a solution to ε-MFG.

Proof. The proof is identical to Theorem 2.3.1.

Similarly to Vε, equation (3.48) is a second order quasilinear PDE involving the

derivative with respect to a probability measure. We will refer to this PDE as the

master equation for Uε. The PDE (3.48) will be useful in our subsequent analysis

particularly in solving the linear variational FBSDE introduced in the next section.

From a similar result in the general FBSDE setting [26], the converse, which says

that Uε defined by (3.38) is a classical solution to (3.48), is expected to hold when

the coefficients are sufficiently regular. This is still open problem for a general MFG

with common noise and was recently shown for the no common noise case in [24].

Having established the relation between Uε and Vε, we can obtain in a similar

fashion the relation between the decoupling function Uε and the solution (uε, mε, vε)

of the FBSPDE

duε(t, x) =

(


σ2

2∂xxu

ε −ε2

2∂xxu

ε − ε2∂xvε

)

dt− εvεdWt

dmε(t, x) =

(


ε)mε) +σ2 + ε2

2∂xxm

ε

)

dt− ε∂xmε dWt

(3.51)


corresponding to ε-MFG as described in Section 2.1. Recall that the first equation

denotes the backward stochastic HJB equation for the value function of each players

given the flow of distribution (mεt )0≤t≤T . Relating to Vε, this simply means

uε(t, x) = Vε(t, x,m0t ),

The second equation is the forward Fokker-Planck equation describing the distribution

of players’ state given all the players adopt the strategy

αε(t, x,mεt ) = α(t, x,mε

t , ∂xVε(t, x,m0

t )) = α(t, x,mεt , ∂xu

ε(t, x))

From this and Ito-Kunita formula, we also have

vε(t, x) = E0[

∂mVε(t, x,mε

t )(X)1]

Combining with (3.47), it follows that

∂xuε(t, x) = Uε(t, x,mε

t ), ∂xvε(t, x) = E

0[

∂mUε(t, x,mε

t )(X)1]

(3.52)

We are particularly interested in the case ε = 0. In that case, we have the original

MFG model proposed by Lasry and Lions in [13] through the forward backward PDE

∂tu0 = −H(t, x,m0, ∂xu

0)−σ2

2∂xxu

0, u0(T, x) = g(x,m0T )

∂tm0 = −∂x(α(t, x,m

0, ∂xu0)m0) +

σ2

2∂xxm

0, m0(0, x) = m0(x) = L(ξ0)

(3.53)

where m0t (·) = m0(t, ·). The relation (3.52) then reads

U0(t, x,m0t ) = ∂xu

0(t, x) (3.54)

We would like to emphasize the relation (3.54) as the functions U0(t, x,m0t ), ∂xU

0(t, x,m0t ),

∂mU0(t, x,m0t )(·) are the main terms that will appear in the asymptotic analysis in

the next chapter. Relation (3.54) tells us that the first two terms can be found from


the system of PDEs describing 0-MFG. The last term, which represents the sensitiv-

ity of the solution around the optimal path (m0t )0≤t≤T , is new and we shall derive its

PDE in the next section.

We end this section with the summary of the relations between all the equations

related to ε-MFG. From the DPP approach, we have a generalized value function Vε

and a solution (uε, vε, mε) from FBSPDE (2.9), which reduces to FBPDE of Lasry and

Lion when ε = 0. From the SMP approach, we have a solution (Xεt , Y

εt , Z

εt , Zt)0≤t≤T

of McKean-Vlasov FBSDE (2.18) and its decoupling function Uε defined by (3.38).

We list all the relations below; for any (t, x,m) ∈ [0, T ]× R×P2(R),

mεt = L(Xε

t |Ft)

Y εt = Uε(t, Xε

t , mεt ) = ∂xV

ε(t, Xεt , m

εt ) = ∂xu

ε(t, Xεt )

Uε(t, x,mεt ) = ∂xV

ε(t, x,mεt ) = ∂xu

ε(t, x)

Uε(t, x,m) = ∂xVε(t, x,m)

E0[

∂mUε(t, x,mε

t )(X)1]

= E0[

∂xmVε(t, x,mε

t )(X)1]

= ∂xvε(t, x)

(3.55)

It is important to note that some of these relations only hold under a verification-

type argument. That is, we need to assume that there exists a classical solution Vε to

the master equation or a classical solution (uε, vε, mε) to the FBSPDE with certain

regularity assumption in order to justify these relations rigorously. However, these

existence results are not known in the case of common noise.

3.4 Proof of lemmas, propositions, and theorems

3.4.1 Proof of Theorem 3.1.1

Proof. Let s ∈ [0, T ], A ∈ Gs and for i = 1, 2, let (X it , Y

it , Z

it , Z

it)s≤t≤T denote a

solution to FBSDE (3.2) with data (ξi, bi, Fi, Gi). Let

∆Xt = X1t −X2

t


and define similarly ∆Yt,∆Zt,∆Zt,∆ξ,∆σ,∆ε. For the coefficients, we let

∆bt = b2(t, X1t , Y

1t )− b2(t, X

2t , Y

2t ), b(·) = b1(·)− b2(·)

and defined similarly ∆Ft, F and ∆G, G. Lastly, we write

θ1(t) = (t, X1t , Y

1t )

By applying Ito’s lemma on ∆Xt∆Yt and using the monotonicity condition (H3), we

have

E[1A∆Xs∆Ys] = E[1A∆XT (∆G+ G(X1T ))]− E

[

1A

∫ T

s

(∆Ft∆Xt +∆Yt∆bt)dt

]

− E

[

1A

∫ T

s

(F (θ1t )∆Xt + b(θ1t )∆Yt +∆σ∆Zt +∆ε∆Zt)dt

]

≥ E[1A∆XT G(X1T )] + β1E

[

1A

∫ T

s

(∆bt)2dt

]

+ β2E

[∫ T

s

1A(∆Yt)2)dt

]

− E

[

1A

∫ T

s

(F (θ1t )∆Xt + b(θ1t )∆Yt +∆σ∆Zt +∆ε∆Zt)dt

]

(3.56)

Thus, we have the estimate

β1E

[

1A

∫ T

s

(∆bt)2dt

]

+ β2E

[∫ T

s

1A(∆Yt)2dt

]

≤ E

[

1A

∫ T

s

(|F (θ1t )∆Xt|+ |b(θ1t )∆Yt|+ |∆σ∆Zt|+ |∆ε∆Zt|)dt

]

+ E[

1A(|∆ξ∆Ys|+ |∆XT G(X1T )|)

]

(3.57)

We will deal with the case β1 > 0 or β2 > 0 separately. For notational convenience,

we will use the same notation CK or CK,T for all the constants that depend only on

K or on K and T while they might be different from line to line.


Case I: β2 > 0. By Ito’s lemma on ∆X2t , Young’s inequality, and Lipschitz prop-

erty of b2, we have

E[1A∆X2t ] ≤ E[1A∆ξ

2] + E

[

1A

∫ t

s

2|∆Xu(∆bu + b(θ1u))|+∆σ2 +∆ε2du

]

≤ E[1A∆ξ2] + CKE

[

1A

∫ t

s

(∆Xu)2 + (∆Yu)

2 + b(θ1u)2 +∆σ2 +∆ε2du

]

(3.58)

By Gronwall lemma, it follows that

E[1A∆X2t ] ≤ CK,TE

[

1A(∆ξ2 +∆σ2 +∆ε2) + 1A

∫ t

s

(∆Yu)2 + b(θ1u)

2du

]

(3.59)

Next, we apply Ito’s lemma on ∆X2t again, then use Doob’s martingale inequality

and (3.59), we have

E[1A sups≤t≤T

∆X2t ] ≤ CK,TE

[

1A(∆ξ2 +∆σ2 +∆ε2) + 1A

∫ T

s

(∆Xt)2 + (∆Yt)

2 + b(θ1t )2dt

]

≤ CK,TE

[

1A(∆ξ2 +∆σ2 +∆ε2) + 1A

∫ T

s

(∆Yt)2 + b(θ1t )

2dt

]

(3.60)

Similarly, applying Ito’s lemma on ∆Y 2t , using Lipschitz property, we arrive at an

estimate

E

[

1A∆Y2t + 1A

∫ T

t

(∆Zu)2 + (∆Zu)

2du

]

≤ CK,TE

[

1A((∆XT )2 + G(X1

T )2) + 1A

∫ T

t

(∆Xu)2 + (∆Yu)

2 + F (θ1u)2du]

(3.61)

To get a bound on the sup norm, we first use Burkholder-Davis-Gundy (see Theorem


3.28 in [38]) and Young’s inequality to bound the martingale term.

E[1A sups≤t≤T

∫ T

t

∆Yu∆ZudWu] ≤ CE

[

1A

(∫ T

s

(∆Yt∆Zt)2dt

)

12

]

≤ CE

[

1A

(

sups≤t≤T

(∆Yt)2

∫ T

s

(∆Zt)2dt

)

12

]

≤ δE

[

1A sups≤t≤T

(∆Yt)2

]

+ CδE

[

1A

∫ T

s

(∆Zt)2dt

]

(3.62)

Using this fact, we can go back to Ito’s lemma on ∆Y 2t , take supremum, and use (3.62),

(3.61), (3.60) and select sufficiently small δ > 0 (independent of all the constants) to

get an estimate

E

[

1A sups≤t≤T

∆Y 2t + 1A

∫ T

s

(∆Zu)2 + (∆Zu)

2du

]

≤ CK,TE

[

1A

(

∆ξ2 +∆σ2 +∆ε2 + G(X1T )

2)

+ 1A

∫ T

s

(∆Yt)2 + b(θ1t )

2 + F (θ1t )2dt]

(3.63)

Combining with (3.60), and using (3.57), we then have an estimate

E

[

1A sups≤t≤T

∆X2t + sup

s≤t≤T∆Y 2

t + 1A

∫ T

s

(∆Zu)2 + (∆Zu)

2du

]

≤ CK,TE

[

1A

(

∆ξ2 +∆σ2 +∆ε2 + G(X1T )

2)

+ 1A

∫ T

s

(∆Yt)2 + b(θ1t )

2 + F (θ1t )2dt]

≤ CK,T

(

E

[

1A

(

∆ξ2 +∆σ2 +∆ε2 + G(X1T )

2)

+ 1A

∫ T

s

b(θ1t )2 + F (θ1t )

2dt

]

+ E

[

1A

∫ T

s

(|F (θ1t )∆Xt|+ |b(θ1t )∆Yt|+ |∆σ∆Zt|+ |∆ε∆Zt|)dt

]

+ E[

1A|∆ξ∆Ys|+ 1A|∆XT G(X1T )|]

)

(3.64)


Using Young’s inequality, we have

E

[

1A sups≤t≤T

∆X2t + sup

s≤t≤T∆Y 2

t + 1A

∫ T

s

(∆Zu)2 + (∆Zu)

2du

]

≤ CK,TE

[

1A

(

∆ξ2 +∆σ2 +∆ε2 + G(X1T )

2)

+ 1A

∫ T

s

b(θ1t )2 + F (θ1t )

2dt]

(3.65)

as desired.

Case II: β1 > 0 This case is fairly similar to the first case except that we keep ∆bt

on the RHS instead of ∆Yt so we can apply (3.57) in the end. Similar to (3.60), we

get the bound

E[1A sups≤t≤T

∆X2t ] ≤ CK,TE

[

1A(∆ξ2 +∆σ2 +∆ε2) + 1A

∫ T

s

(∆bt)2 + b(θ1t )

2dt

]

(3.66)

For the bound on ∆Y 2t , we proceed in the same way as we did in (3.61), but we apply

Gronwall lemma to get an estimate of the form

E

[

1A∆Y2t + 1A

∫ T

t

(∆Zu)2 + (∆Zu)

2du

]

≤ CK,TE

[

1A((∆XT )2 + G(X1

T )2) + 1A

∫ T

t

(∆Xu)2 + F (θ1u)

2du]

(3.67)

Combining with (3.66), we get

E

[

1A∆Y2t + 1A

∫ T

t

(∆Zu)2 + (∆Zu)

2du

]

≤ CK,TE

[

1A(∆ξ2 +∆σ2 +∆ε2 + G(X1

T )2) + 1A

∫ T

t

(∆bu)2 + b(θ1u)

2 + F (θ1u)2du

]

(3.68)

The next step is to get a bound on the sup-norm of the backward process. Similar

to (3.62) and (3.63), we apply Ito’s lemma on ∆Y 2t , take the sup-norm, use BDG


inequality on the martingale term, and use (3.68) to arrive at the estimate

E

[

1A sups≤t≤T

∆Y 2t + 1A

∫ T

s

(∆Zt)2 + (∆Zt)

2dt

]

≤ CK,TE

[

1A(∆ξ2 +∆σ2 +∆ε2 + G(X1

T )2) + 1A

∫ T

s

(∆bt)2 + b(θ1t )

2 + F (θ1t )2dt

]

(3.69)

Then we combine with (3.66), use (3.57), and apply Young’s inequality in the same

way at the first case to get

E

[

1A sups≤t≤T

∆X2t + 1A sup

s≤t≤T∆Y 2

t + 1A

∫ T

s

(∆Zt)2 + (∆Zt)

2dt

]

≤ CK,TE

[

1A(∆ξ2 +∆σ2 +∆ε2 + G(X1

T )2) + 1A

∫ T

s

b(θ1t )2 + F (θ1t )

2dt

]

(3.70)

as desired.


First, we show that FBSDE (3.2) is solvable over small time interval and that the

length of the interval depends only on the Lipschitz constant K of the functionals

b, F,G. Let τ ∈ [0, T ) and ξ ∈ L2Fτ, then we let A = X ∈ H2([τ, T ];R);Xτ = ξ be

a metric space with its metric induced by the sup norm. We define the map

Φτ,ξ : A → A

(Xt)τ≤t≤T → (Xt)τ≤t≤T

as follows; given X = (Xt)τ≤t≤T ∈ H2([τ, T ];R) with initial Xτ = ξ, we solve the

following BSDE

dYt = F (t, Xt, Yt)dt+ ZtdWt + ZtdWt, YT = G(XT ) (3.71)


We then define Φτ,ξ(X) to be X = (Xt)τ≤t≤T where X is given by

dXt = b(t, Xt, Yt)dt+ σdWt + εdWt, Xτ = ξ

Then clearly the fixed point of this map gives a solution to FBSDE (3.2) over [t, T ]

with initial condition Xτ = ξ. First, we note that the BSDE in the definition above

is uniquely solvable under assumption (H1),(H2) by standard SDE argument. Even

though we are dealing with functionals, the result still applies in the same way since

its proof only relies on probabilistic estimates.

Now we show that when T − τ is sufficiently small, the map Φτ,ξ is indeed a con-

traction map and we have a fixed point as desired. Suppose X1, X2 ∈ H2([τ, T ];R)

and for i = 1, 2, let (Y it , Z

it , Z

it)τ≤t≤T denote the solution to the BSDE (3.71) corre-

sponding to X i. We let

∆Xt = X1t −X2

t

and define similarly ∆Yt,∆Zt,∆Zt,∆ξ,∆σ,∆ε. For the coefficients, we let

∆bt = b(t, X1t , Y

1t )− b(t, X2

t , Y2t )

and defined similarly ∆Ft,∆G. Lastly, we write

∆Φ = Φτ,ξ(X1)− Φτ,ξ(X2)

By applying Ito’s lemma on ∆Y 2t , using Lipschitz property of F,G and Gronwall

lemma, we arrive at an estimate

E[

1A∆Y2t

]

≤ CK,TE

[

(∆XT )2 +

∫ T

t

(∆Xu)2du

]

≤ CK,TE

[

supτ≤t≤T

(∆Xt)2

] (3.72)


Using this estimate, we have

E

[

supτ≤t≤T

(∆Xt)2

]

= E

[

supτ≤t≤T

(∫ t

τ

∆budu

)2]

≤ (T − τ)E

[∫ T

τ

(∆bt)2dt

]

≤ 2K(T − τ)E

[∫ T

τ

(∆Xt)2 + (∆Yt)

2dt

]

≤ CK,T (T − τ)E

[

supτ≤t≤T

(∆Xt)2

]

for some constant CK,T depends only on K, T . Let γ =C−1

K,T

2, then for any τ ∈ [0, T )

such that T − τ ≤ γ, we have a contraction map and hence a fixed point for Φτ,ξ as

desired.

Next, we attempt to extend the solution over to [0, T ]. Let Γ denote the set of all

t ∈ [0, T ] such that the following holds: For any ξ ∈ L2Ft, the FBSDE (3.2) over [t, T ]

with initial ξ is solvable. Let τm = inf Γ, thus τm ≤ T − γ. Let δ > 0 be arbitrary

but sufficiently small so that τm + δ < T − γ2, and let τ ∈ Γ ∩ [τm, τm + δ]. Note that

T − τ ≥ γ2. Since τ ∈ Γ, for any ξ ∈ L2

Fτ, the FBSDE (3.2) is solvable over [τ , T ] and

we denote its solution by (Xξt , Y

ξt , Z

ξt , Z

ξt )τ≤t≤T .

Now we define G : L2Fτ

→ L2Fτ

by G(ξ) = Yξτ . Our goal is to show that G satisfies

(3.5) and (3.7) with G replaced by G and the constant denoted by K depends only

on K, T .

Let ξ1, ξ2 ∈ L2Fτ

and, for i = 1, 2, let (X it , Y

it , Z

it , Z

it)τ≤t≤T denote the solution to

FBSDE (3.2) corresponding to the initial X iτ = ξi. Let ∆Xt = X1

t − X2t and define

similarly ∆Yt,∆Zt,∆Zt,∆G,∆Ft,∆bt. Applying Ito’s lemma on ∆Xt∆Yt and using

(3.7) yield

E[1A∆Xs∆Ys] = E[1A∆XT∆G]− E

[

1A

∫ T

s

(∆Ft∆Xt +∆Yt∆bt)dt

]

≥ β1E

[

1A

∫ T

s

(∆bt)2dt

]

+ β2E

[∫ T

s

1A(∆Yt)2)dt

]

≥ 0


That is,

E[1A(G(ξ1)− G(ξ2))(ξ1 − ξ2)] ≥ 0 (3.73)

In addition, from (3.10) in Theorem 3.1.1, we have

E[(G(ξ1)− G(ξ2))2] ≤ KE[(ξ1 − ξ2)

2] (3.74)

where K depends only on K, T and is independent of τ . WLOG, we assume that

K ≥ K, otherwise we will take K to be the maximum of K,K instead.

Having shown (3.73) and (3.74), we can use the same argument as before to show

that there exist γ > 0 sufficiently small depending only on K such that FBSDE (3.2)

is solvable over [τ − γ, τ ] for any initial ξ ∈ L2Fτ−γ

and terminal functional G. By

definition of G, we can patch up a solution from [τ − γ, τ ] and [τ , T ] to get a solution

over [τ−γ, T ] for any arbitrary initial stateXτ−γ = ξ ∈ L2Fτ−γ

and terminal functional

G. As a result, we have shown that τ − γ ∈ Γ.

Note that γ is independent of δ. Thus, if τm > 0, then we can select δ sufficiently

small so that τ − γ < τm yielding a contradiction. Therefore, τm = 0 and, in that

case, we can select δ so that τ − γ = τm = 0. That is, 0 ∈ Γ so FBSDE (3.2) is

solvable over [0, T ].

Next, we show that the solution is in fact unique. For i = 1, 2, let (X it , Y

it , Z

it , Z

it)0≤t≤T

denote solutions to FBSDE (3.2) with the same initial ξ ∈ L2F0. Let ∆Xt = X1

t −

X2t and we define similarly ∆Yt,∆Zt,∆Zt,∆G,∆bt,∆Ft. Applying Ito’s lemma on

∆Xt∆Yt and using the fact that ∆X0 = 0 (due to the same initial ξ) and (H3), it

then follows that

β1E

[∫ T

s

(∆bt)2dt

]

+ β2E

[∫ T

s

(∆Yt)2)dt

]

≤ 0

In either cases (β1 > 0 or β2 > 0), it follows easily that ∆Xt = 0 for all t ∈ [0, T ] a.s.

and thus, so does ∆Yt,∆Zt,∆Zt.

The estimate (3.11) follows from Theorem 3.1.1 by taking the difference to the

zero solution (when all coefficients b, F,G are zero).



Proof. Let η ∈ L2Fs

with L(η) = m and (Xs,ηt , Y

s,ηt , Z

s,ηt , Z

s,ηt )s≤t≤T denote the solution

to FBSDE (3.36) corresponding to the definition of Θs,u, so

Θs,t(m) = L(Xs,ηt |F s

t ), Θs,u(m) = L(Xs,ηu |F s

u)

Since P2(R) is separable, for any δ > 0, there exist a sequence of disjoint Borel

measurable subsets Ann∈N of P2(R) such that diam(An) < δ and ∪n∈NAn = P2(R).

Let mn be a representative element of An so that W2(m,mn) < δ for all m ∈ An. Let

Bn = ω ∈ Ω;L(Xs,ηt |F s

t )(ω) ∈ An. Consider

ξ ,∑

n∈N

1Bnξn

where ξn ∈ L2Ft

has law mn and independent of Ft, thus independent of Bn. That is,

L(ξn|F st ) = L(ξn) = mn

Then it follows by construction that

W2(L(ξ|Fst ),L(X

s,ηt |F s

t )) < δ

Using this type of discretization and Lemma 1.1.6, we can redivide An further and

proceed sequentially to construct a sequence ξNN∈N of the form

ξN ,∑

n∈N

1Bn,Nξn,N

such that ξNN∈N is Cauchy in L2Ft, ξn,N is independent of F s

t and

W2(L(ξN |F s

t ),L(Xs,ηt |F s

t )) <1

N


Let ξ = limN→∞ ξN in L2Ft, then we have

L(ξ|F st ) = L(Xs,η

t |F st ) = Θs,t(m) (3.75)

Now consider FBSDE (t, ξ, F srt≤r≤T ) and denote its solution by (X t,ξ

r , Y t,ξr , Zt,ξ

r , Zt,ξr ).

By (3.75) and Theorem 3.2.6, it follows that

L(X t,ξu |F s

u) = L(Xs,ηu |F s

u) = Θs,u(m) (3.76)

Let

XNr ,

∑

n∈N

1Bn,NXn,N

r

where (Xn,Nr )t≤r≤u is a solution to FBSDE (t, ξn,N , F t

rt≤r≤T ). It is easy to check

that (XNr )t≤r≤u is a solution to FBSDE (t, ξN , F s

rt≤r≤T ) with initial ξN .

Note that Xn,Nr is F t

r-measurable which is independent of Ft, hence independent

of Bn. Thus, we have

L(XNu |F s

u) =∑

n∈N

1Bn,NL(Xn,N

u |F tu) =

∑

n∈N

1BnΘt,u(L(ξn,N))

Taking limit in L2Fu

as N → ∞ both sides, it follows from the fact that E[(ξ−ξN)2] →

0 and from estimate (3.10) that

L(X t,ξu |F s

u) = Θt,u(L(ξ|F st ))

Combine with (3.75) and (3.76), we get (3.35) as desired.

Chapter 4

Asymptotic analysis of mean field

games

In this chapter, we analyze ε-MFG model when ε is small. When the common noise

is small, it is reasonable to seek an approximate solution using the finite-dimensional

information from the 0-MFG. This chapter is organized as follows. In Section 4.1, we

consider the linear variational FBSDE of mean-field type describing the ε-first order

approximation term. We show that it is uniquely solvable and prove the convergence

result. In Section 4.2, we show that the approximate solution using the first order ap-

proximation is ε2-Nash equilibrium. In Section 4.3, we discuss the Gaussian property

of the solution and discuss the decoupling function. Lastly, in Section 4.4, we find

the explicit solution to this FBSDE and compute the covariance functions assuming

regularity of the decoupling function from 0-MFG.

To reduce notations, we will assume throughout this chapter that

f(t, x,m, α) =α2

2

while keeping a general cost function g. We would like to note that all the results in

this chapter still hold for a running cost function f satisfying A,B,C by doing the

same analysis on f as will be done on g. Working with this running cost, we now

97

CHAPTER 4. ASYMPTOTIC ANALYSIS OF MEAN FIELD GAMES 98

have

α(t, x,m, y) = −y, ∂xH(t, x,m, y) = 0 (4.1)

4.1 Linear variational FBSDE

In the previous chapters, we have shown that, under linear-convexity framework,

finding a solution of ε-MFG is equivalent to solving the corresponding McKean-Vlasov

FBSDE (2.2.8). We also show that this FBSDE is in fact uniquely solvable under

assumptions A, B. Let us denote the solution to ε-MFG by (Xεt , Y

εt , Z

εt , Z

εt )0≤t≤T .

Under 4.1, the McKean-Vlasov FBSDE reads

dXεt = −Y ε

t dt+ σdWt + εdWt

dY εt = Zε

t dWt + Zεt dWt

Xε0 = ξ, Y ε

T = ∂xg(XεT ,L(X

εT |FT ))

(4.2)

Solving this FBSDE yields the ε-MFG solution by setting

αεt = −Y ε

t

From the discussion in Section 3.3, we see that solving the 0-MFG problem for

(X0t , Y

0t , Z

0t , Z

0t )0≤t≤T requires us to find U0(t, x,m0

t ), which by (3.54) is reduced to

solving a system of PDEs. However, when adding common noise, we need to seek Uε,

a solution to the master equation (3.48). Instead of solving this infinite dimensinonal

equation, our goal here is to consider the approximation (Xεt , Y

εt , Z

εt , Z

εt )0≤t≤T around

(X0t , Y

0t , Z

0t , Z

0t )0≤t≤T when the common noise is small. Equivalently, we would like

to consider the limit as ε→ 0 of

Xεt −X0

t

ε,

Y εt − Y 0

t

ε(4.3)

As a result, we need additional regularity assumption on g. Thus, from now and

throughout the rest of this chapter, we will also assume C which is the following.

(C1) ∂xxg and ∂xmg exist and are continuous and bounded. Denote their bounds


by the same constant K.

We derive formally the FBSDE for the limit of (4.3) as ε → 0. First, we write

∆Xεt =

Xεt−X0

t

εand denote similarly ∆Y ε

t ,∆Zεt ,∆Z

εt , then they satisfy

d∆Xεt = −∆Y ε

t dt+ dWt

d∆Y εt = ∆ZtdWt +∆ZtdWt

∆Xε0 = 0, ∆Y ε

T =∂xg(X

εT ,L(X

εT |FT ))− ∂xg(X

0T ,L(X

0T |FT ))

ε

(4.4)

Formally taking ε→ 0, we get the following linear variational FBSDE

dUt = −Vtdt+ dWt

dVt = QtdWt + QtdWt

U0 = 0, VT = ∂xxg(X0T , m

0T )UT + E

0[∂xmg(X0T , m

0T )(X

0T )UT ]

(4.5)

where

m0t = L(X0

t |Ft) = L(X0t )

and X0 and U are identical copies ofX0 and U in (Ω, F , P) and E0[·] is the expectation

with respect to P0 only. We can write the terminal function explicitly as

E0[∂xmg(X

0T , m

0T )(X

0T )UT ] =

∫

Ω0

∂xmg(X0T (ω

0), m0T )(X

0T (ω

0))UT (ω0, ω)dP0(ω0)

where we suppress the ω in X0T , X

0T as they do not depend on it. We can see that the

term E0[∂xmg(X0T , m

0T )(X

0T )UT ] is a mean-field term that couples UT (ω

0, ·);ω0 ∈ Ω0

together.

We would like to note here that the description of each term as a function of path

is in the nature of SMP approach which describes the optimal control in an open-loop

form. Intuitively, each different path of Wt being coupled is simply a result of the

interaction between each players.

This FBSDE is called a linear variational FBSDE. Mathematically, it is simply a

linearization of the McKean-Vlasov FBSDE around the 0-MFG solution (X0t , Y

0t , Z

0t , Z

0t )0≤t≤T .

Notice that (4.5) is not in the standard form of McKean-Vlasov FBSDE as it involves


not just the law of Ut, but the joint law of (Ut, X0t ).

4.1.1 Wellposedness result

We first show that the linear variational FBSDE (4.5) is uniquely solvable. Derived

from FBSDE with monotone functional, it still possess the same monotonicity prop-

erty. As a result, the wellposedness theorem of FBSDE with monotone functional

(Theorem 3.1.2) can be applied in the same way. We will state the result for linear

FBSDE over [s, T ].

Theorem 4.1.1. Let (X0t , Y

0t , Z

0t , Z

0t )s≤t≤T denote the solution to McKean-Vlasov

FBSDE (2.18) corresponding to 0-MFG. Assume A,B,C hold. There exists a unique

adapted solution (Ut, Vt, Qt, Qt)s≤t≤T to FBSDE

dUt = −Vtdt+ dWt

dVt = QtdWt + QtdWt

Us = 0, VT = ∂xxg(X0T , m

0T )UT + E

0[∂xmg(X0T , m

0T )(X

0T )UT ]

(4.6)

satisfying

E

[

sups≤t≤T

[U2t + V 2

t ] +

∫ T

s

[Q2t + Q2

t ]dt

]

≤ CK,T (4.7)

Proof. We will apply Theorem 3.1.2. Define G : L2FT

→ L2FT

by

G(ξ) = ∂xxg(X0T , m

0T )ξ + E

0[

∂xmg(X0T , m

0T )(X

0T )ξ]

where X0T , ξ are identitcal copies of X

0T , ξ in the space (Ω, F , P), E0 is the expectation

with respect to Ω0 only (or equivalent conditional on FT ). By assumption (C1), there

exist a constant C depends on K such that

E[

1A(G(ξ)−G(ξ′))2]

≤ 2KE[1A(ξ − ξ)2]


for any ξ, ξ′ ∈ L2FT

, A ∈ FT . From the monotonicity condition (B3),

1

δ2E

[

∂xg(X0T + δξ,L(X0

T + δξ|FT ))− ∂xg(X0T ,L(X

0T |FT )))ξ

∣

∣

∣FT

]

≥ 0

Since ∂xxg and ∂xm are bounded by assumption (C1), we can take δ → 0 which yields,

for any ξ ∈ L2FT

,

E[G(ξ)ξ|FT ] ≥ 0

Since G is linear, we have

E[(G(ξ)−G(ξ′))(ξ − ξ′)|FT ] ≥ 0

or equivalently,

E[1A(G(ξ)−G(ξ′))(ξ − ξ′)] ≥ 0

or any ξ, ξ′ ∈ L2FT

, A ∈ FT . Having shown the Lipschitz and monotonicity property

of G, we can apply Theorem 3.1.2 as desired.

4.1.2 Convergence result

We are now ready to state our first main result of this chapter which justifies the

formal limit taken above. As a result, we characterize the ε-first order expansion

terms of ε-MFG solution as the solution to linear variational mean-field type FBSDE

(4.5).

Theorem 4.1.2. Assume A,B,C hold, for all ε > 0, let (Xεt , Y

εt , Z

εt , Z

εt )s≤t≤T denote

the solution to McKean-Vlasov FBSDE (4.2) corresponding to ε-MFG and (Ut, Vt, Qt, Qt)0≤t≤T

denote the solution to (4.6), then there exist a constant CK,T dependent only on K, T

such that

E sups≤t≤T

[

(

Xεt −X0

t

ε− Ut

)2

+

(

Y εt − Y 0

t

ε− Vt

)2]

≤ CK,Tε2 (4.8)

Proof. The proof is based on the estimates (3.11) and (3.10) with the bounded second

derivative of the cost function g. See Section 4.5.1 for the proof.


4.2 Approximate Nash equilibrium

In the previous section, we have shown that

αεt − α0

t

ε=

−Y εt + Y 0

t

ε→ −Vt as ε → 0

where the limit is in H2([0, T ];R). Using this result, we can construct the first order

approximate strategy by

βεt , α0

t − εVt (4.9)

for all t ∈ [0, T ]. We would like to see if (βεt )0≤t≤T serves as a good approximation.

Being a game, an appropriate notion of approximation is required. In this case,

it is reasonable to assume that each player adopts this approximate solution, and

analyze the gap between the expected cost under this set of strategies and that the

optimal cost each player could have obtained given that others adopt this approximate

strategy. For an exact Nash equilibrium, this gap is precisely zero by definition as

every player is optimal given other players’ strategy. This notion of approximate

optimality is called δ-Nash equilibrium. In the finite player game, it is defined as

follows.

Definition 4.2.1. Under the same notations as defined in Section 1.1, for the N-

player game, a set of admissible strategies (αit)0≤t≤T,1≤i≤N is called a δ-Nash equilib-

rium if for each i = 1, 2, . . . , N ,

J i(

αi|(αj)j 6=i

)

≤ J i(

β|(αj)j 6=i

)

+ δ

for all β = (βt)0≤t≤T ∈ H2([0, T ];R) where J i(·) denote the cost function of player i.

To go from a finite-player symmetric game to a continuum limit version, we for-

mally take N → ∞, assume that each player adopts the same strategy, and use a

single player as a representative player. As a result, we can define an approximate

Nash equilibrium similarly for MFG as follows;

Definition 4.2.2. Under the same notations as defined in Section 1.1, an admissible

strategy α = (αt)0≤t≤T ∈ H2([0, T ];R) is called a δ-Nash equilibrium for ε-MFG


problem if

J ε(α|mα) ≤ J ε(β|mα) + δ

for all β = (βt)0≤t≤T where J ε(·) denotes the cost function and mαt denotes the

conditional law of Xαt with (Xα

t )0≤t≤T being the state process corresponding to α.

Remark 4.2.3. By definition, an ε-MFG solution is a 0-Nash equilibrium for ε-MFG

problem.

Remark 4.2.4. It is conventionally called ε-Nash equilibrium. We use parameter δ

here to avoid confusion with parameter ε denoting the level of common noise.

The notion of approximate Nash equilibrium is important in the theory of stochas-

tic games with infinite horizon. In many problems, there is no exact Nash equilibrium

while there exists a δ-Nash equilibrium for any δ > 0. It is also a widely used notion in

an area called algorithmic game theory. In algorithmic game theory, we are interested

in finding polynomial time algorithms that yield an approximate Nash equilibrium

solution for problems where finding exact Nash equilibrium is computationally ex-

pensive.

In MFG, this notion is used mainly in the study of the relation between an MFG

and a symmetric N -player stochastic differential game. Recall that the motivation

for considering an MFG model is in its application for finding a good approximate

strategy for an N -player game when N is large. In [17], Carmona and Delarue showed,

under a linear-convexity MFG model without common noise, the 0-MFG strategy is

εN -Nash equilibrium for the corresponding N -player game where εN ∼ O(N−1/(d+4))

where d is the dimension of the underlying Euclidean space. See also [22, 37, 39] for

other similar results. The converse, which asks whether the Nash equilibrium from

N -player game converges to a corresponding MFG solution, is also of interest and is

more challenging. For interested readers, we refer to [27] and reference therein for

results in this direction all of which are for MFG models without common noise.

In this work, we are only concerned with the model at the continuum limit. We are

particularly interested in an approximate solution for ε-MFG using the information

from 0-MFG solution. Our main result for this section is the following theorem


Theorem 4.2.5. Assume A,B,C hold. For ε > 0, let αε = (αεt )0≤t≤T denote the

solution to the ε-MFG and (Ut, Vt, Qt, Qt)0≤t≤T denote the solution to the linear vari-

ation FBSDE (4.5). Define a first order approximate strategy βε = (βεt )0≤t≤T by

βεt , α0

t − εVt (4.10)

Then βε is an ε2-Nash equilibrium for ε-MFG as defined in 4.2.2.

Proof. See Section 4.5.2

4.3 Gaussian property and decoupling function

Having characterized the first order approximation of ε-MFG solution as the solution

of a linear variational FBSDE of mean-field type, we now proceed to analyze some

properties of the solution (Ut, Vt)0≤t≤T .

4.3.1 Centered Gaussian process

While the FBSDE (4.5) describing them seems complicated as it involves both the

individual noise and common noise, this is simply in the nature of SMP approach

as it describes the control in the open-loop form (a function of path) instead of the

closed-loop feedback form (a function of state). However, if we only analyze the effect

from the common noise, or equivalently if we fix a path ω ∈ Ω0 of (Wt)0≤t≤T , hence of

(X0t )0≤t≤T , and look at the distribution of (Ut(ω, ·), Vt(ω, ·))0≤t≤T with respect to the

common noise ω ∈ Ω only, then (Ut(ω, ·), Vt(ω, ·))0≤t≤T are simply centered Gaussian

processes.

Theorem 4.3.1. Let (Ut, Vt, Qt, Qt)0≤t≤T denote the solution to FBSDE (4.5). Then

for all ω ∈ Ω0, (Ut(ω, ·), Vt(ω, ·))0≤t≤T are Gaussian processes in (Ω, Ft0≤t≤T , P)

with mean zero.

Proof. The Gaussian property is from the linearity of FBSDE, while the mean zero

is proved by taking the conditional expectation and show that the resulting system

has a unique solution which is zero. See Section 4.5.3 for the proof.


Having shown that (Ut, Vt)0≤t≤T are Gaussian processes with mean zero, we would

like to compute its covariance functions. In a classical linear SDE, we can achieve

this by using Ito’s lemma on the SDE to derive a deterministic equation describing

the covariance function. However, when dealing with FBSDE, even though it is a

linear system, to describe the covariance structure of (4.5) will require the decoupling

function.

4.3.2 Decoupling function

Despite being a linear FBSDE, the mean field term E0[

∂xmg(X0T , m

0T )(X

0T )UT

]

in the

terminal condition makes solving this FBSDE explicitly a nontrivial task. However,

we proceed in the similar way as solving a classical FBSDE by attempting to find a

decoupling function describing the relation between Vt and Ut. To do so, recall that

we have a decoupling function Uε which satisfies the relation

Y εt = Uε(t, Xε

t ,L(Xεt |Ft))

Therefore, we have

Vt = limε→0

Y εt − Y 0

t

ε

= limε→0

Uε(t, Xεt ,L(X

εt |Ft))− U0(t, X0

t ,L(X0t ))

ε

= limε→0

Uε(t, Xεt ,L(X

εt |Ft))− U0(t, Xε

t ,L(Xεt |Ft)) + U0(t, Xε

t ,L(Xεt |Ft))− U0(t, X0

t ,L(X0t ))

ε

= limε→0

Uε(t, Xεt ,L(X

εt |Ft))− U0(t, Xε

t ,L(Xεt |Ft))

ε+ lim

ε→0

U0(t, Xεt ,L(X

εt |Ft))− U0(t, X0

t ,L(X0t ))

ε

where the limit is in L2F . The following proposition shows that the first part is in fact

zero.

Theorem 4.3.2. Let Uε denote the decoupling function of FBSDE (2.18) as defined

in (3.27), then the following holds;

limε→0

Uε(t, x,m)− U0(t, x,m)

ε= 0 (4.11)


uniformly in (t, x,m) ∈ [0, T ]× R× P2(R).

Proof. See Section 4.5.4

The interpretation of (4.11) is the following; to approximate the ε-MFG solu-

tion at the first order, we simply need to use 0-MFG solution along the trajectory

(t, Xεt ,L(X

εt |Ft)), i.e.

αεt = −Y ε

t = −Uε(t, Xεt ,L(X

εt |Ft)) ≈ −U0(t, Xε

t ,L(Xεt |Ft))

However, we would like to emphasize that we do not usually know U0(t, x,m) for all

(t, x,m) since that would require us to solve the master equation (2.8) or (3.48) which

is infinite-dimensional problem and is non-trivial to do so. On the other hand, we can

solve for U0(t, x,m) along the trajectory (m0t )0≤t≤T where m0

t = L(X0t ) corresponds

to the 0-MFG solution, since U0(t, x,m0t ) is simply the gradient of the solution of

FBPDE (2.10) of Lasry and Lions. So unless we know the function U0(t, x,m), this

process means that to get our optimal control at time t, we need to resolve 0-MFG

problem over [t, T ] with initial mεt = L(Xε

t |Ft). This method is very computationally

expensive, so we will instead approximate U0 at the current state (t, Xεt ,L(X

εt |Ft))

around (t, X0t , m

0t ).

In fact, it is not necessary to approximate at X0t if the current state Xε

t is observ-

able. In other words, making use of (4.11), we can get a slightly simpler approximation

of αεt as follows. Assuming that ∂mU0 exists and is bounded,

αεt = −Y ε

t = −Uε(t, Xεt ,L(X

εt |Ft))

= −U0(t, Xεt ,L(X

εt |Ft)) + o(ε)

= −U0(t, Xεt , m

0t ) + εE0[∂mU

0(t, X0t , m

0t )(X

0t )Ut] + o(ε)

(4.12)

From both (4.12) and (4.13) below, we see that in any case, the crucial term in our

approximation is E0[∂mU0(t, X0t , m

0t )(X

0t )Ut].

The master equation for MFG with common noise is rather difficult and most of

the discussions so far has been formal. There is a recent paper by Chassagneux et

al.[24] which shows the existence and uniqueness of a classical solution to the master


equation (3.48) when ε = 0, MFG without common noise, which implies that U0 is

continuously differentiable in (x,m). To apply this result, we need assumption D

which provides additional regularity on the cost functions involving second derivative

with respect to the m-argument. Combining with the discussion above, we have the

decoupling function for linear variational FBSDE (4.5) as follow.

Proposition 4.3.3. Assume that A,B,C,D holds. Let (Ut, Vt, Qt, Qt)0≤t≤T denote

the unique solution to FBSDE (4.5), then

Vt = ∂xU0(t, X0

t , m0t )Ut + E

0[∂mU0(t, X0

t , m0t )(X

0t )Ut] (4.13)

Proof. Under A,B,C,D, it follows from [24] (see specifically Theorem 5.3) that U0 is

continuously differentiable. From Theorem 4.1.2 and 4.3.2, we have that

Vt = limε→0

U0(t, Xεt ,L(X

εt |Ft))− U0(t, X0

t ,L(X0t ))

ε

Note that from Lemma 3.2.11, ∂xU0, ∂mU0 are bounded. Using Theorem 4.1.2, it

follows that

limε→0

U0(t, Xεt ,L(X

εt |Ft))− U0(t, X0

t ,L(X0t ))

ε= ∂xU

0(t, X0t , m

0t )Ut+E

0[∂mU0(t, X0

t , m0t )(X

0t )Ut]

where all the limits above are in L2F .

Plugging (4.13) back in and we have decoupled the FBSDE (4.5) and reduced it

to solving the following forward SDE

dUt = −[

∂xU0(t, X0

t , m0t )Ut + E

0[∂mU0(t, X0

t , m0t )(X

0t )Ut]

]

dt+ dWt, U0 = 0

(4.14)

Recall that (Ut)0≤t≤T is a copy of (Ut)0≤t≤T sharing the common noise space. We

would like to point out that this SDE is non-standard due to the mean field term

in the form of the conditional law of Ut. Even if we have explicitly the functional

∂mU0(t, x,m0t )(·), solving this linear mean-field SDE is non-trivial since it involves

the joint law of (Ut, X0t ).


Next, we proceed by letting γt, βs,t, ηt ∈ L2Ft, η : [0, T ]×R× Ω → R be defined as

γt , ∂xU0(t, X0

t , m0t )

βs,t , e−∫ t

sγrdr

η(t, x, ω) , E0[∂mU

0(t, x,m0t )(X

0t )Ut]

ηt , η(t, X0t , ω)

then from (4.13) and (4.14);

Ut = −

∫ t

0

βs,tηsds−

∫ t

0

βs,tdWs

Vt = γtUt + ηt

(4.15)

is a solution to FBSDE (4.5). The terms βs,t, γt can be obtained from a 0-MFG

solution straightforwardly, by solving the system of FBPDE (2.10) and using the

relation

U0(t, x,m0t ) = ∂xV

0(t, x,m0t ) = ∂xu

0(t, x)

We are now left to analyze a more non-trivial term, the random function η(t, x, ω)

which involves analyzing the derivative with respect to the m-argument of U0.

4.4 Explicit solution and covariance function

At this point, we have proved all the results using only assumptions A,B,C,D. To

proceed and further analyze the linear mean-field FBSDE (4.5), particularly the term

η(t, x), extra regularity assumptions on the decoupling function of 0-MFG U0 is re-

quired to derive the equations for η(t, x) and compute the covariance functions. While

there are some results that give sufficient conditions for the differentiability with re-

spect to (t, x) when m = m0t since it is related to u0(t, x) from Lasry and Lions’s

FBSDE, the only work that provides similar result for the derivative with respect to

m-argument for U0 is [24]. As mentioned above, they showed existence and unique-

ness of a classical solution to the master equation U0 when ε = 0, as a by product,


showing that U0 is continuously differentiable in m. However, their result is still not

sufficient for our application below. In this section, we will proceed formally assum-

ing that U0 is sufficiently regular with bounded derivatives. Our goal is to derive the

equation for η(t, x) and compute the covariance function of the process (Ut, Vt)0≤t≤T .

SPDE for η(t, x, ω)

Notice that η involves the law of Ut, and Ut depends on η in its dynamic (4.14). To

derive an equation for η, we need to resort to the master equation. Having shown

(4.8), we write

η(t, x, ω) = E0[∂mU

0(t, x,m0t )(X

0t )Ut] = lim

ε→0

U0(t, x,mεt (ω))− U0(t, x,m0

t )

ε

where mεt = L(Xε

t |Ft), m0t = L(X0

t ). Now we apply Ito’s lemma on each term and

use the master equation (3.48);

dU0(t, x,mεt ) =

[

∂tU0(t, x,mε

t )− E0[

∂mU0(t, x,mε

t )(Xεt )(U

ε(t, Xεt , m

εt ))]

+σ2

2∂mmU

0(t, x,mεt )(X

εt )[ζ, ζ ]

]

dt+ εE0[∂mU0(t, x,mε

t )]dWt

=

[

U0(t, x,mεt )∂xU

0(t, x,mεt )−

σ2

2∂xxU

0(t, x,mεt )

]

dt

+ εE0[

∂mU0(t, x,mε

t )(Xεt )]

dWt

and

dU0(t, x,m0t ) =

[

U0(t, x,m0t )∂xU

0(t, x,m0t )−

σ2

2∂xxU

0(t, x,m0t )

]

dt

Thus, taking the difference and ε→ 0, we arrive at the SPDE for η;

dη(t, x) =

[

η(t, x)∂xU0(t, x,m0

t ) + ∂xη(t, x)U0(t, x,m0

t )−σ2

2∂xxη(t, x)

]

dt

− E0[

∂mU0(t, x,m0

t )(X0t )]

dWt

(4.16)


Due to the fact that mε(0, x) = m0(0, x) = m0(x), we have the initial condition

η(0, x) = 0

Notice that the diffusion term in the dynamic of η is deterministic due to the absence

of common noise. We define w : [0, T ]× R → R by

w(t, x) , E0[

∂mU0(t, x,m0

t )(X0t )]

(4.17)

To fully describe the dynamic for η, we need to solve for a deterministic w(t, x). Note

that

w(t, x) = E0[

∂mU0(t, x,m0

t )(X0t ) · 1

]

= limε→0

U0(t, x,L(X0t + ε))− U0(t, x,L(X0

t ))

ε

That is, w(t, ·) measures the sensitivity of the solution of 0-MFG PDE system at time

t with respect to the spatial shift of the equilibrium distribution m0t . However, to

do so, we will need to compute ∂mU0(t, x,m0t )(·). We will refer to this term as the

sensitivity functional of 0-MFG and we shall derive its equation in the next section.

4.4.1 Sensitivity functional

Our goal in this section is to study ∂mU0(t, x,m0t ) or more specifically to compute

w(t, x) = E0[

∂mU0(t, x,m0t )(X

0t ) · 1

]

. This function represents the sensitivity of 0-

MFG solution with respect to a perturbation on m0t . As it turns out, the Gauteax

derivative of U0(t, x, ·) is more suitable for this calculation. See [10] for instance. In

this framework, we are working with the density function, so we denote the density

function of m0t by the same notation m0(t, x). The existence of a density function for

m0t is guaranteed for t > 0 given that σ > 0 (recall that σ is the diffusion term of the

individual noise). We assume that U0 is Gauteax differentiable, that is , there exist

a unique ∂mU0(t, x,m) ∈ L2(R;R) such that

limε→0

U0(t, x,m+ εψ)− U0(t, x,m)

ε=

∫

R

∂mU0(t, x,m)(z)ψ(z)dz


for all test function ψ ∈ C∞(R;R) with compact support and∫

Rψ(x)dx = 0. We

use the notation ∂m to avoid confusion with the notion of the derivative of the lifting

function. We are specifically interested in this functional along m = (m0t )0≤t≤T , so

we let h(t, x, z) = ∂mU0(t, x,m0t )(z). Also, we let 〈∂mU0(t, x,m0

t )(·), ψ〉 or simply

〈∂mU0, ψ〉 to be∫

R∂mU0(t, x,m)(z)ψ(z)dz. Using this notion, the master equation

for U0 reads

∂tU0(t, x,m)− U0(t, x,m)∂xU

0(t, x,m) +σ2

2∂xxU

0(t, x,m)

+

⟨

∂mU0(t, x,m), ∂x(U

0(t, x,m)m) +σ2

2∂xxm

⟩

= 0


U0(T, x,m) = ∂xg(x,m)

See [10] for the master equation for V0(t, x,m) and recall that U0 = ∂xV0. Using this

fact, we have

d

dtU0(t, x,m0

t + εψ)

= ∂tU0(t, x,m0

t + εψ) +

⟨

∂mU0,−∂z(−U0(t, z,m0

t )m0t ) +

σ2

2∂zzm

0t

⟩

= U0(t, x,m0t + εψ)∂xU

0(t, x,m0t + εψ)−

σ2

2∂xxU

0(t, x,m0t + εψ)

+⟨

∂mU0, ∂z(U

0(t, z,m0t )m

0t ) +

σ2

2∂zzm

0t − ∂z(U

0(t, z,m0t + εψ)(m0

t + εψ))

−σ2

2∂zz(m

0t + εψ)

⟩

(4.18)

andd

dtU0(t, x,m0

t ) = U0(t, x,m0t )∂xU

0(t, x,m0t )−

σ2

2∂xxU

0(t, x,m0t )


where we use the z-variable to denote the variable in the inner product. Therefore,

we have

d

dt

⟨

∂mU0, ψ⟩

=⟨

∂mU0, ψ⟩

∂xU0 +

⟨

∂xmU0, ψ⟩

U0 −σ2

2

⟨

∂xxmU0, ψ⟩

+

⟨

∂mU0,−∂z

(⟨

∂mU0(t, z,m0

t )(k), ψ(k)⟩

m0(t, z) + U0(t, z,m0t )ψ)

−σ2

2∂zzψ

⟩

(4.19)

If we write h(t, x, z) = ∂mU0(t, x,m0t )(z) as mentioned above and recall that U0(t, x,m0

t ) =

∂xu0(t, x) where u0(t, x) is the solution to FBPDE (2.10) of Lasry and Lions, then

the equation above reads

∂t

⟨

h(t, x, z), ψ(z)⟩

=⟨

h(t, x, z), ψ(z)⟩

∂xxu0(t, x) +

⟨

∂xh(t, x, z), ψ(z)⟩

∂xu0 −

σ2

2

⟨

∂xxh(t, x, z), ψ(z)⟩

+

⟨

∂zh(t, x, z),⟨

h(t, z, k), ψ(k)⟩

m0(t, z) + ∂zu0(t, z)ψ(z)−

σ2

2ψzz

⟩

(4.20)


⟨

h(T, x, z), ψ(z)⟩

=⟨

∂mxg(x,m0T )(z), ψ(z)

⟩

Note that the domain for this functional is only those test functions ψ such that∫

Rψ(x)dx = 0. We can then write an equation for h(t, x, z) by formally plugging in

ψ = δ′z which yields

∂tzh(t, x, z)

= ∂zh(t, x, z)∂xxu0(t, x) + ∂xzh(t, x, z)∂xu

0 −σ2

2∂xxzh(t, x, z)

+

∫

R

∂zh(t, x, u)∂zh(t, u, z)m0(t, u)du+ ∂z

(

∂zh(t, x, z)∂zu0(t, z)

)

−σ2

2∂zzzh(t, x, z),

h(T, x, z) = ∂mxg(x,m0T )(z)


Let h : [0, T ]× R× R be a weak-derivative of h with respect to z, then h(t, x, z) is a

weak solution to

∂th(t, x, z)

= h(t, x, z)∂xxu0(t, x) + ∂xh(t, x, z)∂xu

0 −σ2

2∂xxh(t, x, z)

+

∫

R

h(t, x, u)h(t, u, z)m0(t, u)du+ ∂z(h(t, x, z)∂zu0(t, z))−

σ2

2∂zzh(t, x, z)

h(T, x, z) = ∂z∂mxg(x,m0T )(z)

(4.21)

To find w(t, x), letmεt denote the right shift by ε ofm

0t , that is, m

ε(t, x−ε) = m0(t, x).

Recall the definition of w(t, x);

w(t, x) = limε→0

U0(t, x,mεt )− U0(t, x,m0

t )

ε

Assuming ∂mU0 is continuous in m and ∂xxm0 is bounded, then we have


t )

ε=

⟨

∂mU0(t, x,mλ∗

t ),mε

t −m0t

ε

⟩

for some λ∗ ∈ [0, 1] where mλt = m0

t + λ(mεt −m0

t ). Thus, we have


t )

ε=⟨

∂mU0(t, x,m0

t ,−∂zm0t

⟩

= −

∫

R

h(t, x, z)∂zm0(t, z)dz

=

∫

R

h(t, x, z)m0(t, z)dz

where h is a solution to (4.21)


4.4.2 Explicit solution

Having shown that the random function η satisfies a linear SPDE, we can solve (4.16)

by means of Duhamel’s Principle. Let P sf denote a solution to the problem;

∂tφ(t, x) = ∂xφ(t, x)U0(t, x,m0

t ) + φ(t, x)∂xU0(t, x,m0

t )−σ2

2∂xxφ(t, x), (t, x) ∈ [s, T ]× R

φ(s, x) = f(x), x ∈ R

(4.22)

Then, from the initial condition η(0, x) = 0, it follows that

η(t, x) = (P 0η(0, ·))(t, x) +

∫ t

0

(P sw(s, ·))(t, x)dWs =

∫ t

0

(P sw(s, ·))(t, x)dWs (4.23)

We let

ψ(s, t, x) , (P sw(s, ·))(t, x), 0 ≤ s ≤ t ≤ T, x ∈ R

Combining with (4.15), we have an explicit solution to FBSDE (4.5)

Ut = −

∫ t

0

∫ s

0

βs,tψ(r, s,X0s )dWrds−

∫ t

0

βs,tdWs

Vt = ∂xU0(t, X0

t , m0t )Ut +

∫ t

0

ψ(s, t, X0t )dWs

Equivalently, by relation (3.54), we can write these in terms of the solution (u0, m0)

of the system of PDE (2.10) from 0-MFG and the function ψ.

Ut = −

∫ t

0

∫ s

0

e−∫ t

s∂xxu0(k,X0

k)dkψ(r, s,X0

s )dWrds−

∫ t

0

e−∫ t

s∂xxu0(k,X0

k)dkdWs

Vt = ∂xxu0(t, X0

t )Ut +

∫ t

0

ψ(s, t, X0t )dWs

(4.24)

4.4.3 Covariance function

Having shown that the solution to (4.5) is a centered Gaussian process with respect to

the common noise, we now compute its covariance function making use of the explicit


solution we just derived. Recall our notations defined earlier;

γt , ∂xU0(t, X0

t , m0t )

βs,t , e−∫ t

sγrdr

η(t, x, ω) , E0[∂mU

0(t, x,m0t )(X

0t )Ut]

ηt , η(t, X0t , ω)

We define the following covariance functions

ϕUt , E

[

U2t

]

, ϕVt , E

[

V 2t

]

, ϕUVt , E [UtVt]

ϕUηt , E [Utηt] , ϕ

V ηt , E [Vtηt] , ϕ

ηt , E

[

η2t]

where E[·] is the expectation with respect to P only, i.e .with respect to the common

Brownian motion only. Observe that the full covariance functions can be derived

easily in terms of these functions. Using Ito’s lemma and the decoupling relation

(4.13), it is easy to check that

d

dtϕUt = −γtϕ

Ut − ϕ

Uηt + 1, ϕU

0 = 0

ϕUVt = γtϕ

Ut + ϕ

V ηt

ϕVt = γ2tϕ

Ut + 2γtϕ

Uηt + ϕ

ηt

ϕV ηt = γtϕ

Uηt + ϕ

ηt

(4.25)

From this system, we see that every term can be expressed in terms of ϕUηt and ϕη

t .

Using the explicit form of Ut, ηt in (4.15) and (4.23);

Ut = −

∫ t

0

βs,tηsds−

∫ t

0

βs,tdWs, ηt =

∫ t

0

ψ(s, t, X0t )dWs


we have

ϕUηt = −

∫ t

0

βs,tE[ηsηt]ds− E

[

ηt

∫ t

0

βs,tdWs

]

= −

∫ t

0

βs,t

∫ s

0

ψ(r, t, X0t )ψ(r, s,X

0s )drds−

∫ t

0

βs,tψ(s, t, X0t )ds

= −

∫ t

0

e−∫ t

s∂xxu0(k,X0

k)dk

∫ s

0

ψ(r, t, X0t )ψ(r, s,X

0s )drds−

∫ t

0

e−∫ t

s∂xxu0(k,X0

k)dkψ(s, t, X0

t )ds

and

ϕηt =

∫ t

0

(ψ(s, t, X0t ))

2ds

Thus, the covariance structure of (Ut(ω, ·), Vt(ω, ·))0≤t≤T can be fully described in

terms of ∂xxu0, where u0 is a solution from 0-MFG system (2.10), and ψ, a solution

to PDE (4.22).

4.5 Proof of lemmas, propositions, and theorems


Proof. Let ∆Xεt =

Xεt −X0

t

εand δX,ε

t = ∆Xεt −Ut and define similarly ∆Y ε

t , ∆Zεt , ∆Z

εt ,

δY,εt , δZ,εt , δZ,ε

t , then (δX,εt , δ

Y,εt , δ

Z,εt , δ

Z,εt )0≤t≤T satisfies

dδX,εt = −δY,εt dt, dδ

Y,εt = dδ

Z,εt dWt + δ

Z,εt dWt, δ

X,ε0 = 0

δY,εT =

∂xg(XεT ,L(X


0T , m

0T )

ε− ∂xxg(X

0T , m

0T )UT + E

0[

∂xmg(X0T , m

0T )(X

0T )UT

]

Let

Xλ,εt := X0

t + λ(Xεt −X0

t ), 0 ≤ λ ≤ 1


Note that

∂xg(XεT ,L(X


0T ,L(X

0T ))

ε− ∂xxg(X

0T ,L(X

0T ))UT + E

0[

∂xmg(X0T , m

0T )(X

0T )UT

]

=

∫ 1

0

(

∂xxg(Xλ,εT ,L(Xλ,ε

T |FT ))∆Xεt + E

0[

∂xmg(Xλ,εT ,L(Xλ,ε

T |FT ))(Xλ,εT )∆Xε

t

])

dλ

− ∂xxg(X0T , m

0T )UT + E

0[

∂xmg(X0T , m

0T )(X

0T )UT

]

=

[∫ 1

0


T |FT ))dλ

]

δX,εT +

∫ 1

0

E0[


T |FT ))(Xλ,εT )δX,ε

T

]

dλ

+

[∫ 1

0


T |Ft))dλ− ∂xxg(X0T ,L(X

0T ))

]

UT

+

∫ 1

0

E0[(


T |FT ))(Xλ,εT )− ∂xmg(X

0T ,L(X

0T ))(X

0T ))

UT

]

dλ

= Iε1(δX,εT ) + Iε2

where Iε1 : L2FT

→ L2FT

is a linear functional defined by

Iε1(ξ) =

[∫ 1

0


T |FT ))dλ

]

ξ+

∫ 1

0

E0[


T |FT ))(Xλ,εT )ξ

]

dλ

and Iε2 ∈ L2FT

is given by

Iε2 =

[∫ 1

0


T |Ft))dλ− ∂xxg(X0T ,L(X

0T ))

]

UT

+

∫ 1

0

E0[(


T |FT ))(Xλ,εT )− ∂xmg(X

0T ,L(X

0T ))(X

0T ))

UT

]

dλ

Because ∂xxg, ∂xmg are bounded and UT , UT are bounded in L2, it follows that Iε2 is

bounded in L2. Thus, we can apply estimate (3.11) by setting

G(X) := Iε1(X) + Iε2

to deduce that

E[ sup0≤t≤T

(δX,εt )2] ≤ CK,T


where CK,T depends only on K, T . In particular, it is independent of ε. As a result,

we get

E[ sup0≤t≤T0≤λ≤1

(Xλ,εt −X0

t )2] ≤ CK,Tε

2

Thus, there exist a constant CK,T depending only on K, T such that

E[(Iε2)2] ≤ CK,Tε

2

Thus, by estimate (3.11) again, we get (4.8) as desired.


Proof. Let αε = (αεt )0≤t≤T denote the ε-MFG solution, βε = (βε

t )0≤t≤T be the approx-

imate strategy defined by

βεt = α0

t − εVt

where V = (Vt)0≤t≤T is the backward process of the linear variational FBSDE (4.5).

For notational convenience, we will write J ε(α|β) to denote J ε(α|mβ) for any α, β ∈

H2([0, T ];R). Recall the definition of J ε(α|mβ) and mβ in Section 1.1.2.

For any control α, β(1), β(2) ∈ H2([0, T ];R), let Xα, Xβ(1), Xβ(2)

denote the corre-

sponding state processes, then we have

E[(Xβ(1)

T −Xβ(2)

T )2] ≤ CT

∫ T

0

|β(1)t − β

(2)t |2dt

Thus, combining with Lipschitz assumption on g, it follows that

|J ε(α|β(1))− J ε(α|β(2))| ≤ E

[

g(XαT ,L(X

β(1)

T |FT )− g(XαT ,L(X

β(2)

T |FT ))]

≤ K(E[(Xβ(1)

T −Xβ(2)

T )2])12

≤ CK,T

(∫ T

0

|β(1)t − β

(2)t |2dt

)

12

(4.26)

Also, since αε is the ε-MFG solution, we can use the estimate (2.15) in Theorem 2.2.4


to get

J ε(αε|αε) + C

∫ T

0

|αεt − αt|

2dt ≤ J ε(α|αε) (4.27)

for any α ∈ H2([0, T ];R). Lastly, from the definition of ε-MFG strategy, we have

J ε(αε|αε) ≤ J ε(α|αε) (4.28)

for any α ∈ H2([0, T ];R). Combining (4.26),(4.27), and (4.28) yields

J ε(βε|βε)− J ε(α|βε) ≤ J ε(βε|βε)−J ε(αε|αε) + J ε(α|αε)− J ε(α|βε)

= J ε(βε|βε)− J ε(βε|αε) + J ε(βε|αε)−J ε(αε|αε)

+ J ε(α|αε)− J ε(α|βε)

≤ C

[

(∫ T

0

|αεt − βε

t |2dt

)

12

+

∫ T

0

|αεt − βε

t |2dt

]

Using estimate (4.8) in Theorem 4.1.2,

(∫ T

0

|αεt − βε

t |2dt

)

12

= |ε|

∥

∥

∥

∥

αε − α0

ε− Vt

∥

∥

∥

∥

H2([0,T ];R)

≤ CK,Tε2

and we have

J ε(βε|βε)− J ε(α|βε) ≤ CK,Tε2

for any α ∈ H2([0, T ];R) and some constant CK,T as desired.


Proof. Since FBSDE (4.5) is linear, one can easily check that for any ω ∈ Ω0 and

ω1, ω2 ∈ Ω

(Ut(ω, ω1) + Ut(ω, ω2), Vt(ω, ω1) + Vt(ω, ω2))0≤t≤T


satisfy the FBSDE for a given path (ω, ω1 + ω2). Thus, by pathwise uniqueness of

this FBSDE, one get that

Ut(ω, ω1 + ω2) = Ut(ω, ω1) + Ut(ω, ω2)

Vt(ω, ω1 + ω2) = Vt(ω, ω1) + Vt(ω, ω2)

Thus, we can conclude that they are both Gaussian processes. Next, we find their

mean and covariance functions for each fixed ω ∈ Ω0. Let first denote the mean by

µU , µV , µQ : [0, T ]× Ω0 → R. By direct calculation, i.e. taking expectation E[·] with

respect to P, they satisfies

dµUt = −µV

t dt

dµVt = µ

Qt dWt

µU0 = 0, µV

T = ∂xxg(X0T ,L(X

0T ))µ

UT + E

0[

∂xmg(X0T , m

0T )(X

0T )µ

UT

]

Applying Ito’s lemma on µUt µ

Vt , it follows that

µUTµ

VT − µU

0 µV0 =

∫

µVt µ

Qt dWt −

∫

(µVt )

2dt

Thus,

E

[∫

(µVt )

2dt

]

= −∂xxg(X0T ,L(X

0T ))(µ

UT )

2 − E0[

∂xmg(X0T , m

0T )(X

0T )µ

UT

]

µUT ≤ 0

Then it follows easily that µUt = µV

t = 0 for all t ∈ [0, T ] a.s.


Proof. Fix (s, x,m) ∈ [0, T ]×R×P2(R). Let (Xεt , Y

εt , Z

εt , Z

εt )0≤t≤T denote the solution

to FBSDE (3.30) corresponding to ε-MFG over [s, T ] with the initial distributionms =

m and let (Xεt , Y

εt , Z

εt ,

˜Zεt )s≤t≤T denote the solution to FBSDE (3.36) corresponding

to an individual problem given (ms,mt )s≤t≤T and the initial state Xs = x. Recall the


definition of ms,m,Uε, we have

ms,mt = L(Xε

t |Fst ), Y ε

s = Uε(s, x,m)

By Theorem 4.1.2, we know that

E sups≤t≤T

[

(

Xεt −X0

t

ε− Ut

)2

+

(

Y εt − Y 0

t

ε− Vt

)2]

≤ Cε2 (4.29)

where (Ut, Vt, Qt, Qt)0≤t≤T is the solution to FBSDE (4.6) which reads

dUt = −Vtdt+ dWt

dVt = QtdWt + QtdWt


0T )UT + E

0[∂xmg(X0T , m

0T )(X

0T )UT ]

(4.30)

By the same argument as in Theorem 4.1.2, we also get

E sups≤t≤T

[

(

Xεt − X0

t

ε− Ut

)2

+

(

Y εt − Y 0

t

ε− Vt

)2]

≤ Cε2 (4.31)

where C is a constant depends only onK, T and not on ε, s, x,m, and (Ut, Vt, Qt,˜Qt)s≤t≤T

is the linear variational process for FBSDE (3.36) satisfying

dUt = −Vtdt+ dWt

dVt = QtdWt +˜QtdWt


0T )UT + E

0[∂xmg(X0T , m

0T )(X

0T )UT ],

(4.32)

Notice a slight difference between the FBSDE (4.32) and (4.30) in the mean-field

term. The mean-field term in (4.32) is given exogenously by (Ut, Vt)0≤t≤T , or more

precisely its copy (Ut, Vt)0≤t≤T . Thus, (4.32) is an ordinary FBSDE with random

coefficients. On the other hands, the mean-field term in (4.30) is part of the solution,

so we are dealing with mean-field type FBSDE in this case.


From Theorem 4.3.1, we have

E[Ut] = E[Vt] = 0, for s ≤ t ≤ T (4.33)

where E[·] denote the expectation with respect to the common Brownian motion

(Wt)0≤t≤T , i.e. with respect to P, only. We claim that the same holds for (Ut, Vt)0≤t≤T .

We take expectation in FBSDE (4.32) to get that (At, Bt)0≤t≤T , (E[Ut], E[Vt])0≤t≤T

satisfiesdAt = −Btdt, dBt = CtdWt

As = 0, BT = ∂xxg(X0T , m

0T )AT

(4.34)

Note that zero is a solution to this FBSDE and from convexity assumption on g, this

FBSDE is in fact monotone, so we can apply Theorem 3.1.2 to conclude that it has

a unique solution. Therefore, we have

E[Ut] = E[Vt] = 0, for s ≤ t ≤ T (4.35)

Combining with (4.31) and the fact that Uε(s, x,m),U0(s, x,m) are deterministic, we

get, uniformly in (s, x,m),

limε→0

Uε(s, x,m)− U0(s, x,m)

ε= 0

Chapter 5

Linear-quadratic MFG with

common noise

In this chapter, we consider a linear-quadratic MFG (LQMFG for short) with common

noise. Similar to a classical stochastic control problem, LQMFG model is explicitly

solvable and we can illustrate all the concepts discussed in the previous chapters

through this model. We assume a linear state process is given by

dXt = αtdt+ σdWt + εWt, X0 = ξ0

and the running and terminal cost functions are of the form

f(t, x,m, α) =1

2

(

qx2 + α2 + q(x− sm)2)

g(x,m) =1

2

(


)

where q, q, s, qT , qT , sT are constant and

m =

∫

R

ydm(y)

123

CHAPTER 5. LINEAR-QUADRATIC MFG WITH COMMON NOISE 124

With these cost functions, the generalized Hamiltonian becomes

H(t, a, x, y,m) =1

2

(

qx2 + α2 + q(x− sm)2)

+ αy

As a result, the minimizer is given by

α(t, x, y,m) = −y

so the Hamiltonian is

H(t, x, y,m) = −y2

2+

1

2(q + q)x2 − qsxm+

1

2qs2m2

5.1 DPP approach

As discussed in Chapter 2, there are two ways to formulate an MFG problem through

DPP approach. The first one is to study the master equation of the generalize value

function denoted by Vε. The second method is to solve the FBSPDE of (uε, mε) where

the backward SPDE of uε describes a stochastic HJB of the value function along the

optimal distribution and the forward one gives the stochastic Fokker-Planck equation

of mε. Note that we have the relation

uε(t, x, ω) = Vε(t, x,mεt (ω)) (5.1)

So we will begin by solving the master equation to find Vε, then use the relation

above to get uε.


5.1.1 Master Equation

Using (2.8), the master equation for LQMFG model reads

∂tVε(t, x,m) + H(t, x, ∂xV (t, x,m), m) +

σ2 + ε2

2∂xxV

ε(t, x,m)

− E0[

∂mVε(t, x,m)(X)(∂xV

ε(t, X,m))]

+σ2

2∂mmV

ε(t, x,m)(X)[ζ, ζ ]

+ε2

2∂mmV

ε(t, x,m)(X)[1, 1] + ε2E0[


]

= 0

(5.2)


Vε(T, x,m) = qTx+ (x− sT m)qT = (qT + qT )x− sT qT m

Solving this equation yields the optimal control in the feedback form

αε(t, x,m) = −∂xVε(t, x,m) (5.3)

Motivated by classical LQ problems, we seek a solution of the form

Vε(t, x,m) =1

2p(t)x2 + q(t)xm+

1

2r(t)m2 + s(t)

where p(t), q(t), r(t), s(t) : [0, T ] → R are deterministic functions. Using this form,

we have

∂tVε(t, x,m) =

1

2p′(t)x2 + q′(t)xm+

1

2r′(t)m2 + s′(t)

∂xVε(t, x,m) = p(t)x+ q(t)m

∂xxVε(t, x,m) = p(t)

E0[∂mV

ε(t, x,m)(X)ξ] = (q(t)x+ r(t)m)ξ

∂mmVε(t, x,m)(X)[ξ, ξ′] = r(t)ξξ′

E0[∂xmV

ε(t, x,m)(X)ξ] = q(t)ξ

(5.4)


for any (t, x,m) ∈ [0, T ]× R×P2(R), X is a lifting random variable with L(ξ) = m,

ξ, ξ′ are arbitrary random variables in the same lifting space and ξ, ξ′ denote the mean

of ξ, ξ′ respectively. Therefore, we have

E0[∂mV

ε(t, x,m)(X)∂xVε(t, X,m)] = (q(t)x+ r(t)m)(p(t) + q(t))m

= q(t)(p(t) + q(t))xm+ r(t)(p(t) + q(t))m2

∂mmVε(t, x,m)(X)[1, 1] = r(t)

∂mmVε(t, x,m)(X)[ζ, ζ ] = 0

E0[∂xmV

ε(t, x,m)(X)1] = q(t)

(5.5)

Plugging (5.4),(5.5) into (5.2) yields

1

2p′(t)x2 + q′(t)xm+

1

2r′(t)m2 + s′(t)−

1

2p2(t)x2 −

1

2q2(t)m2 − p(t)q(t)xm

+1

2(q + q)x2 − qsxm+

1

2qs2m2 +

1

2(σ2 + ε2)p(t)− q(t)(p(t) + q(t))xm

− r(t)(p(t) + q(t))m2 +ε2

2r(t) + ε2q(t) = 0

Grouping the coefficients of x2, xm, m2, 1 and using the terminal conditions, we get

the system of ODEs

p′(t) = p2(t)− q − q, p(T ) = qT + qT

q′(t) = 2p(t)q(t) + q2(t) + qs, q(T ) = −sT qT

r′(t) = 2(p(t) + q(t))r(t) + q(t)2 − qs2, r(T ) = s2T qT

s′(t) = −1

2(σ2 + ε2)p(t)−

1

2ε2r(t)− ε2q(t), s(T ) = 0

(5.6)

The system above is decoupled since we can solve in successive order from the top

equation of p(t) to the bottom equation of s(t). From the ODE of s(t), we know that

s(t) is of the form

s(t) = a(t) + ε2b(t)


where

a(t) =σ2

2

∫ T

t

p(s)ds, b(t) =

∫ T

t

(

1

2(p(s) + r(s)) + q(s)

)

ds (5.7)

So we have the generalized value function

Vε(t, x,m) =1

2p(t)x2 + q(t)xm+

1

2r(t)m2 + a(t) + ε2b(t)

where p(t), q(t), r(t) are given by the ODEs (5.6) and a(t), b(t) are given by (5.7).

Going back to (5.3), we have the optimal control in the feedback form

αε(t, x,m) = −p(t)x− q(t)m

5.1.2 Optimal controlled process and stochastic value func-

tion

Now we would like to write out explicitly the solution (uε, mε, vε) of the FBSPDE

(2.9). First, note that the optimal controlled process is given by

dXεt = (−p(t)Xε

t − q(t)mεt )dt+ σdWt + εdWt, Xε

0 = ξ0

where

mεt , E[Xε

t |Ft]

Thus, we have

dmεt = −(p(t) + q(t))mε

tdt + εdWt, mε0 = E[ξ0] (5.8)

Solving this SDE yields

mεt = β0,tm

ε0 + ε

∫ t

0

βs,tdWs, βs,t = exp

(

−

∫ t

s

[p(r) + q(r)]dr

)

(5.9)


Using relation (5.1), we have the stochastic value function

uε(t, x) =1

2p(t)x2 + q(t)x

(

β0,tmε0 + ε

∫ t

0

βs,tdWs

)

+1

2r(t)

(

β0,tmε0 + ε

∫ t

0

βs,tdWs

)2

+ a(t) + ε2b(t)

We also have the diffusion term for duε(t, x) which is given by

vε(t, x) = E0[∂mV

ε(t, x,mεt )(X)1] = q(t)x+r(t)mε

t = q(t)x+r(t)

(

β0,tmε0 + ε

∫ t

0

βs,tdWs

)

5.2 SMP approach

Recall that the Hamiltonian and the optimal feedback control are given by

H(t, x, y,m) = −y2

2+

1

2(q + q)x2 − qsxm+

1

2qs2m2, α(t, x, y) = −y

Therefore, the corresponding McKean-Vlasov FBSDE is

dXt = −Ytdt+ σdWt + εdWt

dYt =(

(q + q)Xt − qsE[Xt|Ft])

dt+ ZtdWt + ZtdWt

X0 = ξ, YT = (qT + qT )XT − sT qTE[XT |FT ]

(5.10)

We can take the ansatz

Yt = P (t)Xt +Q(t)E[Xt|Ft], P (T ) = qT + qT , Q(T ) = −sT qT

to solve this system. However, having found the generalized value function, we can

use Theorem (2.3.1) to see that the decoupling function is simply

Uε(t, x,m) = ∂xVε(t, x,m) = p(t)x+ q(t)m


Thus, we have that

Y εt = p(t)Xε

t + q(t)E[Xεt |Ft], Zε

t = σp(t), Zεt = −εq(t)(p(t) + q(t))E[Xε

t |Ft]

solve FBSDE (5.11) where (Xεt )0≤t≤T satisfies the mean-field SDE

dXεt = (−p(t)Xε

t − q(t)E[Xεt |Ft])dt+ σdWt + εWt, Xε

0 = ξ0

The SDE above can be solved explicitly by plugging in (mεt )0≤t≤T from (5.9).

5.3 Asymptotic analysis

The linear variational process reads

dUt = −Vtdt+ dWt

dVt =(

(q + q)Ut − qsE[Ut|Ft])

dt+QtdWt + QtdWt

U0 = 0, YT = (qT + qT )UT − sT qTE[UT |FT ]

(5.11)

Note that

∂xU0(t, x,m) = p(t), E

0[∂mU(t, x,m)(X)ξ] = q(t)E0[ξ], h(t, x, z) = q(t)

We can use (4.13) to get

Vt = p(t)Ut + q(t)E[Ut|Ft] (5.12)

where (Ut)0≤t≤T solves

dUt =(

−p(t)Ut − q(t)E[Ut|Ft])

dt+ dWt, U0 = 0

We can deduce from the SDE above that (Ut)0≤t≤T is independent of ω ∈ Ω0 and is

Ft-adpated, thus

Ut = E[Ut|Ft]


In other words, Ut is independent of the current state X0t . The main reason for this

fact is the fact that ∂xxu0(t, x) is constant in linear-quadratic case. Thus, the SDE

for (Ut)0≤t≤T is simply

dUt = −(p(t) + q(t))Utdt+ dWt, U0 = 0 (5.13)

From (5.12), we also have all the related terms from Section 4.4.2

η(t, x) = q(t)Ut, w(t, x) = h(t, x, z) = q(t)

Lastly, we describe the distribution of (Ut, Vt)0≤t≤T . The fact that (Ut, Vt)0≤t≤T are

centered Gaussian processes follows directly from (5.12) and (5.13). Now we compute

the covariance function

ϕUt , E

[

U2t

]

, ϕVt , E

[

V 2t

]

, ϕUVt , E [UtVt]

directly using the (5.12) and (5.13) again to get

dϕUt

dt= −2(p(t) + q(t))ϕU

t + 1, ϕU0 = 0

ϕUVt = (p(t) + q(t))ϕU

t

ϕVt = (p(t) + q(t))2ϕU

t

Notice that all the terms in the asymptotic analysis is trivial; (Ut)0≤t≤T is independent

of (X0t )0≤t≤T and η, w, ∂mU0 are independent of x. The main reason for this is due

to the fact that ∂xxf, ∂xxg are constant, hence so does ∂xxu0 and other second order

terms.

Bibliography

[1] Yves Achdou and Italo Capuzzo-Dolcetta. Mean field games: Numerical meth-

ods. SIAM Journal on Numerical Analysis, 48(3):1136–1162, 2010.

[2] Yves Achdou, Jean-Michel Lasry, Pierre-Louis Lions, and Benjamin Moll. Het-

erogeneous agent models in continuous time. Technical report, mimeo, 2013.

[3] Saran Ahuja. Wellposedness of mean field games with common noise under a

weak monotonicity condition. arXiv preprint arXiv:1406.7028, 2014.

[4] Luigi Ambrosio, Nicola Gigli, and Giuseppe Savarae. Gradient flows. Springer,

2005.

[5] Fabio Antonelli. Backward-forward stochastic differential equations. The Annals

of Applied Probability, pages 777–793, 1993.

[6] Fabio Antonelli and Jin Ma. Weak solutions of forward–backward sde. Stochastic

Analysis and Applications, 21(3):493–514, 2003.

[7] Martino Bardi and Fabio S Priuli. Linear-quadratic n-person and mean-

field games with ergodic cost. SIAM Journal on Control and Optimization,

52(5):3022–3052, 2014.

[8] Alain Bensoussan, Michael Chau, and Phillip Yam. Mean field games with a

dominating player. arXiv preprint arXiv:1404.4148, 2014.

[9] Alain Bensoussan, Jens Frehse, and Phillip Yam. Mean Field Games and Mean

Field Type Control Theory. Springer, 2013.

131

BIBLIOGRAPHY 132

[10] Alain Bensoussan, Jens Frehse, and Sheung Chi Phillip Yam. The master equa-

tion in mean field theory. Journal de Mathematiques Pures et Appliquees, 2014.

[11] Alain Bensoussan, Joseph Sung, Phillip Yam, and Siu Pang Yung. Linear-

quadratic mean field games. arXiv preprint arXiv:1404.5741, 2014.

[12] Jean-Michel Bismut. Theorie probabiliste du controle des diffusions, volume 167.

American Mathematical Soc., 1976.

[13] Pierre Cardaliaguet. Notes on mean field games. from P.-L. Lions’ lectures at

College de France, 2010.

[14] Pierre Cardaliaguet, P Jameson Graber, Alessio Porretta, and Daniela Tonon.

Second order mean field games with degenerate diffusion and local coupling.

Nonlinear Differential Equations and Applications NoDEA, pages 1–31, 2014.

[15] Pierre Cardaliaguet, J-M Lasry, P-L Lions, and Alessio Porretta. Long time

average of mean field games with a nonlocal coupling. SIAM Journal on Control

and Optimization, 51(5):3558–3591, 2013.

[16] Rene Carmona and Francois Delarue. Forward-backward stochastic differ-

ential equations and controlled mckean vlasov dynamics. arXiv preprint

arXiv:1303.5835, 2013.

[17] Rene Carmona and Francois Delarue. Probabilistic analysis of mean-field games.

SIAM Journal on Control and Optimization, 51(4):2705–2734, 2013.

[18] Rene Carmona and Francois Delarue. The master equation for large popula-

tion equilibriums. In Stochastic Analysis and Applications 2014, pages 77–128.

Springer, 2014.

[19] Rene Carmona, Francois Delarue, and Aime Lachapelle. Control of mckean–

vlasov dynamics versus mean field games. Mathematics and Financial Eco-

nomics, 7(2):131–166, 2013.

BIBLIOGRAPHY 133

[20] Rene Carmona, Francois Delarue, and Daniel Lacker. Mean field games with

common noise. arXiv preprint arXiv:1407.6181, 2014.

[21] Rene Carmona, Jean-Pierre Fouque, and Li-Hsien Sun. Mean field games and

systemic risk. Available at SSRN 2307814, 2013.

[22] Rene Carmona and Daniel Lacker. A probabilistic weak formulation of mean

field games and applications. arXiv preprint arXiv:1307.1152, 2013.

[23] Rene Carmona and Xiuneng Zhu. A probabilistic approach to mean field games

with major and minor players. arXiv preprint arXiv:1409.7141, 2014.

[24] Jean-Francois Chassagneux, Dan Crisan, and Francois Delarue. Classical so-

lutions to the master equation for large population equilibria. arXiv preprint

arXiv:1411.3009, 2014.

[25] Donald Dawson and Jean Vaillancourt. Stochastic mckean-vlasov equations.

Nonlinear Differential Equations and Applications NoDEA, 2(2):199–229, 1995.

[26] Francois Delarue. On the existence and uniqueness of solutions to fbsdes in a

non-degenerate case. Stochastic processes and their applications, 99(2):209–286,

2002.

[27] Markus Fischer. On the connection between symmetric n-player games and mean

field games. arXiv preprint arXiv:1405.1345, 2014.

[28] Wendell H Fleming and Halil Mete Soner. Controlled Markov processes and

viscosity solutions, volume 25. Springer Science & Business Media, 2006.

[29] Diogo A Gomes, Joana Mohr, and Rafael Rigao Souza. Discrete time, finite

state space mean field games. Journal de mathematiques pures et appliquees,

93(3):308–328, 2010.

[30] Diogo A Gomes, Joana Mohr, and Rafael Rigao Souza. Continuous time finite

state mean field games. Applied Mathematics & Optimization, pages 1–45, 2012.

BIBLIOGRAPHY 134

[31] Diogo A Gomes and Vardan K Voskanyan. Extended mean field

games-formulation, existence, uniqueness and examples. arXiv preprint

arXiv:1305.2600, 2013.

[32] DiogoA. Gomes and Joao Saude. Mean field games models—a brief survey.

Dynamic Games and Applications, pages 1–45, 2013.

[33] Olivier Gueant, Jean-Michel Lasry, and Pierre-Louis Lions. Mean field games

and applications. In Paris-Princeton Lectures on Mathematical Finance 2010,

pages 205–266. Springer, 2011.

[34] Ying Hu and Shige Peng. Maximum principle for semilinear stochastic evolution

control systems. Stochastics and Stochastic Reports, 33(3-4):159–180, 1990.

[35] Ying Hu and Shige Peng. Solution of forward-backward stochastic differential

equations. Probability Theory and Related Fields, 103(2):273–283, 1995.

[36] Minyi Huang. Large-population lqg games involving a major player: the nash

certainty equivalence principle. SIAM Journal on Control and Optimization,

48(5):3318–3353, 2010.

[37] Minyi Huang, Peter E Caines, and Roland P Malhame. Large-population cost-

coupled lqg problems with nonuniform agents: Individual-mass behavior and

decentralized ε-nash equilibria. Automatic Control, IEEE Transactions on,

52(9):1560–1571, 2007.

[38] Ioannis Karatzas. Brownian motion and stochastic calculus, volume 113. Springer

Science & Business Media, 1991.

[39] Vassili N Kolokoltsov, Jiajie Li, and Wei Yang. Mean field games and nonlinear

markov processes. arXiv preprint arXiv:1112.3744, 2011.

[40] Daniel Lacker. A general characterization of the mean field limit for stochastic

differential games. arXiv preprint arXiv:1408.2708, 2014.

BIBLIOGRAPHY 135

[41] Daniel Lacker and Kevin Webster. Translation invariant mean field games with

common noise. arXiv preprint arXiv:1409.7345, 2014.

[42] Jean-Michel Lasry and Pierre-Louis Lions. Jeux a champ moyen. i–le cas sta-

tionnaire. Comptes Rendus Mathematique, 343(9):619–625, 2006.

[43] Jean-Michel Lasry and Pierre-Louis Lions. Jeux a champ moyen. ii–horizon fini

et controle optimal. Comptes Rendus Mathematique, 343(10):679–684, 2006.

[44] Jean-Michel Lasry and Pierre-Louis Lions. Mean field games. Japanese Journal

of Mathematics, 2(1):229–260, 2007.

[45] Jean-Michel Lasry, Pierre-Louis Lions, Olivier Gueant, et al. Application of mean

field games to growth theory. 2008.

[46] Jin Ma, Philip Protter, and Jiongmin Yong. Solving forward-backward stochastic

differential equations explicitly—a four step scheme. Probability Theory and

Related Fields, 98(3):339–359, 1994.

[47] Jin Ma, Zhen Wu, Detao Zhang, and Jianfeng Zhang. On wellposedness of

forward-backward sdes—a unified approach. arXiv preprint arXiv:1110.4658,

2011.

[48] Jin Ma, Hong Yin, and Jianfeng Zhang. On non-markovian forward–backward

sdes and backward stochastic pdes. Stochastic Processes and Their Applications,

122(12):3980–4004, 2012.

[49] Jin Ma and Jiongmin Yong. Forward-backward stochastic differential equations

and their applications. Number 1702. Springer, 1999.

[50] Son Luu Nguyen and Minyi Huang. Linear-quadratic-gaussian mixed games with

continuum-parametrized minor players. SIAM Journal on Control and Optimiza-

tion, 50(5):2907–2937, 2012.

[51] Mojtaba Nourian and Peter E Caines. ǫ-nash mean field game theory for nonlin-

ear stochastic dynamical systems with major and minor agents. SIAM Journal

on Control and Optimization, 51(4):3302–3331, 2013.

BIBLIOGRAPHY 136

[52] Shige Peng and Zhen Wu. Fully coupled forward-backward stochastic differential

equations and applications to optimal control. SIAM Journal on Control and

Optimization, 37(3):825–843, 1999.

[53] Huyen Pham. Continuous-time stochastic control and optimization with financial

applications, volume 61. Springer, 2009.

[54] Alessio Porretta. Weak solutions to fokker–planck equations and mean field

games. Archive for Rational Mechanics and Analysis, 216(1):1–62, 2015.

[55] Svetlozar T Rachev and Ludger Ruschendorf. Mass Transportation Problems:

Volume I: Theory, volume 1. Springer Science & Business Media, 1998.

[56] Alain-Sol Sznitman. Topics in propagation of chaos. In Ecole d’ete de probabilites

de Saint-Flour XIX—1989, pages 165–251. Springer, 1991.

[57] Cedric Villani. Optimal transport: old and new, volume 338. Springer, 2009.

[58] Hong Yin. Solvability of forward–backward stochastic partial differential equa-

tions. Stochastic Processes and their Applications, 124(8):2583–2604, 2014.

[59] Jiongmin Yong. Finding adapted solutions of forward–backward stochastic differ-

ential equations: method of continuation. Probability Theory and Related Fields,

107(4):537–572, 1997.

[60] Jiongmin Yong. Linear forward-backward stochastic differential equations with

random coefficients. Probability theory and related fields, 135(1):53–83, 2006.

[61] Jiongmin Yong and Xun Yu Zhou. Stochastic controls: Hamiltonian systems and

HJB equations, volume 43. Springer, 1999.

Documents

MEAN FIELD GAMES WITH COMMON NOISE A …vt075xr1988/Thesis_main-augmented.pdfWe characterize the ﬁrst order approximation terms as the solution to a linear FBSDE of mean-ﬁeld type