Upload
others
View
9
Download
0
Embed Size (px)
Citation preview
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Introduction. Consider two countries and the exchange ratebetween their currencies. Unfortunately, the countries disagree:
Country 1 would like the exchange rate to be greater than s1,Country 2 would like the exchange rate to be smaller than s2.
In a static situation (fixed rate), the countries would choose s1+s22 .
Consider now a dynamic situation: the rate varies in time and isstochastic. If the rate gets too low (resp. high), Country 1 (resp. 2)may want to intervene and shift it. When should each countryintervene? What should the new rate be? Generalization to N>2?
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Introduction. Consider two countries and the exchange ratebetween their currencies. Unfortunately, the countries disagree:
Country 1 would like the exchange rate to be greater than s1,Country 2 would like the exchange rate to be smaller than s2.
In a static situation (fixed rate), the countries would choose s1+s22 .
Consider now a dynamic situation: the rate varies in time and isstochastic. If the rate gets too low (resp. high), Country 1 (resp. 2)may want to intervene and shift it. When should each countryintervene? What should the new rate be? Generalization to N>2?
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
More generally, we are here interested in the following problems,which model many practical situations (like the one above).
A stochastic process X : dXt=b(Xt)dt+�(Xt)dWt , with Xt 2Rd .
N players can intervene and shift the process from Xt to Xt + �.
The behaviour of each player is defined by a sequence ofintervention times and shifts: ⇣i = {(⌧i,k , �i,k)}k2N, 1 iN
(impulse control).
Each player aims at maximizing his own payoff, consisting in thesum of: running cost, intervention cost, final cost.
Hence, from a mathematical point of view, we here deal with:N-player nonzero-sum stochastic games with impulse controls.
This is a very recent research area...
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
More generally, we are here interested in the following problems,which model many practical situations (like the one above).
A stochastic process X : dXt=b(Xt)dt+�(Xt)dWt , with Xt 2Rd .
N players can intervene and shift the process from Xt to Xt + �.
The behaviour of each player is defined by a sequence ofintervention times and shifts: ⇣i = {(⌧i,k , �i,k)}k2N, 1 iN
(impulse control).
Each player aims at maximizing his own payoff, consisting in thesum of: running cost, intervention cost, final cost.
Hence, from a mathematical point of view, we here deal with:N-player nonzero-sum stochastic games with impulse controls.
This is a very recent research area...
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Literature, part 1. Quick literature review on stochastic games.
Stopping time Impulse control
One-pl. control problemTwo-pl. zero-sum gameTwo-pl. nonzero-sum game
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Literature, part 1. Quick literature review on stochastic games.
Stopping time Impulse control
One-pl. control problem Several authorsTwo-pl. zero-sum gameTwo-pl. nonzero-sum game
Several authors: the player chooses ⌧ so as to maximize
E Z ⌧
0e�⇢t
f (Xt)dt + e�⇢⌧
h(X⌧ )
�.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Literature, part 1. Quick literature review on stochastic games.
Stopping time Impulse control
One-pl. control problem Several authorsTwo-pl. zero-sum game FriedmanTwo-pl. nonzero-sum game
Friedman: the players choose ⌧1, ⌧2 so as to maximize
E Z ⌧1^⌧2
0e�⇢t
f (Xt)dt + e�⇢(⌧1^⌧2)h(X⌧1^⌧2)
�.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Literature, part 1. Quick literature review on stochastic games.
Stopping time Impulse control
One-pl. control problem Several authorsTwo-pl. zero-sum game FriedmanTwo-pl. nonzero-sum game Bensoussan-Friedman
Bensoussan-Friedman: the players choose ⌧1, ⌧2 so as to maximize
E Z ⌧1^⌧2
0e�⇢i t fi (Xt)dt + e
�⇢i (⌧1^⌧2)hi (X⌧1^⌧2)
�.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Literature, part 1. Quick literature review on stochastic games.
Stopping time Impulse control
One-pl. control problem Several authors Several authorsTwo-pl. zero-sum game FriedmanTwo-pl. nonzero-sum game Bensoussan-Friedman
Several authors: the player chooses u = {(⌧k , �k)}k to maximize
E Z 1
0e�⇢t
f (Xt)dt +X
k
e�⇢⌧k�
�X(⌧k )� , �k
��.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Literature, part 1. Quick literature review on stochastic games.
Stopping time Impulse control
One-pl. control problem Several authors Several authorsTwo-pl. zero-sum game Friedman CossoTwo-pl. nonzero-sum game Bensoussan-Friedman
Cosso: the players choose ui = {(⌧ ik , �
ik)}k so as to maximize
E Z 1
0e�⇢t
f (Xt)dt +X
k
e�⇢⌧i,k�
�X(⌧i,k )� , �i ,k
�
+X
k
e�⇢⌧j,k
�X(⌧j,k )� , �j ,k
��.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Literature, part 1. Quick literature review on stochastic games.
Stopping time Impulse control
One-pl. control problem Several authors Several authorsTwo-pl. zero-sum game Friedman CossoTwo-pl. nonzero-sum game Bensoussan-Friedman ???
Recent problem: the players choose ui = {(⌧ ik , �
ik)}k to maximize
E Z 1
0e�⇢i t fi (Xt)dt +
X
k
e�⇢i⌧i,k�i
�X(⌧i,k )� , �i ,k
�
+X
k
e�⇢i⌧j,k i
�X(⌧j,k )� , �j ,k
��.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Literature, part 2. Literature on nonzero-sum impulse games.
Formulation and first results for 2-player gamesAïd, Basei, Callegaro, Campi, Vargiolu, Nonzero-sum stochastic
differential games with impulse controls: a verification theorem
with applications, to appear in Mathematics of Operations Research
A numerical method to solve 2-player gamesAïd, Bernal, Mnif, Zabaljauregui, Zubelli, A policy iteration
algorithm for nonzero-sum stochastic impulse games, to appear in
Esaim Proceedings and Surveys
Nonzero impulse games to model a pollution control problemFerrari, Koch, On a strategic model of pollution control, to appear
in Annals of Operations Research
Generalization to N-player games and mean-field gamesBasei, Cao, Guo, Nonzero-sum stoch. games with impulse controls
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
The process. The underlying process, when none of the playerintervenes, is modelled by dYs = b(Ys)ds + �(Ys)dWs 2 Rd . Thegame ends at ⌧S , the exit time of Y from a fixed subset S ✓ Rd .
Interventions of the players. When player i 2 {1, . . . ,N} decidesto interv. with impulse �, the process is shifted from state x to state�i (x , �), player i pays �i (x , �), player j 6= i pays/receives j ,i (x , �).
Impulse controls. The action of player i is modelled by asequence (impulse control) in the form ui = {(⌧i ,k , �i ,k)}k�1,where {⌧i ,k}k are increasing stopping times (the intervention times)and {�i ,k}k are random variables (the corresponding impulses).
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
The process. The underlying process, when none of the playerintervenes, is modelled by dYs = b(Ys)ds + �(Ys)dWs 2 Rd . Thegame ends at ⌧S , the exit time of Y from a fixed subset S ✓ Rd .
Interventions of the players. When player i 2 {1, . . . ,N} decidesto interv. with impulse �, the process is shifted from state x to state�i (x , �), player i pays �i (x , �), player j 6= i pays/receives j ,i (x , �).
Impulse controls. The action of player i is modelled by asequence (impulse control) in the form ui = {(⌧i ,k , �i ,k)}k�1,where {⌧i ,k}k are increasing stopping times (the intervention times)and {�i ,k}k are random variables (the corresponding impulses).
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
The process. The underlying process, when none of the playerintervenes, is modelled by dYs = b(Ys)ds + �(Ys)dWs 2 Rd . Thegame ends at ⌧S , the exit time of Y from a fixed subset S ✓ Rd .
Interventions of the players. When player i 2 {1, . . . ,N} decidesto interv. with impulse �, the process is shifted from state x to state�i (x , �), player i pays �i (x , �), player j 6= i pays/receives j ,i (x , �).
Impulse controls. The action of player i is modelled by asequence (impulse control) in the form ui = {(⌧i ,k , �i ,k)}k�1,where {⌧i ,k}k are increasing stopping times (the intervention times)and {�i ,k}k are random variables (the corresponding impulses).
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Strategies. The behaviour of the players, modelled by impulsecontrols, is driven by strategies.
A strategy for player i 2 {1, . . . ,N} is a pair 'i = (Ai , ⇠i ), whereAi is a fixed subset of Rd and ⇠i is a continuous function.
Once a N-uple of strategies ' = ('1, . . . ,'N), with 'i = (Ai , ⇠i ),and a starting point x have been chosen, N impulse controls and acontrolled process X = X
x ;' are uniquely defined as follows:
- player i intervenes if and only if the process enters the region Ai ,in which case the impulse is given by ⇠i (y), where y is the state;
- The game ends when the process exits from S .
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Strategies. The behaviour of the players, modelled by impulsecontrols, is driven by strategies.
A strategy for player i 2 {1, . . . ,N} is a pair 'i = (Ai , ⇠i ), whereAi is a fixed subset of Rd and ⇠i is a continuous function.
Once a N-uple of strategies ' = ('1, . . . ,'N), with 'i = (Ai , ⇠i ),and a starting point x have been chosen, N impulse controls and acontrolled process X = X
x ;' are uniquely defined as follows:
- player i intervenes if and only if the process enters the region Ai ,in which case the impulse is given by ⇠i (y), where y is the state;
- The game ends when the process exits from S .
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Functionals. Let ' = ('1, . . . ,'N) be a N-uple of strategies andlet x be the initial state. Player i aims at maximizing the followingfunctional (running cost, intervention costs, final cost):
Ji (x ;') := Ex
Z ⌧S
0e�⇢i s fi (Xs)ds +
X
k2Ne�⇢i⌧i,k�i
⇣X(⌧i,k )� , �i ,k
⌘
+X
j 6=i
X
k2Ne�⇢i⌧j,k i ,j
⇣X(⌧j,k )� , �j ,k
⌘+ e
�⇢i⌧Shi�X(⌧S )�
�{⌧S<+1}
�.
Nash equilibrium. We look for strategies '⇤ = ('⇤1, . . . ,'
⇤N) s.t.
Vi (x) := Ji (x ;'⇤) � J
i (x ; si ,'i ('⇤)), 8 i 2 {1, . . . ,N}, 8'i ,
where si ,'i (') substitutes the i-th element in ' by 'i .
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Functionals. Let ' = ('1, . . . ,'N) be a N-uple of strategies andlet x be the initial state. Player i aims at maximizing the followingfunctional (running cost, intervention costs, final cost):
Ji (x ;') := Ex
Z ⌧S
0e�⇢i s fi (Xs)ds +
X
k2Ne�⇢i⌧i,k�i
⇣X(⌧i,k )� , �i ,k
⌘
+X
j 6=i
X
k2Ne�⇢i⌧j,k i ,j
⇣X(⌧j,k )� , �j ,k
⌘+ e
�⇢i⌧Shi�X(⌧S )�
�{⌧S<+1}
�.
Nash equilibrium. We look for strategies '⇤ = ('⇤1, . . . ,'
⇤N) s.t.
Vi (x) := Ji (x ;'⇤) � J
i (x ; si ,'i ('⇤)), 8 i 2 {1, . . . ,N}, 8'i ,
where si ,'i (') substitutes the i-th element in ' by 'i .
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Functionals. Let ' = ('1, . . . ,'N) be a N-uple of strategies andlet x be the initial state. Player i aims at maximizing the followingfunctional (running cost, intervention costs, final cost):
Ji (x ;') := Ex
Z ⌧S
0e�⇢i s fi (Xs)ds +
X
k2Ne�⇢i⌧i,k�i
⇣X(⌧i,k )� , �i ,k
⌘
+X
j 6=i
X
k2Ne�⇢i⌧j,k i ,j
⇣X(⌧j,k )� , �j ,k
⌘+ e
�⇢i⌧Shi�X(⌧S )�
�{⌧S<+1}
�.
Nash equilibrium. We look for strategies '⇤ = ('⇤1, . . . ,'
⇤N) s.t.
Vi (x) := Ji (x ;'⇤) � J
i (x ; si ,'i ('⇤)), 8 i 2 {1, . . . ,N}, 8'i ,
where si ,'i (') substitutes the i-th element in ' by 'i .
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Functionals. Let ' = ('1, . . . ,'N) be a N-uple of strategies andlet x be the initial state. Player i aims at maximizing the followingfunctional (running cost, intervention costs, final cost):
Ji (x ;') := Ex
Z ⌧S
0e�⇢i s fi (Xs)ds +
X
k2Ne�⇢i⌧i,k�i
⇣X(⌧i,k )� , �i ,k
⌘
+X
j 6=i
X
k2Ne�⇢i⌧j,k i ,j
⇣X(⌧j,k )� , �j ,k
⌘+ e
�⇢i⌧Shi�X(⌧S )�
�{⌧S<+1}
�.
Nash equilibrium. We look for strategies '⇤ = ('⇤1, . . . ,'
⇤N) s.t.
Vi (x) := Ji (x ;'⇤) � J
i (x ; si ,'i ('⇤)), 8 i 2 {1, . . . ,N}, 8'i ,
where si ,'i (') substitutes the i-th element in ' by 'i .
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Functionals. Let ' = ('1, . . . ,'N) be a N-uple of strategies andlet x be the initial state. Player i aims at maximizing the followingfunctional (running cost, intervention costs, final cost):
Ji (x ;') := Ex
Z ⌧S
0e�⇢i s fi (Xs)ds +
X
k2Ne�⇢i⌧i,k�i
⇣X(⌧i,k )� , �i ,k
⌘
+X
j 6=i
X
k2Ne�⇢i⌧j,k i ,j
⇣X(⌧j,k )� , �j ,k
⌘+ e
�⇢i⌧Shi�X(⌧S )�
�{⌧S<+1}
�.
Nash equilibrium. We look for strategies '⇤ = ('⇤1, . . . ,'
⇤N) s.t.
Vi (x) := Ji (x ;'⇤) � J
i (x ; si ,'i ('⇤)), 8 i 2 {1, . . . ,N}, 8'i ,
where si ,'i (') substitutes the i-th element in ' by 'i .
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Functionals. Let ' = ('1, . . . ,'N) be a N-uple of strategies andlet x be the initial state. Player i aims at maximizing the followingfunctional (running cost, intervention costs, final cost):
Ji (x ;') := Ex
Z ⌧S
0e�⇢i s fi (Xs)ds +
X
k2Ne�⇢i⌧i,k�i
⇣X(⌧i,k )� , �i ,k
⌘
+X
j 6=i
X
k2Ne�⇢i⌧j,k i ,j
⇣X(⌧j,k )� , �j ,k
⌘+ e
�⇢i⌧Shi�X(⌧S )�
�{⌧S<+1}
�.
Nash equilibrium. We look for strategies '⇤ = ('⇤1, . . . ,'
⇤N) s.t.
Vi (x) := Ji (x ;'⇤) � J
i (x ; si ,'i ('⇤)), 8 i 2 {1, . . . ,N}, 8'i ,
where si ,'i (') substitutes the i-th element in ' by 'i .
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Our goal. Looking for a verification theorem: if N functions areregular enough and satisfy suitable equations, they are actually thevalue funct. of the game and we can characterize a Nash strategy.
First of all, some heuristics about the appropriate equations for Vi
and the Nash equilibria. To simplify, let �i (x , �) = x + �.
Heuristics on Nash equilibria. Assume for a moment we knowthe value functions Vi . Define, for i , j 2{1, . . . ,N}, i 6= j , x 2 S :
{�i (x)} = arg max��Vi�x + �
�+ �i (x , �)
�,
MiVi (x) = Vi�x + �i (x)
�+ �i
�x , �i (x)
�,
Hi ,jVi (x) = Vi�x + �j(x)
�+ i ,j
�x , �j(x)
�.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Our goal. Looking for a verification theorem: if N functions areregular enough and satisfy suitable equations, they are actually thevalue funct. of the game and we can characterize a Nash strategy.
First of all, some heuristics about the appropriate equations for Vi
and the Nash equilibria. To simplify, let �i (x , �) = x + �.
Heuristics on Nash equilibria. Assume for a moment we knowthe value functions Vi . Define, for i , j 2{1, . . . ,N}, i 6= j , x 2 S :
{�i (x)} = arg max��Vi�x + �
�+ �i (x , �)
�,
MiVi (x) = Vi�x + �i (x)
�+ �i
�x , �i (x)
�,
Hi ,jVi (x) = Vi�x + �j(x)
�+ i ,j
�x , �j(x)
�.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Our goal. Looking for a verification theorem: if N functions areregular enough and satisfy suitable equations, they are actually thevalue funct. of the game and we can characterize a Nash strategy.
First of all, some heuristics about the appropriate equations for Vi
and the Nash equilibria. To simplify, let �i (x , �) = x + �.
Heuristics on Nash equilibria. Assume for a moment we knowthe value functions Vi . Define, for i , j 2{1, . . . ,N}, i 6= j , x 2 S :
{�i (x)} = arg max��Vi�x + �
�+ �i (x , �)
�,
MiVi (x) = Vi�x + �i (x)
�+ �i
�x , �i (x)
�,
Hi ,jVi (x) = Vi�x + �j(x)
�+ i ,j
�x , �j(x)
�.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Let x be the current state of the process. Interpretation:Vi (x) is the value of the game for player i ;�i (x) is the optimal impulse of player i in case of an immediateintervention by player i himself;MiVi (x) (resp. Hi ,jVi (x)) is the value of the game for playeri in case of an immediate interv. by player i (resp. player j).
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Let x be the current state of the process. Interpretation:Vi (x) is the value of the game for player i ;�i (x) is the optimal impulse of player i in case of an immediateintervention by player i himself;MiVi (x) (resp. Hi ,jVi (x)) is the value of the game for playeri in case of an immediate interv. by player i (resp. player j).
To help with the interpretation, we here recall the definitions:
Vi (x) = J i (x ;'⇤),
{�i (x)} = arg max�
�Vi
�x + �
�+ �i (x , �)
�,
MiVi (x) = Vi
�x + �i (x)
�+ �i
�x , �i (x)
�,
Hi,jVi (x) = Vi
�x + �j(x)
�+ i,j
�x , �j(x)
�.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Let x be the current state of the process. Interpretation:Vi (x) is the value of the game for player i ;�i (x) is the optimal impulse of player i in case of an immediateintervention by player i himself;MiVi (x) (resp. Hi ,jVi (x)) is the value of the game for playeri in case of an immediate interv. by player i (resp. player j).
As a consequence, we (heuristically) argue that the Nash policy is:
player i intervenes if and only if MiVi (x) = Vi (x)
and shifts the process from x to x + �i (x).
So, provided that Vi is known, we can build a Nash equilibrium.We now need to characterize Vi , by means of suitable equations.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Heuristics on Vi . We consider the following quasi-variationalinequalities (QVI) for the Vi ’s, where i , j 2 {1, . . . ,N} and i 6= j :
Vi = hi , in @S ,
MjVj � Vj 0, in S ,
Hi ,jVi � Vi = 0, in {MjVj�Vj =0},max
�AVi�⇢iVi+fi ,MiVi�Vi} = 0, in \j 6=i {MjVj�Vj <0},
where AVi = b ·rVi + tr(��tD2Vi )/2 (infinitesimal generator).
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Heuristics on Vi . We consider the following quasi-variationalinequalities (QVI) for the Vi ’s, where i , j 2 {1, . . . ,N} and i 6= j :
Vi = hi , in @S ,
MjVj � Vj 0, in S ,
Hi ,jVi � Vi = 0, in {MjVj�Vj =0},max
�AVi�⇢iVi+fi ,MiVi�Vi} = 0, in \j 6=i {MjVj�Vj <0},
where AVi = b ·rVi + tr(��tD2Vi )/2 (infinitesimal generator).
First equation. Standard terminal condition.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Heuristics on Vi . We consider the following quasi-variationalinequalities (QVI) for the Vi ’s, where i , j 2 {1, . . . ,N} and i 6= j :
Vi = hi , in @S ,
MjVj � Vj 0, in S ,
Hi ,jVi � Vi = 0, in {MjVj�Vj =0},max
�AVi�⇢iVi+fi ,MiVi�Vi} = 0, in \j 6=i {MjVj�Vj <0},
where AVi = b ·rVi + tr(��tD2Vi )/2 (infinitesimal generator).
Second equation. We expect MjVj � Vj 0 thanks to theinterpretation above. Standard condition in impulse control theory.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Heuristics on Vi . We consider the following quasi-variationalinequalities (QVI) for the Vi ’s, where i , j 2 {1, . . . ,N} and i 6= j :
Vi = hi , in @S ,
MjVj � Vj 0, in S ,
Hi ,jVi � Vi = 0, in {MjVj�Vj =0},
max�AVi�⇢iVi+fi ,MiVi�Vi} = 0, in \j 6=i {MjVj�Vj <0},
where AVi = b ·rVi + tr(��tD2Vi )/2 (infinitesimal generator).
Third equation. If player j intervenes (i.e. MjVj � Vj = 0), by thedefinition of Nash equilibrium we expect that player i does not loseanything: this is modelled by Hi ,jVi � Vi = 0.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Heuristics on Vi . We consider the following quasi-variationalinequalities (QVI) for the Vi ’s, where i , j 2 {1, . . . ,N} and i 6= j :
Vi = hi , in @S ,
MjVj � Vj 0, in S ,
Hi ,jVi � Vi = 0, in {MjVj�Vj =0},max
�AVi�⇢iVi+fi ,MiVi�Vi} = 0, in \j 6=i {MjVj�Vj <0},
where AVi = b ·rVi + tr(��tD2Vi )/2 (infinitesimal generator).
Fourth equation. If none of player i ’s competitors intervenes(i.e. MjVj�Vj <0, for each j 6= i), then Vi satisfies the PDE of astandard one-player impulse problem.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Heuristics on Vi . We consider the following quasi-variationalinequalities (QVI) for the Vi ’s, where i , j 2 {1, . . . ,N} and i 6= j :
Vi = hi , in @S ,
MjVj � Vj 0, in S ,
Hi ,jVi � Vi = 0, in {MjVj�Vj =0},max
�AVi�⇢iVi+fi ,MiVi�Vi} = 0, in \j 6=i {MjVj�Vj <0},
where AVi = b ·rVi + tr(��tD2Vi )/2 (infinitesimal generator).
Verification theorem. We are now ready to state and prove theverification theorem for our class of problems.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Verification theoremLet V1, . . . ,VN be functions from S to R satisfying some (veryweak) technical assumptions and such that:
- Vi is a classical solution to (QVI),- Vi 2 C
2(D�i \ @Di ) \ C1(D�i ) \ C (D�i ),
where i , j 2{1, . . . ,N}, Di ={MiVi�Vi <0}, D�i =\j 6=iDj .Then the Vi are the value functions and a Nash equil. is as follows.
Player i intervenes if and only if X enters {MiVi�Vi =0}.
When intervening, player i shifts X from the current state x tothe state x + �i (x), where �i (x) is the (unique) maximizer of� 7! Vi
�x + �
�+ �i (x , �).
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Regularity assumptions. When solving the QVI problem, onedeals with functions which are piecewise defined (this will be clearlater). The regularity conditions are set as suitable smooth-pastingconditions: what we get is a system of algebraic equations.
If the regularity conditions are too strong, the system has moreequations than parameters, with no possibility to apply the theorem.
Then, it is fundamental to provide regularity conditions which allowto prove the verification theorem but which also make possible topractically apply this theorem.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Regularity assumptions. When solving the QVI problem, onedeals with functions which are piecewise defined (this will be clearlater). The regularity conditions are set as suitable smooth-pastingconditions: what we get is a system of algebraic equations.
If the regularity conditions are too strong, the system has moreequations than parameters, with no possibility to apply the theorem.
Then, it is fundamental to provide regularity conditions which allowto prove the verification theorem but which also make possible topractically apply this theorem.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Regularity assumptions. When solving the QVI problem, onedeals with functions which are piecewise defined (this will be clearlater). The regularity conditions are set as suitable smooth-pastingconditions: what we get is a system of algebraic equations.
If the regularity conditions are too strong, the system has moreequations than parameters, with no possibility to apply the theorem.
Then, it is fundamental to provide regularity conditions which allowto prove the verification theorem but which also make possible topractically apply this theorem.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Example: a game with explicit solutions
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
The problem. Let us consider the following one-dimensional game(in the paper, a more general example):
J1(x ;'1,'2)=Ex
Z 1
0e�⇢s(Xs�s1)ds�
X
k�1
e�⇢⌧1,k c+
X
k�1
e�⇢⌧2,k c
�,
J2(x ;'1,'2)=Ex
Z 1
0e�⇢s(s2�Xs)ds�
X
k�1
e�⇢⌧2,k c+
X
k�1
e�⇢⌧1,k c
�.
where s1<s2 and, in case of no interventions, we have dXs=�dWs .
Practically, two players control a one-dimensional BM. They havesymmetric linear payoffs: player 1 (resp. player 2) needs a high(resp. low) value for the state. Fixed intervention cost/penalty.Possible interpretation: control of the exchange rate, as above.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
The problem. Let us consider the following one-dimensional game(in the paper, a more general example):
J1(x ;'1,'2)=Ex
Z 1
0e�⇢s(Xs�s1)ds�
X
k�1
e�⇢⌧1,k c+
X
k�1
e�⇢⌧2,k c
�,
J2(x ;'1,'2)=Ex
Z 1
0e�⇢s(s2�Xs)ds�
X
k�1
e�⇢⌧2,k c+
X
k�1
e�⇢⌧1,k c
�.
where s1<s2 and, in case of no interventions, we have dXs=�dWs .
Practically, two players control a one-dimensional BM. They havesymmetric linear payoffs: player 1 (resp. player 2) needs a high(resp. low) value for the state. Fixed intervention cost/penalty.Possible interpretation: control of the exchange rate, as above.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
The solving procedure is as follows.Step 1: we solve the QVI problem, to get a pair of parametriccandidates V1,V2 for the value functions.Step 2: we impose the regularity conditions, to get the value of theparameters in the candidates V1,V2 (solution to algebraic system).Step 3: we apply the verification theorem, to get a Nash equilibriumby means of V1,V2, after checking that all the assumptions hold.
Step 1: building a candidate. As player 1 needs a high value forXt , we assume his interv. region to be in the form ]�1,x1].Similarly, we expect the interv. region of player 2 to be in the form[x2,+1[. The real line is, heuristically, divided into three intervals:
]�1, x1] = {M1V1 � V1 = 0}, where player 1 intervenes,]x1, x2[ = {M1V1�V1<0, M2V2�V2<0}, where no one intervenes,
[x2,+1[ = {M2V2 � V2 = 0}, where player 2 intervenes.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
The solving procedure is as follows.Step 1: we solve the QVI problem, to get a pair of parametriccandidates V1,V2 for the value functions.Step 2: we impose the regularity conditions, to get the value of theparameters in the candidates V1,V2 (solution to algebraic system).Step 3: we apply the verification theorem, to get a Nash equilibriumby means of V1,V2, after checking that all the assumptions hold.
Step 1: building a candidate. As player 1 needs a high value forXt , we assume his interv. region to be in the form ]�1,x1].Similarly, we expect the interv. region of player 2 to be in the form[x2,+1[. The real line is, heuristically, divided into three intervals:
]�1, x1] = {M1V1 � V1 = 0}, where player 1 intervenes,]x1, x2[ = {M1V1�V1<0, M2V2�V2<0}, where no one intervenes,
[x2,+1[ = {M2V2 � V2 = 0}, where player 2 intervenes.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
The equations in the QVI problem here read
HiVi � Vi = 0, in {MjVj � Vj = 0},max
�AVi � ⇢Vi + fi ,MiVi � Vi} = 0, in {MjVj � Vj < 0},
which we can rewrite as (where 'i is a sol. to AVi � ⇢Vi + fi = 0):
Vi =
8><
>:
MiVi , in {MiVi � Vi = 0},'i , in {MiVi � Vi < 0,MjVj � Vj < 0},HiVi , in {MjVj � Vj = 0}.
Finally, by the previous partition of the real line, we get
V1 =
8><
>:
M1V1, in ]�1, x1],'1, in ]x1, x2[,H1V1, in [x2,+1[,
V2 =
8><
>:
M2V2, in [x2,+1[,'2, in ]x1, x2[,H2V2, in ]�1, x1].
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
The equations in the QVI problem here read
HiVi � Vi = 0, in {MjVj � Vj = 0},max
�AVi � ⇢Vi + fi ,MiVi � Vi} = 0, in {MjVj � Vj < 0},
which we can rewrite as (where 'i is a sol. to AVi � ⇢Vi + fi = 0):
Vi =
8><
>:
MiVi , in {MiVi � Vi = 0},'i , in {MiVi � Vi < 0,MjVj � Vj < 0},HiVi , in {MjVj � Vj = 0}.
Finally, by the previous partition of the real line, we get
V1 =
8><
>:
M1V1, in ]�1, x1],'1, in ]x1, x2[,H1V1, in [x2,+1[,
V2 =
8><
>:
M2V2, in [x2,+1[,'2, in ]x1, x2[,H2V2, in ]�1, x1].
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
The equations in the QVI problem here read
HiVi � Vi = 0, in {MjVj � Vj = 0},max
�AVi � ⇢Vi + fi ,MiVi � Vi} = 0, in {MjVj � Vj < 0},
which we can rewrite as (where 'i is a sol. to AVi � ⇢Vi + fi = 0):
Vi =
8><
>:
MiVi , in {MiVi � Vi = 0},'i , in {MiVi � Vi < 0,MjVj � Vj < 0},HiVi , in {MjVj � Vj = 0}.
Finally, by the previous partition of the real line, we get
V1 =
8><
>:
M1V1, in ]�1, x1],'1, in ]x1, x2[,H1V1, in [x2,+1[,
V2 =
8><
>:
M2V2, in [x2,+1[,'2, in ]x1, x2[,H2V2, in ]�1, x1].
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
By heuristic arguments we can estimate MiVi and HiVi . Thisleads to the following (class of) candidates, where x
⇤i is a local
maximum of 'i in the interval ]x1, x2[:
eV1(x) =
8><
>:
'1(x⇤1 ;A11,A12)� c , if x 2 ]�1, x1],'1(x ;A11,A12), if x 2 ]x1, x2[,'1(x⇤2 ;A11,A12) + c , if x 2 [x2,+1[,
eV2(x) =
8><
>:
'2(x⇤1 ;A21,A22) + c , if x 2 ]�1, x1],'2(x ;A21,A22), if x 2 ]x1, x2[,'2(x⇤2 ;A21,A22)� c , if x 2 [x2,+1[.
Notice that some free parameters are present at the moment. Wenow set such parameters by imposing the regularity conditions.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
By heuristic arguments we can estimate MiVi and HiVi . Thisleads to the following (class of) candidates, where x
⇤i is a local
maximum of 'i in the interval ]x1, x2[:
eV1(x) =
8><
>:
'1(x⇤1 ;A11,A12)� c , if x 2 ]�1, x1],'1(x ;A11,A12), if x 2 ]x1, x2[,'1(x⇤2 ;A11,A12) + c , if x 2 [x2,+1[,
eV2(x) =
8><
>:
'2(x⇤1 ;A21,A22) + c , if x 2 ]�1, x1],'2(x ;A21,A22), if x 2 ]x1, x2[,'2(x⇤2 ;A21,A22)� c , if x 2 [x2,+1[.
Notice that some free parameters are present at the moment. Wenow set such parameters by imposing the regularity conditions.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Step 2: conditions on the coefficients. We apply the regularityconditions. To sum up, we have to solve the following problem:
8>>><
>>>:
'01(x
⇤1 ;A11,A12) = 0, '00
1 (x⇤1 ;A11,A12) 0, (optimality of x⇤1 )
'01(x1;A11,A12) = 0, (C 1-pasting in x1)
'1(x1;A11,A12) = '1(x⇤1 ;A11,A12)� c , (C 0-pasting in x1)'1(x2;A11,A12) = '1(x⇤2 ;A11,A12) + c , (C 0-pasting in x2)
8>>><
>>>:
'02(x
⇤2 ;A21,A22) = 0, '00
2 (x⇤2 ;A21,A22) 0, (optimality of x⇤2 )
'02(x2;A21,A22) = 0, (C 1-pasting in x2)
'2(x1;A21,A22) = '2(x⇤1 ;A21,A22) + c , (C 0-pasting in x1)'2(x2;A21,A22) = '2(x⇤2 ;A21,A22)� c , (C 0-pasting in x2)
(x1 < x
⇤1 < x2, (order condition 1)
x1 < x⇤2 < x2, (order condition 2)
where 'i (x ; a, a)=ae✓x+ae
�✓x+(�1)i+1(x � si )/⇢ and ✓=p
2⇢/�.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Step 2: conditions on the coefficients. We apply the regularityconditions. To sum up, we have to solve the following problem:
8>>><
>>>:
'01(x
⇤1 ;A11,A12) = 0, '00
1 (x⇤1 ;A11,A12) 0, (optimality of x⇤1 )
'01(x1;A11,A12) = 0, (C 1-pasting in x1)
'1(x1;A11,A12) = '1(x⇤1 ;A11,A12)� c , (C 0-pasting in x1)'1(x2;A11,A12) = '1(x⇤2 ;A11,A12) + c , (C 0-pasting in x2)
8>>><
>>>:
'02(x
⇤2 ;A21,A22) = 0, '00
2 (x⇤2 ;A21,A22) 0, (optimality of x⇤2 )
'02(x2;A21,A22) = 0, (C 1-pasting in x2)
'2(x1;A21,A22) = '2(x⇤1 ;A21,A22) + c , (C 0-pasting in x1)'2(x2;A21,A22) = '2(x⇤2 ;A21,A22)� c , (C 0-pasting in x2)
(x1 < x
⇤1 < x2, (order condition 1)
x1 < x⇤2 < x2, (order condition 2)
where 'i (x ; a, a)=ae✓x+ae
�✓x+(�1)i+1(x � si )/⇢ and ✓=p
2⇢/�.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Step 2: conditions on the coefficients. We apply the regularityconditions. To sum up, we have to solve the following problem:
8>>><
>>>:
'01(x
⇤1 ;A11,A12) = 0, '00
1 (x⇤1 ;A11,A12) 0, (optimality of x⇤1 )
'01(x1;A11,A12) = 0, (C 1-pasting in x1)
'1(x1;A11,A12) = '1(x⇤1 ;A11,A12)� c , (C 0-pasting in x1)'1(x2;A11,A12) = '1(x⇤2 ;A11,A12) + c , (C 0-pasting in x2)
8>>><
>>>:
'02(x
⇤2 ;A21,A22) = 0, '00
2 (x⇤2 ;A21,A22) 0, (optimality of x⇤2 )
'02(x2;A21,A22) = 0, (C 1-pasting in x2)
'2(x1;A21,A22) = '2(x⇤1 ;A21,A22) + c , (C 0-pasting in x1)'2(x2;A21,A22) = '2(x⇤2 ;A21,A22)� c , (C 0-pasting in x2)
(x1 < x
⇤1 < x2, (order condition 1)
x1 < x⇤2 < x2, (order condition 2)
where 'i (x ; a, a)=ae✓x+ae
�✓x+(�1)i+1(x � si )/⇢ and ✓=p
2⇢/�.Notice: 8 equations
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Step 2: conditions on the coefficients. We apply the regularityconditions. To sum up, we have to solve the following problem:
8>>><
>>>:
'01(x
⇤1 ;A11,A12) = 0, '00
1 (x⇤1 ;A11,A12) 0, (optimality of x⇤1 )
'01(x1;A11,A12) = 0, (C 1-pasting in x1)
'1(x1;A11,A12) = '1(x⇤1 ;A11,A12)� c , (C 0-pasting in x1)'1(x2;A11,A12) = '1(x⇤2 ;A11,A12) + c , (C 0-pasting in x2)
8>>><
>>>:
'02(x
⇤2 ;A21,A22) = 0, '00
2 (x⇤2 ;A21,A22) 0, (optimality of x⇤2 )
'02(x2;A21,A22) = 0, (C 1-pasting in x2)
'2(x1;A21,A22) = '2(x⇤1 ;A21,A22) + c , (C 0-pasting in x1)'2(x2;A21,A22) = '2(x⇤2 ;A21,A22)� c , (C 0-pasting in x2)
(x1 < x
⇤1 < x2, (order condition 1)
x1 < x⇤2 < x2, (order condition 2)
where 'i (x ; a, a)=ae✓x+ae
�✓x+(�1)i+1(x � si )/⇢ and ✓=p
2⇢/�.Notice: 8 equations and 8 parameters.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Step 2: conditions on the coefficients. We apply the regularityconditions. To sum up, we have to solve the following problem:
8>>><
>>>:
'01(x
⇤1 ;A11,A12) = 0, '00
1 (x⇤1 ;A11,A12) 0, (optimality of x⇤1 )
'01(x1;A11,A12) = 0, (C 1-pasting in x1)
'1(x1;A11,A12) = '1(x⇤1 ;A11,A12)� c , (C 0-pasting in x1)'1(x2;A11,A12) = '1(x⇤2 ;A11,A12) + c , (C 0-pasting in x2)
8>>><
>>>:
'02(x
⇤2 ;A21,A22) = 0, '00
2 (x⇤2 ;A21,A22) 0, (optimality of x⇤2 )
'02(x2;A21,A22) = 0, (C 1-pasting in x2)
'2(x1;A21,A22) = '2(x⇤1 ;A21,A22) + c , (C 0-pasting in x1)'2(x2;A21,A22) = '2(x⇤2 ;A21,A22)� c , (C 0-pasting in x2)
(x1 < x
⇤1 < x2, (order condition 1)
x1 < x⇤2 < x2, (order condition 2)
where 'i (x ; a, a)=ae✓x+ae
�✓x+(�1)i+1(x � si )/⇢ and ✓=p
2⇢/�.Notice: 8 equations and 8 parameters. We here have explicit solutions!
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Step 3: equilibrium. After checking of the assumptions, we finallyapply the theorem and use V1, V2 to get Nash equilibria.
PropositionA family of Nash equilibria is characterized as follows (s 2 R).
Player i intervenes if and only if the process hits xi , where
x2, x1 = s ±1
✓log
"s⌘ + ⇠⌘ � ⇠
s✓(c � c)
4⇠+ 1 +
s✓(c � c)
4⇠
!#.
When intervening, player i shifts the process to the state x⇤i , where
x⇤2 , x
⇤1 = s ±
1
✓log
"s1 � ⇠1 + ⇠
s✓(c � c)
4⇠+ 1 +
s✓(c � c)
4⇠
!#.
Here, ✓=p
2⇢� and ⇠ is the unique zero of F (x)=2x+✓c�log
⇣1+x1�x
⌘.
Explicit expressions for both the value functions are available as well.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Step 3: equilibrium. After checking of the assumptions, we finallyapply the theorem and use V1, V2 to get Nash equilibria.
PropositionA family of Nash equilibria is characterized as follows (s 2 R).
Player i intervenes if and only if the process hits xi , where
x2, x1 = s ±1
✓log
"s⌘ + ⇠⌘ � ⇠
s✓(c � c)
4⇠+ 1 +
s✓(c � c)
4⇠
!#.
When intervening, player i shifts the process to the state x⇤i , where
x⇤2 , x
⇤1 = s ±
1
✓log
"s1 � ⇠1 + ⇠
s✓(c � c)
4⇠+ 1 +
s✓(c � c)
4⇠
!#.
Here, ✓=p
2⇢� and ⇠ is the unique zero of F (x)=2x+✓c�log
⇣1+x1�x
⌘.
Explicit expressions for both the value functions are available as well.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Example. Let us consider, as above, s1 = 0.9 and s2 = 1.1. Also,consider ⇢ = 0.02, � = 0.1, c = 0.5, c = 0.
Then, the solution is:
x1 = 0.814, x2 = 1.186, (Intervention points)
x⇤1 = 1.078, x
⇤2 = 0.922. (States after intervention)
0.814 0.922 1.078 1.186
This is what happens practically (assume X0 = 1).As long as the state belongs to ]x1, x2[, no one intervenes.When the state reaches x1 or x2, one of the player intervenes:
- if Xt = x1, player 1 intervenes and moves the state to x⇤1 ,
- if Xt = x2, player 2 intervenes and moves the state to x⇤2 .
...and so on.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Example. Let us consider, as above, s1 = 0.9 and s2 = 1.1. Also,consider ⇢ = 0.02, � = 0.1, c = 0.5, c = 0. Then, the solution is:
x1 = 0.814, x2 = 1.186, (Intervention points)
x⇤1 = 1.078, x
⇤2 = 0.922. (States after intervention)
0.814 0.922 1.078 1.186
This is what happens practically (assume X0 = 1).As long as the state belongs to ]x1, x2[, no one intervenes.When the state reaches x1 or x2, one of the player intervenes:
- if Xt = x1, player 1 intervenes and moves the state to x⇤1 ,
- if Xt = x2, player 2 intervenes and moves the state to x⇤2 .
...and so on.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Example. Let us consider, as above, s1 = 0.9 and s2 = 1.1. Also,consider ⇢ = 0.02, � = 0.1, c = 0.5, c = 0. Then, the solution is:
x1 = 0.814, x2 = 1.186, (Intervention points)
x⇤1 = 1.078, x
⇤2 = 0.922. (States after intervention)
0.814 0.922 1.078 1.186
This is what happens practically (assume X0 = 1).As long as the state belongs to ]x1, x2[, no one intervenes.When the state reaches x1 or x2, one of the player intervenes:
- if Xt = x1, player 1 intervenes and moves the state to x⇤1 ,
- if Xt = x2, player 2 intervenes and moves the state to x⇤2 .
...and so on.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Here the value function of player 2:
notice the three parts, thesmooth pasting in x1, the maximum point in x
⇤2 .
0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.42.0
2.1
2.2
2.3
2.4
2.5
x1 x⇤2 x⇤
1 x2
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Here the value function of player 2: notice the three parts,
thesmooth pasting in x1, the maximum point in x
⇤2 .
0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.42.0
2.1
2.2
2.3
2.4
2.5
x1 x⇤2 x⇤
1 x2
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Here the value function of player 2: notice the three parts, thesmooth pasting in x1,
the maximum point in x⇤2 .
0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.42.0
2.1
2.2
2.3
2.4
2.5
x1 x⇤2 x⇤
1 x2
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Here the value function of player 2: notice the three parts, thesmooth pasting in x1, the maximum point in x
⇤2 .
0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.42.0
2.1
2.2
2.3
2.4
2.5
x1 x⇤2 x⇤
1 x2
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Here the value function of player 2: notice the three parts, thesmooth pasting in x1, the maximum point in x
⇤2 . And of player 1.
0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.42.0
2.1
2.2
2.3
2.4
2.5
x1 x⇤2 x⇤
1 x2
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Properties. The explicit formulas for xi , x⇤i let us state someproperties w.r.t. the parameters c (interv. cost) and c (interv. gain).
As c ! 0+ (interv. cost to zero, assume c = 0), thecontinuation region shrinks and finally collapses to {s}.Reasonable: small costs imply frequent interventions, in the limitsituation the controlled process gets constant.
As c ! +1 (interv. cost to 1), the continuation regionexpands and finally coincides with R.Reasonable: big costs imply rare interventions, in the limit situationthe players never intervene.
If c= c>0, we have x⇤i = xj : infinite sequence of simultaneous
interventions, the Nash strategies are not admissible.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Properties. The explicit formulas for xi , x⇤i let us state someproperties w.r.t. the parameters c (interv. cost) and c (interv. gain).
As c ! 0+ (interv. cost to zero, assume c = 0), thecontinuation region shrinks and finally collapses to {s}.Reasonable: small costs imply frequent interventions, in the limitsituation the controlled process gets constant.
As c ! +1 (interv. cost to 1), the continuation regionexpands and finally coincides with R.Reasonable: big costs imply rare interventions, in the limit situationthe players never intervene.
If c= c>0, we have x⇤i = xj : infinite sequence of simultaneous
interventions, the Nash strategies are not admissible.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Properties. The explicit formulas for xi , x⇤i let us state someproperties w.r.t. the parameters c (interv. cost) and c (interv. gain).
As c ! 0+ (interv. cost to zero, assume c = 0), thecontinuation region shrinks and finally collapses to {s}.Reasonable: small costs imply frequent interventions, in the limitsituation the controlled process gets constant.
As c ! +1 (interv. cost to 1), the continuation regionexpands and finally coincides with R.Reasonable: big costs imply rare interventions, in the limit situationthe players never intervene.
If c= c>0, we have x⇤i = xj : infinite sequence of simultaneous
interventions, the Nash strategies are not admissible.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
When N ! 1: mean-field games with impulse controls
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Idea. Let us consider impulse games where the players’ payoffsdepend on the average of the processes X
1, . . . ,XN :
Ji (x ;') = Ex
Z 1
0e�⇢s
fi
✓Xs ,
1N
NX
j=1
Xjs
◆ds +
X
k�1
e�⇢⌧k�i (�i,k)
�,
dXs = b(Xs)ds + �(Xs)dWs + impulses.
For example, each player may want to keep her own process closeto the average 1
N
PNj=1 X
js .
Very complicated to solve...
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Mean-field games. N-player games are complicated to solve. Idea:N = 1 (! stationary mean-field games with impulse controls).
J(x ;') = Ex
Z 1
0e�⇢s
f (Xs ,m)ds +X
k�1
e�⇢⌧k�(�k)
�,
dXs = b(Xs)ds + �(Xs)dWs + impulses,
m = lims!1
Ex [Xs ].
Step 1. For a fixed m 2 R, find the opt. strategy '⇤ = '⇤(m).
Step 2. Let �(m) := lims!1 Ex [Xx,'⇤(m)s ].
Step 3. Solve the fixed point problem �(m) = m.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
Mean-field games. N-player games are complicated to solve. Idea:N = 1 (! stationary mean-field games with impulse controls).
J(x ;') = Ex
Z 1
0e�⇢s
f (Xs ,m)ds +X
k�1
e�⇢⌧k�(�k)
�,
dXs = b(Xs)ds + �(Xs)dWs + impulses,
m = lims!1
Ex [Xs ].
Step 1. For a fixed m 2 R, find the opt. strategy '⇤ = '⇤(m).
Step 2. Let �(m) := lims!1 Ex [Xx,'⇤(m)s ].
Step 3. Solve the fixed point problem �(m) = m.
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
A class of problems. For suitable ↵ (e.g., a contraction), consider:
J(x ;') = Ex
Z 1
0e�⇢s
|Xs � ↵(m)|ds +X
k�1
e�⇢⌧k (K + k |�k |)
�,
dXs = �dWs + impulses, m = lims!1
Ex [Xs ].
Following the procedure above (Steps 1-2-3), we get the followingoptimal control.
Let m be the (unique) fixed point of a suitable contraction. Letu,U be determined by a suitable system of algebraic equations.
The controller intervenes when |Xs � ↵(m)| � u.
If Xs � ↵(m) � u (resp. �u), she moves the process so thatXs � ↵(m) = U (resp. = �U).
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
A class of problems. For suitable ↵ (e.g., a contraction), consider:
J(x ;') = Ex
Z 1
0e�⇢s
|Xs � ↵(m)|ds +X
k�1
e�⇢⌧k (K + k |�k |)
�,
dXs = �dWs + impulses, m = lims!1
Ex [Xs ].
Following the procedure above (Steps 1-2-3), we get the followingoptimal control.
Let m be the (unique) fixed point of a suitable contraction. Letu,U be determined by a suitable system of algebraic equations.
The controller intervenes when |Xs � ↵(m)| � u.
If Xs � ↵(m) � u (resp. �u), she moves the process so thatXs � ↵(m) = U (resp. = �U).
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
N vs. MFG. Let us compare the N-player and 1-player problems.
(N) V i (x) = J i (x ;'⇤) J i (x ; si,'i ('⇤)), 8i , 'i (Nash eq.)
'⇤ = ('⇤1 , . . . ,'
⇤N), '⇤
i = (A⇤i , ⇠
⇤i ) (action region/function)
J i (x ;') = Ex
Z 1
0e�⇢s
����Xis �
1
N
NX
j=1
X js
����ds +X
k�1
e�⇢⌧i,kK
�
(MFG) VMFG (x) = JMFG (x ;'⇤) = inf'
JMFG (x ;') (optimal ctrl.)
'⇤ = (A⇤, ⇠⇤) (action region/function)
JMFG (x ;')=Ex
Z 1
0e�⇢s
|Xs �m|ds +X
k�1
e�⇢⌧kK
�, m= lim
sEx [Xs ]
Question: does the MFG solution ’provide’ a good approximation ofthe N-player solution?
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
N vs. MFG. Let us compare the N-player and 1-player problems.
(N) V i (x) = J i (x ;'⇤) J i (x ; si,'i ('⇤)), 8i , 'i (Nash eq.)
'⇤ = ('⇤1 , . . . ,'
⇤N), '⇤
i = (A⇤i , ⇠
⇤i ) (action region/function)
J i (x ;') = Ex
Z 1
0e�⇢s
����Xis �
1
N
NX
j=1
X js
����ds +X
k�1
e�⇢⌧i,kK
�
(MFG) VMFG (x) = JMFG (x ;'⇤) = inf'
JMFG (x ;') (optimal ctrl.)
'⇤ = (A⇤, ⇠⇤) (action region/function)
JMFG (x ;')=Ex
Z 1
0e�⇢s
|Xs �m|ds +X
k�1
e�⇢⌧kK
�, m= lim
sEx [Xs ]
Question: does the MFG solution ’provide’ a good approximation ofthe N-player solution?
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
The solution to the MFG is characterized by (�u⇤,�U
⇤,U⇤, u⇤)(±U
⇤ intervention threshold, ±u⇤ new state after intervention).
Consider now the following strategy '⇤ for the N-player game.
If X is �m
⇤� u
⇤ (resp. �u⇤), player i moves the process so
that X is �m
⇤ = U⇤ (resp. = �U
⇤).Suitable priority rules in case of simultaneous interventions.
The following result holds.
PropositionThe strategies '⇤, described as above, form a "-Nash equilibriumfor the N-player game, where " = O( 1p
N).
Notice: "-Nash means that J i (x ;'⇤) Ji (x ; si ,'i ('
⇤)) + ".
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
The solution to the MFG is characterized by (�u⇤,�U
⇤,U⇤, u⇤)(±U
⇤ intervention threshold, ±u⇤ new state after intervention).
Consider now the following strategy '⇤ for the N-player game.
If X is �m
⇤� u
⇤ (resp. �u⇤), player i moves the process so
that X is �m
⇤ = U⇤ (resp. = �U
⇤).Suitable priority rules in case of simultaneous interventions.
The following result holds.
PropositionThe strategies '⇤, described as above, form a "-Nash equilibriumfor the N-player game, where " = O( 1p
N).
Notice: "-Nash means that J i (x ;'⇤) Ji (x ; si ,'i ('
⇤)) + ".
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
1. Nonzero-sum impulse games
1.1 Naturally arise in energy finance but never studied
1.2 Our model: strategies ! impulse controls ! controlled process
1.3 Payoff: running cost, intervention costs, intervention gains, terminal cost
2. Verification theorem
2.1 Sufficient conditions to characterize the value functions
2.2 Fundamental assumptions: QVI problem + regularity conditions
2.3 Key-points for the QVI problem: operators MiVi and Hi,jVi
3. Examples and extension to MFG
3.1 Example: two players with linear payoff control a one-dimensional BM
3.2 Explicit formulas and asymptotic properties
3.3 Extension to MFG
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
1. Nonzero-sum impulse games2. Verification theorem
3. Examples and extension to MFG
A. Bensoussan, A. Friedman,Non-zero sum stochastic differential games with stopping timesand free boundary problems,Trans. Amer. Math. Society 231 (1977), no. 2, 275–327.
A. Cosso,Stochastic differential games involving impulse controls anddouble-obstacle quasi-variational inequalities,SIAM J. Control Optim. 51 (2013), no. 3, 2102–2131.
A. Friedman,Stochastic games and variational inequalities,Arch. Rational Mech. Anal. 51 (1973), no. 5, 321–346.