No man is an island - UniTrentobagagiol/Talk-TrentoMFG.pdf · No man is an island No man is an Iland, intire of itselfe; every man is a peece of the Continent, a part of the maine

Mean field gamesExamples, (O. Gueant et al. 2011)

Robust mean field games

No man is an island

No man is an Iland, intire of itselfe;every man is a peece of the Continent, apart of the maine . . .

MEDITATION XVII Devotions uponEmergent Occasions John Donne

Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games



Who?




Robust Mean Field Games

Fabio & Dario H. Tembine2 Q. Zhu3 T. Basar3

2Supelec

3University of Illinois Urbana-Champaign

Padua, 22 March 2013




Outline

Mean field gamesIntroduction

Examples, (O. Gueant et al. 2011)Mexican waveLarge populationOil production

Robust mean field gamesOil production with uncertainty...generalizing



Robust mean field gamesIntroduction

Advection

I N →∞ homogeneous agents withdynamics

x(t) = u(x(t), t), x0 ∈ Rn

I u(x, t) is vector field in Rn

I density m(x, t) in x evolvesaccording to advection equation

∂tm+div(m·u(x, t)) = 0, in Rn × [0, T ]




Mean field games

I Agents wish to minimize∫ T0

[ 1

2|u(x(t), t)|2︸︷︷︸

penalty on control

+ g(x(t),m(·, t))︸︷︷︸...on state & distribution

]dt+G(x(T ),m(·, T ))︸︷︷︸

...on final state

I opt. control u(x(t), t) = −∇xJ(x(t), t), [J(., .) is opt. cost]

I coupled partial differential equations in Rn × [0, T ]:

−∂tJ + 12 |∇xJ |2 = g(x,m)

u

��

(HJB) - backward

∂tm+ div(m · u(x)) = 0

m

TT

(advection) - forward

I boundary conditionsm(·, 0) = m0, J(x, T ) = G(x,m(·, T ))




Hamilton Jacobi Bellman

I From Bellman

J(x0, t0)︸︷︷︸today’s cost

= minu [1

2|u0|2 + g(x0,m0)]dt︸︷︷︸

stage cost

+ J(x0 + dx, t0 + dt)︸︷︷︸future cost

I Taylor expanding future cost

J(x0 + dx, t0 + dt) = J(x0, t0) + ∂tJdt+∇xJxdt

I minu [1

2|u|2 + g(x,m) + ∂tJ +∇xJ

u︷︸︸︷x ]︸︷︷︸

Hamiltonian

= 0 (drop index 0)

I optimal control u = −∇xJ yields

−∂tJ +1

2|∇xJ |2 = g(x,m) HJB




Stochastic differential game

I stochastic dynamics is

dx = udt+ σdBt

I dBt infinitesimal Brownian motion

I Mean field games (∆ =∑n

i=1∂2

∂x2i

Laplacian)

−∂tJ + 12 |∇xJ |2 − σ2

2 ∆J = g(x,m)

u��

(HJB)-backward

∂tm+ div(m · u(x))− σ2

2 ∆m = 0

m

TT

(Kolmogorov)-forward




Hamilton Jacobi Bellman (with dx = udt+ σdBt)I From Bellman

J(x0, t0)︸︷︷︸today’s cost

= minu [1

2|u|2 + g(x,m)]dt︸︷︷︸

stage cost

+ EJ(x0 + dx, t0 + dt)︸︷︷︸exp. future cost

I Taylor expanding future cost (EdBt = 0, EdB2t → dt)

J(x0+dx, t0+dt) = J(x0, t0)+∂tJdt+E∇xJdx︸︷︷︸∇xJudt

+ E1

2dx′∇2

xJdx︸︷︷︸σ2

2∆JEdB2

t

I minu [1

2|u|2 + g(x,m) + ∂tJ +∇xJu+

σ2

2∆J ]︸︷︷︸

Hamiltonian

= 0

I optimal control u(t) = −∇xJ yields

−∂tJ +1

2|∇xJ |2 −

σ2

2∆J = g(x,m) HJB




Average cost (ergodic)

I J = E lim supT→∞1T

∫ T0

[12 |u(t)|2 + g(x(t),m(·, t))

]dt


i=1∂2

∂x2i

Laplacian), solve in Rn

λ+ 12 |∇xJ |2 − σ2

2 ∆J = g(x, m)

u��

(HJB)

div(m · u(x))− σ2

2 ∆m = 0

m

TT

(Kolmogorov)




Discounted cost

I J = E∫∞

0 e−ρt[

12 |u(t)|2 + g(x(t),m(·, t))

]dt


i=1∂2

∂x2i

Laplacian), solve in

Rn × [0, T ]

−∂tJ + 12 |∇xJ |2 − σ2

2 ∆J + ρJ = g(x,m)

u��

(HJB)


2 ∆m = 0

m

TT

(Kolmogorov)




Mexican waveLarge populationOil production

Mexican wave (mimicry & fashion)

1z

yL

m(y, t)

0

I state x = [y, z], y ∈ [0, L) coordinate, z position:

z =

{1 standing0 seated

, z ∈ (0, 1) intermediate

I dynamics dz = udt (u control)I penalty on state and distribution g(x,m) =

Kzα(1− z)β︸︷︷︸comfort

+1

ε2

∫(z − z)2m(y; t, z)

1

εs(y − yε

)dzdy︸︷︷︸mimicry





Meeting starting time (coordination with externality)

−xmaxx0

m(x, t)

I dynamics dxi = uidt+ σdBt

I τi = mins(xi(s) = 0) arrival time, ts scheduled time,t actual starting time

I penalty on final state and distributionG(x(τi),m(·, τi)) = c1[τi − ts]+︸︷︷︸

reputation

+ c2[τi − t]+︸︷︷︸inconvenience

+ c3[t− τi]+︸︷︷︸waiting

I people arrived up to time s: F (s) = −∫ s

0 ∂xm(0, v)dv

I starting time t = F−1(θ), (θ is quorum)





Large population (herd behaviour)

I behaviour dynamicsdxi = uidt+ σdBt

I penalty

g(x,m) = β(x−∫ym(y, t)dy︸︷︷︸average

)2

I discounted cost J = E∫∞

0 e−ρt[

12 |u(t)|2 + g(x(t),m(·, t))

]dt

I mean field game with discounted cost

−∂tJ + 12 |∇xJ |2 − σ2

2 ∆J + ρJ = g(x,m)

u��

(HJB)


2 ∆m = 0

m

TT

(Kolmogorov)





Oil production

I stock market model,

dx = [αtx+ βtu] dt+ σtxdBt

I βtu produced quantity

I penalty (- total income + production costs)

g(x, u,m) = −h(m)u+ [a

2u2 + bu]

I h(m) is sale price of oil (decreasing in m)

I penalty on final state accounts for unexploited reserve:

G(x(T )) = φ|x(T )|2, φ > 0




Oil production with uncertainty...generalizing

Uncertain inflation or taxation

I stock market model,

dx = [αtx+ βtu+ σtζ] dt+ σtxdBt

I σtζ taxation or inflation on theproduction

I penalty (- total income + production costs):

g(x, u,m, ζ) = −h(m, ζ)u+ [a

2u2 + bu]

I cost under worst disturbance [Basar, Bernhard, 1995]:

inf{u}t

sup{ζ}t

E(G(x(T )) +

∫ T

0g(x, u,m, ζ)dt−γ2

∫ T

0|ζ|2dt

).





coupled PDEs

∂tJ +Ht(x, ∂xJ,m) +

(σt2γ

)2

|∂xJ |2 +1

2σ2t x

2∂2xxJ = 0,

ζ∗ =σt

2γ2∂xJ, u∗ =

1

βt

[∂pHt(xt,

2γ2

σtζ∗t ,m)− αtxt

]

J,u∗,ζ∗

��m

XX

∂tm+ ∂x (m∂pHt(x, ∂xJ,m)) +σ2t

2γ2∂x(m∂xJ)− 1

2σ2t ∂

2xx

[x2m

]= 0





a new equilibrium concept

I worst-case disturbance feedback Nash equilibrium(H∞ literature)

I feedback mean field equilibrium (mean field gamesliterature)

⇓

I worst-case disturbance feedback mean fieldequilibrium





Simulations 1/3





Simulations 2/3





Simulations 3/3





Conclusions

I Mean field games require solving coupled partialdifferential equations (HJB-Kolmogorov)

I bring robustness within the picture and solve HJB underworst disturbance (robust mean field games)

I worst-case disturbance feedback mean field equilibrium





Main references

I J.-M. Lasry and P.-L. Lions. Mean field games. JapaneseJournal of Mathematics, 2(1), Mar. 2007.

I O. Gueant, J.-M. Lasry and P.-L. Lions. Mean field gamesand applications. Lect. notes in math., Springer 2011.

I T. Basar, P. Bernhard, H∞-Optimal Control and RelatedMinimax Design Problems: A Dynamic Game Approach.Birkhauser, Boston, MA, 1995

I H. Tembine, Q. Zhu, and T. Basar, Risk-sensitive meanfield stochastic games, IFAC WC 2011, Milano, Italy, 2011.

I M. Bardi, Explicit solutions of some Linear-QuadraticMean Field Games, Network and Heterog. Media, 2012.

I D. Bauso, H. Tembine, T. Basar, Robust Mean FieldGames with Application to Production of ExhaustibleResource. ROCOND ’12





Links

I Workshop on MFG and related topics (Rome, 2011)http://www.mat.uniroma1.it/ricerca/convegni/2011/mfg/

I Short course on MFG (videos and notes)http://www.ima.umn.edu/2012-2013/SW11.12-13.12/

I MFG Labhttp://mfglabs.com/





Questions?

Thank you!


Documents

No man is an island - UniTrentobagagiol/Talk-TrentoMFG.pdf · No man is an island No man is an Iland, intire of itselfe; every man is a peece of the Continent, a part of the maine