Game Theory - MAT UPC Theory.pdf · Game Theory Applications to Partial Differential Equations Fernando Charro Caballero Department of Mathematics The University of Texas at Austin

Differential equations and probabilityBrownian Motion and the Laplacian

Tug-of-war games and the Infinity Laplacian

Game TheoryApplications to Partial Differential Equations

Fernando Charro Caballero

Department of MathematicsThe University of Texas at Austin

San Juan, April 15th 2011

Fernando Charro Caballero Game Theory



1 Differential equations and probabilityRandom walks in 1-dRandom Tug-of-War games in 1-d

2 Brownian Motion and the Laplacian

3 Tug-of-war games and the Infinity LaplacianThe concept of game value and perfect rational playersRandom Tug-of-War




Random walks in 1-dRandom Tug-of-War games in 1-d

Random walk in a segment

Consider a movement by random increments of step ε > 0 smallstarting at a point x0 ∈ [−1, 1],:

From x0 the particle can jump either to x0 + ε or x0 − ε at random withprobability 1/2.

Let uε(x0) be the probability of reaching x = 1 starting at x0.

Applying conditional expectations we get,uε(x) =

12

[uε(x + ε) + uε(x − ε)

], x ∈ (−1, 1),

uε(−1) = 0,

uε(1) = 1.





Taylor’s expansions yield,

uε(x + ε) = uε(x) + ε u′ε(x) +ε2

2u′′ε (x) + o(ε2)

uε(x − ε) = uε(x)− ε u′ε(x) +ε2

2u′′ε (x) + o(ε2)

Hence,

0 = uε(x + ε) + uε(x − ε)− 2uε(x) = ε2 u′′ε (x) + o(ε2).

Assuming uniform convergence and letting ε→ 0 we get the differentialequation,

u′′(x) = 0, x ∈ (−1, 1),

u(−1) = 0,

u(1) = 1.





Random Tug-of-War

Two players are in contest and the total earnings of one are the lossesof the other (two-person, zero-sum game).

Player I plays trying to maximize his expected outcome.

Player II is trying to minimize Player I’s outcome (and maximize its own).

Rules of the game

A token is placed at the starting point x0 ∈ (−1, 1).

A fair coin is tossed and the winner chooses a new positionx1 ∈ [x0 − ε, x0 + ε] ∩ [−1, 1].

At each turn, the coin is tossed again, and the winner chooses a newgame state xk ∈ [xk−1 − ε, xk−1 + ε] ∩ [−1, 1].

Game ends when the token reaches x = ±1.

At x = 1, Player I receives $1 from Player II and at x = −1, pays $1 toPlayer II.





Denote by uε(x) the expected value of this game for player I, whenstarted from x .

Dynamic Programming Principle (DPP):

uε(x) =12

[max|x−y|≤ε

uε(y) + min|x−y|≤ε

uε(y)]

The obvious strategies for I and II turn out to be optimal:

I aims towards x = 1 (where would receive $1 from II).II aims towards x = −1 (where would receive $1 from I).

Hence, the DPP can be rewritten,

uε(x) =12

[uε(x + ε) + uε(x − ε)

].

Arguing as before, it leads to the differential equation u′′(x) = 0.




Brownian motion

Random movement of a particle immersed in a fluid.

It originated in 1827, when the botanist Robert Brown observed thistype of random movement in pollen particles suspended in water.

Einstein in a famous paper of 1905 (same year of Relativity) observedthat heat diffusion could be explained via Brownian motion of particles.

Can be constructed as a limit case of random walks.




Random walk in a lattice:

Approximate the movement by randomincrements of step ε > 0 (small) in eachof the axes directions.

From (x , y) the particle has probability1/4 to jump at random to either (x + ε, y),(x − ε, y), (x , y + ε), or (x , y − ε).uε(x , y) is the probability that a particlestarting at (x , y) hits a given window Γ.

Applying conditional expectations we get,

uε(x , y) =14

[uε(x + ε, y) + uε(x − ε, y)

+uε(x , y + ε) + uε(x , y − ε)].








uε(x , y) =14

[uε(x + ε, y) + uε(x − ε, y)

+uε(x , y + ε) + uε(x , y − ε)].








uε(x , y) =14

[uε(x + ε, y) + uε(x − ε, y)

+uε(x , y + ε) + uε(x , y − ε)].




Rearranging, and after some Taylor’s expansions,

0 =uε(x + ε, y) + uε(x − ε, y) + uε(x , y + ε) + uε(x , y − ε)− 4uε(x , y)

=(

uε(x , y)+ε∂uε∂x

(x , y) +ε2

2∂2uε∂x2 (x , y)

)+(

uε(x , y)−ε∂uε∂x

(x , y) +ε2

2∂2uε∂x2 (x , y)

)+(

uε(x , y)+ε∂uε∂y

(x , y) +ε2

2∂2uε∂y2 (x , y)

)+(

uε(x , y)−ε∂uε∂y

(x , y) +ε2

2∂2uε∂y2 (x , y)

)−4uε(x , y) + o(ε2)

=ε2(∂2uε∂x2 (x , y) +

∂2uε∂y2 (x , y)

)+ o(ε2) = ε2∆uε(x , y) + o(ε2).

Assuming uniform convergence as ε→ 0,∆u(x , y) = 0 in Ω

u = 1 on Γ

u = 0 on ∂Ω \ Γ.




Rearranging, and after some Taylor’s expansions,

0 =uε(x + ε, y) + uε(x − ε, y) + uε(x , y + ε) + uε(x , y − ε)− 4uε(x , y)

=(

uε(x , y)+ε∂uε∂x

(x , y) +ε2

2∂2uε∂x2 (x , y)

)+(

uε(x , y)−ε∂uε∂x

(x , y) +ε2

2∂2uε∂x2 (x , y)

)+(

uε(x , y)+ε∂uε∂y

(x , y) +ε2

2∂2uε∂y2 (x , y)

)+(

uε(x , y)−ε∂uε∂y

(x , y) +ε2

2∂2uε∂y2 (x , y)

)−4uε(x , y) + o(ε2)

=ε2(∂2uε∂x2 (x , y) +

∂2uε∂y2 (x , y)

)+ o(ε2) = ε2∆uε(x , y) + o(ε2).

Assuming uniform convergence as ε→ 0,∆u(x , y) = 0 in Ω

u = 1 on Γ

u = 0 on ∂Ω \ Γ.




∆ =n∑

j=1

∂2

∂x2j

is the Laplacian operator.

(After Pierre-Simon, marquis de Laplace 1749-1827).

Paradigm of elliptic, linear equation: ∆(u + v) = ∆u + ∆v .

Applications in describing many physical phenomena:

Electric and gravitational potentialsDiffusion equation for heat and fluid flowWave propagationQuantum mechanics, etc.

Other examples of elliptic operators?




∆ =n∑

j=1

∂2

∂x2j










∆ =n∑

j=1

∂2

∂x2j










∆ =n∑

j=1

∂2

∂x2j










More general, but still linear equations (the sum of solutions is a solution)

The Laplacian: ∆u = div(∇u) = trace(D2u).

Same models but in inhomogeneous or anisotropic media yield,

div(A(x)∇u) =n∑

i,j=1

(aij(x)uxi )xj

trace(A(x)D2u) =n∑

i,j=1

aij(x)uxi xj

with A(x) a symmetric matrix such that,

0 ≤ θ(x)|ξ|2 ≤ 〈A(x)ξ, ξ〉 ≤ Θ|ξ|2, ∀ξ ∈ Rn.

This is called an ellipticity condition.




More general, but still linear equations (the sum of solutions is a solution)

The Laplacian: ∆u = div(∇u) = trace(D2u).

Same models but in inhomogeneous or anisotropic media yield,

div(A(x)∇u) =n∑

i,j=1

(aij(x)uxi )xj

trace(A(x)D2u) =n∑

i,j=1

aij(x)uxi xj

with A(x) a symmetric matrix such that,

0 ≤ θ(x)|ξ|2 ≤ 〈A(x)ξ, ξ〉 ≤ Θ|ξ|2, ∀ξ ∈ Rn.

This is called an ellipticity condition.




Nonlinear equations (the sum of solutions is not necessarily a solution)

The p-Laplacian (1 < p <∞):

∆pu = div(|∇u|p−2∇u)

= trace[|∇u|p−2

(Id + (p − 2)

∇u ⊗∇u|∇u|2

)D2u

].

The∞-Laplacian:

∆∞u =⟨

D2u∇u|∇u| ,

∇u|∇u|

⟩= trace

[(∇u ⊗∇u|∇u|2

)D2u

]=

1|∇u|2

n∑i,j=1

uxi xj uxi uxj .

The Monge-Ampère operator: det(D2u).




The concept of game value and perfect rational playersRandom Tug-of-War

Question:

What is the equation associated to Tug-of-War games in 2 or moredimensions?

Is it the Laplacian, as in the 1-dimensional case?

The answer is "No".

The correct equation is the∞−Laplacian:

∆∞u =⟨

D2u∇u|∇u| ,

∇u|∇u|

⟩= trace

[(∇u ⊗∇u|∇u|2

)D2u

]=

1|∇u|2

n∑i,j=1

uxi xj uxi uxj .





Question:

What is the equation associated to Tug-of-War games in 2 or moredimensions?

Is it the Laplacian, as in the 1-dimensional case?

The answer is "No".

The correct equation is the∞−Laplacian:

∆∞u =⟨

D2u∇u|∇u| ,

∇u|∇u|

⟩= trace

[(∇u ⊗∇u|∇u|2

)D2u

]=

1|∇u|2

n∑i,j=1

uxi xj uxi uxj .





Important notions in game theory:

Game value.

Rational (perfect) players.

Equilibrium.

W. Poundstone, (1992) Prisoner’s Dilemma.





The car sale

Bob (buyer) wants to buy a used car from Sam (seller), who is movingout.

They want to agree on a fair price for the car.

As they know a used-car dealer, they take the car to the dealer and askhim his selling and buying price for the car in "as is" condition. Bob andSam plan on splitting the difference between them.

Let’s say the dealer’s buying price was $5, 000. Sam could, if hewanted, sell the car to the dealer for that price.

Likewise, Bob could buy a car just as good as Sam’s from the dealer forthe dealer’s selling price, say, $8,000.

By not going through the dealer, Bob and Sam have an extra $3,000 tosplit between themselves.

How should Bob and Sam split the $3,000 profit?





The car sale












The car sale












The car sale












The car sale: Analysis

They could divide it right down the middle.

Sam’s would get $6,500 for his car, instead of $5,000.Bob would get an $8,000 car for only $6,500.

Sounds fair.

But Bob could be obstinate and insist that he won’t pay any more than$6,000... or $5,500, or $5,250, or even $5,001.

Sam could tell him to go take a hike, but still, the dealer will only givehim $5,000. He’s hurting himself by not accepting Bob’s offer.

It works the other way, too. Sam can be just as obstinate and name aprice near the dealer’s selling price.

The peculiar thing is that the party that is more unreasonable is apt toget the better of the deal.








Sounds fair.












Sounds fair.









Prisoner’s Dilemma

Two suspects are arrested by the police.

The police have insufficient evidence for a conviction, and, havingseparated the prisoners, visit each of them to offer the same deal.

If one testifies for the prosecution against the other (defects) and theother remains silent (cooperates), the defector goes free and the silentaccomplice receives the full 10-year sentence.

If both remain silent, both prisoners are sentenced to only 6 months injail for a minor charge.

If each betrays the other, each receives a 5-year sentence.

Each one is assured that the other would not know about the betrayalbefore the end of the investigation.

Each prisoner must choose to betray the other or to remain silent.

How should the prisoners act?































Prisoner’s Dilemma: Analysis

The game has a Nash equilibrium when both prisoners/players defect(no one can improve his outcome by changing his choice unilaterally).

B stays silent(cooperates)

B betrays(defects)

A stays silent(cooperates) Each serves 6 months A serves 10 years

B goes freeA betrays(defects)

A goes freeB serves 10 years Each serves 5 years

The dilemma: Cooperation is desirable for both players, but a rationalplayer should always defect.





Russian roulette

The player place a single round in a revolver (6-chambered gun), spinthe cylinder, and place the muzzle against his/her head.

The player is offered $10k for pulling the trigger once, and then, forevery extra time he/she pulls the trigger without re-spinning is offered10 times his/her stakes at the moment. That is,

Play once: $10k.Play twice: $100kPlay three times: $1 million.Play four times: $10 million.Play five times (the maximum): $100 million.

The player will only receive the money if survives the game.

Would you play? Why not?

We assign value to a game in terms of the worst expected outcome.





Perfect "rational" player

Greedy

Paranoid

Pessimistic

Fearful





Now we can go back to Tug-of-War.





Random Tug-of-War

Two-person, zero-sum game: two players are in contest and the totalearnings of one are the losses of the other.

Player I plays trying to maximize his expected outcome.

Player II is trying to minimize Player I’s outcome.

Rules of the game

Ω ⊂ Rn, bounded domain; ∂Ω = ΓD ∪ ΓN .

Final payoff function: F : ΓD → R.

Starting point x0 ∈ Ω \ ΓD . A fair coin is tossed and the winner choosesa new position x1 ∈ Bε(x0) ∩ Ω.

At each turn, the coin is tossed again, and the winner chooses a newgame state xk ∈ Bε(xk−1) ∩ Ω.

Game ends when xτ ∈ ΓD , and Player I earns F (xτ ) (paid by Player II).





For a given game sequence x0, x1, · · · , xN, where xN ∈ ΓD , Player I’soutcome is F (xN).

Notice that the sequence x0, x1, · · · , xN has some probability, whichdepends on:

The starting point x0.The strategies of players, SI and SII .

Expected outcome

When we fix the initial value and both player’s strategies, Player I’s expectedoutcome is:

Ex0SI ,SII

[F (xN)

],

with the expectation associated to the probability measure induced by theinitial value and the strategies.





Perfect rational players

Player I’s strategy allows to choose at every step the game position thatmaximizes his expected outcome.

Player II’s strategy allows to choose at every step the game positionthat minimizes Player I’s expected outcome (or maximize his own).

Game value

Extremal cases:

Player I expects to earn at least:

uI(x) = supSI

infSII

ExSI ,SII

[F (xN)

]Player II expects that Player I will earn at most:

uII(x) = infSII

supSI

ExSI ,SII

[F (xN)

]By definition uI ≤ uII . When uI = uII , the game has a (discrete) value.





Theorem

Under very general hypotheses, the game has a value.

Peres-Schram-Sheffield-Wilson: "Tug of war and the infinity-Laplacian"

Main Property (Dynamic Programming Principle)

The value of the game satisfies:

u(x) =12

[max|x−y|≤ε

u(y) + min|x−y|≤ε

u(y)]





0 = max|x−y|≤ε


u(y)− 2u(x)

=u(

x + ε∇u(x)

|∇u(x)| + o(1)

)+u(

x − ε ∇u(x)

|∇u(x)| + o(1)

)− 2u(x)

=u(x) + ε|∇u(x)|+ ε2

2

⟨D2u(x)

∇u|∇u| ,

∇u|∇u|

⟩+ o(ε2)+

u(x)− ε|∇u(x)|+ ε2

2

⟨D2u(x)

∇u|∇u| ,

∇u|∇u|

⟩+ o(ε2)− 2u(x)

=ε2∆∞u(x) + o(ε2)

Theorem [Peres-Schramm-Sheffield-Wilson]

Existence and uniqueness of the limit of the values of ε-Tug-of-war gamesMoreover, the limit satisfies the problem,

∆∞u = 0 in Ω

u = F on ∂Ω.





Probability and PDE meet at the Mean Value formulas:

u(x , y) =14

[u(x + ε, y) + u(x − ε, y) + u(x , y + ε) + u(x , y − ε)

]Linear.

Neither the probabilistic argument or the second derivatives in theLaplacian privilege directions.

u(x) =12

[max|x−y|≤ε


u(y)]

Nonlinear.

By using game theory, we are able to introduce intentionality in theprobabilistic argument. As a consequence, the∞−Laplacian is thesecond derivative in the direction of the gradient.





Gracias!


Documents

Game Theory - MAT UPC Theory.pdf · Game Theory Applications to Partial Differential Equations Fernando Charro Caballero Department of Mathematics The University of Texas at Austin