Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Games with Perfect Information
Yiling Chen
September 7, 2011
Non-Cooperative Game Theory
� What is it?
Mathematical study of interactions between rational andself-interested agents.
� Non-Cooperative
Focus on agents who make their own individual decisionswithout coalition
Perfect Information
� All players know the game structure.
� Each player, when making any decision, is perfectlyinformed of all the events that have previously occurred.
Definition of Normal-Form Game
� A finite n-person game, G =< N , A, u >
� N = {1, 2, ..., n} is the set of players.
� A = {A1, A2, ..., An} is a set of available actions.a = (a1, a2, ..., an) ∈ A is an action profile (or a purestrategy profile).
� u = {u1, u2, ...un} is a set of utility functions for n
agents.
� A strategy is a complete contingent plan that defines theaction an agent will take in all states of the world. Purestrategies (as opposed to mixed strategies) are the sameas actions of agents.
� Players move simultaneously.� Matrix representation for 2-person games
Example: Battle of the Sexes
� A husband and wife want to go to movies. They canselect between “Devils wear Parada” and ”Iron Man”.They prefer to go to the same movie, but while the wifeprefers “Devils wear Parada” the husband prefers “IronMan”. They need to make the decision independently.
10, 5 0, 0
0, 0 5, 10
DWP
IM
DWP IM
Wife
Husband
Example: Battle of the Sexes
� A husband and wife want to go to movies. They canselect between “Devils wear Parada” and ”Iron Man”.They prefer to go to the same movie, but while the wifeprefers “Devils wear Parada” the husband prefers “IronMan”. They need to make the decision independently.
10, 5 0, 0
0, 0 5, 10
DWP
IM
DWP IM
Wife
Husband
Best-Response Correspondences
� The best-response correspondence BRi(a−i) ∈ Ai are theset of strategies that maximizes agent i ’s utility givenother agents’ strategy a−i .
� Compute every agent’s best-response correspondences.The fixed points of the best-response correspondences,i.e. a
∗ ∈ BR(a∗), are the NEs of the game.
Example: Continuous Strategy Space
Cournot Competition
� Two suppliers producing a homogeneous good need tochoose their production quantity, qi and q2. The demandthat they are facing is p(Q) = 1000− Q, whereQ = q1 + q2. Unit cost c > 0.
� Utility: u1(q1, q2) = q1 ∗ [p(q1 + q2)− c]
� Best-response of supplier 1 isBR1(q2) = arg maxq1 u1(q1, q2)
Example: Matching Pennies
� Each of the two players has a penny. They independentlychoose to display either heads or tails. If the two penniesare the same, player 1 takes both pennies. If they aredifferent, player 2 takes both pennies.
1, -1 -1, 1
-1, 1 1, -1
Heads
Tails
Heads Tails
Mixed Strategies
� A mixed strategy of agent i , σi ∈ ∆(Ai), defines aprobability, σi(ai) for each pure strategy ai ∈ Ai .
� Agent i ’s expected utility of following strategy σi is
ui(σ) =�
a∈Ai
Pσ(a)ui(a)
� The support of σi is the set of pure strategies{ai : σ(ai) > 0}.
� Pure strategies are special cases of mixed strategies.
Mixed Strategy Nash Equilibrium
� Mixed-strategy prole σ∗ = {σ∗1, σ∗2, ...,σ∗n} is a Nashequilibrium in a game if, for all i ,
ui(σ∗i , σ
∗−i) ≥ ui(σi , σ
∗−i), ∀σi ∈ ∆(Ai).
� Theorem (Nash 1951): Every game with a finite numberof players and action profiles has at least one Nashequilibrium.
� All pure strategies in the support of agent i at a mixedstrategy Nash Equilibrium have the same expected utility.
Finding Mixed Strategy Nash Equilibrium
1, -1 -1, 1
-1, 1 1, -1
Heads
Tails
Heads Tails
� Find the fixed point of the
Best-Response
correspondences.
Fixed Point
p∗(r)
r∗(p)
Player 1 r
Heads
Tails
Tails Heads Player 2 p
Extensive-Form Game with Perfect Information
Example: The Sharing Game
� Alice and Bob try to split two indivisible and identical gifts.
First, Alice suggests a split: which can be “Alice keeps both”,
“they each keep one”, and “Bob keeps both”. Then, Bob
chooses whether to Accept or Reject the split. If Bob accepts
the split, they each get what the split specifies. If Bob rejects,
they each get nothing.
Alice
Bob Bob Bob
2-0 1-1 0-2
A R A R A R
(2,0) (0,0) (1,1) (0,0) (0,2) (0,0)
Loosely Speaking...
� Extensive Form� A detailed description of the sequential structure of
the decision problems encountered by the players in agame.
� Often represented as a game tree
� Perfect Information� All players know the game structure (including the
payoff functions at every outcome).
� Each player, when making any decision, is perfectlyinformed of all the events that have previouslyoccurred.
Def. of Perfect-Information Extensive-Form Games
� A perfect-information extensive-form game,G = (N , H , P , u)
� N = {1, 2, ..., n} is the set of players.
N={Alice, Bob}
Alice
Bob Bob Bob
2-0 1-1 0-2
A R A R A R
(2,0) (0,0) (1,1) (0,0) (0,2) (0,0)
Def. of Perfect-Information Extensive-Form Games� A perfect-information extensive-form game,
G = (N , H , P , u)� H is a set of sequences (finite or infinite)
� Φ ∈ H
� h = (ak)k=1,...,K ∈ H is a history
� If (ak)k=1,...,K ∈ H and L < K , then
(ak)k=1,...,L ∈ H
� (ak)∞k=1 ∈ H if (a
k)k=1,...,L ∈ H for all positive L
� Z is the set of terminal histories.
H = {Φ, 2− 0, 1− 1, 0− 2, (2− 0,A), (2− 0,R), (1−1,A), (1− 1,R), (0− 2,A), (0− 2,R)}
Alice
Bob Bob Bob
2-0 1-1 0-2
A R A R A R
(2,0) (0,0) (1,1) (0,0) (0,2) (0,0)
Def. of Perfect-Information Extensive-Form Games� A perfect-information extensive-form game,
G = (N , H , P , u)� P is the player function, P : H\Z → N.
P(Φ)=Alice
P(2− 0)=Bob
P(1− 1) = Bob
P(0− 2) = Bob
Alice
Bob Bob Bob
2-0 1-1 0-2
A R A R A R
(2,0) (0,0) (1,1) (0,0) (0,2) (0,0)
Def. of Perfect-Information Extensive-Form Games� A perfect-information extensive-form game,
G = (N , H , P , u)� u = {u1, u2, ...un} is a set of utility functions,
ui : Z → R.
u1((2− 0,A)) = 2
u2((2− 0,A)) = 0
...
Alice
Bob Bob Bob
2-0 1-1 0-2
A R A R A R
(2,0) (0,0) (1,1) (0,0) (0,2) (0,0)
Pure Strategies inPerfect-Information Extensive-Form Games
� A pure strategy of player i ∈ N in an extensive-form gamewith perfect information, G = (N , H , P , u), is a functionthat assigns an action in A(h) to each non-terminalhistory h ∈ H\Z for which P(h) = i .
� A(h) = {a : (h, a) ∈ H}
� A pure strategy is a contingent plan that specifies the
action for player i at every decision node of i .
Pure Strategies for the Sharing Game
Alice
Bob Bob Bob
2-0 1-1 0-2
A R A R A R
(2,0) (0,0) (1,1) (0,0) (0,2) (0,0)
S = {S1,S2}E.g. s1 = (2− 0 if h = Φ),
s2 = (A if h = 2− 0; R if h = 1-1; R if h = 0− 2).
Pure Strategies
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
S = {S1,S2}E.g. s1 = (A if h = Φ; J if h = BF )
s2 = (C if h = A; F if h = B)
Normal-Form Representation
A perfect-information extensive-form game ⇒ A normal-formgame
Alice
Bob Bob Bob
2-0 1-1 0-2
A R A R A R
(2,0) (0,0) (1,1) (0,0) (0,2) (0,0)
2-0
1-1
0-2
(A,A,A) (A,A,R) (A,R,A) (A,R,R) (R,A,A) (R,A,R) (R,R,A) (R,R,R)
2,0 2,0 2,0 2,0 0,0 0,0 0,0 0,0
1,1 1,1 0,0 0,0 1,1 1,1 0,0 0,0
0,2 0,0 0,2 0,0 0,2 0,0 0,2 0,0
A normal-form game � A perfect-information extensive-formgame
Normal-Form Representation: Example 2
A perfect-information extensive-form game ⇒ A normal-formgame
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
(A, J)
(A, K)
(B, J)
(B, K)
(C, E) (C, F) (D, E) (D, F)
3, 8
3, 8
5, 5
5, 5
3, 8
3, 8
2,10
1, 0
8, 3
8, 3
5, 5
5, 5
8, 3
8, 3
1, 10
1, 0
A normal-form game � A perfect-information extensive-formgame
Pure Strategy Nash Equilibrium inPerfect-Information Extensive-Form Games
� A pure strategy profile s is a weak Nash Equilibrium if, for all
agents i and for all strategies s�i �= si , ui (si , s−i ) ≥ ui (s
�i , s−i ).
(Same as in normal-form games)
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
(A, J)
(A, K)
(B, J)
(B, K)
(C, E) (C, F) (D, E) (D, F)
3, 8
3, 8
5, 5
5, 5
3, 8
3, 8
2,10
1, 0
8, 3
8, 3
5, 5
5, 5
8, 3
8, 3
1, 10
1, 0
Pure Strategy Nash Equilibrium inPerfect-Information Extensive-Form Games
� A pure strategy profile s is a weak Nash Equilibrium if, for all
agents i and for all strategies s�i �= si , ui (si , s−i ) ≥ ui (s
�i , s−i ).
(Same as in normal-form games)
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
(A, J)
(A, K)
(B, J)
(B, K)
(C, E) (C, F) (D, E) (D, F)
3, 8
3, 8
5, 5
5, 5
3, 8
3, 8
2,10
1, 0
8, 3
8, 3
5, 5
5, 5
8, 3
8, 3
1, 10
1, 0
Pure Strategy Nash Equilibrium inPerfect-Information Extensive-Form Games
� A pure strategy profile s is a weak Nash Equilibrium if, for all
agents i and for all strategies s�i �= si , ui (si , s−i ) ≥ ui (s
�i , s−i ).
(Same as in normal-form games)
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
(A, J)
(A, K)
(B, J)
(B, K)
(C, E) (C, F) (D, E) (D, F)
3, 8
3, 8
5, 5
5, 5
3, 8
3, 8
2,10
1, 0
8, 3
8, 3
5, 5
5, 5
8, 3
8, 3
1, 10
1, 0
Pure Strategy Nash Equilibrium inPerfect-Information Extensive-Form Games
� A pure strategy profile s is a weak Nash Equilibrium if, for all
agents i and for all strategies s�i �= si , ui (si , s−i ) ≥ ui (s
�i , s−i ).
(Same as in normal-form games)
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
(A, J)
(A, K)
(B, J)
(B, K)
(C, E) (C, F) (D, E) (D, F)
3, 8
3, 8
5, 5
5, 5
3, 8
3, 8
2,10
1, 0
8, 3
8, 3
5, 5
5, 5
8, 3
8, 3
1, 10
1, 0
Pure Strategy Nash Equilibrium inPerfect-Information Extensive-Form Games
� A pure strategy profile s is a weak Nash Equilibrium if, for all
agents i and for all strategies s�i �= si , ui (si , s−i ) ≥ ui (s
�i , s−i ).
(Same as in normal-form games)
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
(A, J)
(A, K)
(B, J)
(B, K)
(C, E) (C, F) (D, E) (D, F)
3, 8
3, 8
5, 5
5, 5
3, 8
3, 8
2,10
1, 0
8, 3
8, 3
5, 5
5, 5
8, 3
8, 3
1, 10
1, 0
Nash Equilibrium and Non-Credible Threat
� Nash Equilibrium is not a very satisfactory solution concept
for perfect-information extensive-form games.
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
(A, J)
(A, K)
(B, J)
(B, K)
(C, E) (C, F) (D, E) (D, F)
3, 8
3, 8
5, 5
5, 5
3, 8
3, 8
2,10
1, 0
8, 3
8, 3
5, 5
5, 5
8, 3
8, 3
1, 10
1, 0
Let’s play a game now
� You are part of a fictitious pride of lions that has a stricthierarchy.
� You are ordered from largest (left) to smallest (right)� All lions prefer to become larger and can do so by eating
the next largest lion if that lion is asleep� Lions sleep after eating
One day a deer passes by the pride. The largest lion would liketo eat it, but is afraid of being eaten by the second largest liononce he falls asleep. Should the largest lion eat the deer?
Subgame Perfect Equilibrium
� Sequential Rationality: A player’s equilibrium strategyshould specify optimal actions at every point in the gametree.
� A Subgame Perfect Equilibrium (SPE) of aperfect-information extensive-form game G is a strategyprofile s such that for any subgame G
� of G , therestriction of s to G
� is a NE.
� Every SPE is a NE, but not vice versa.
� Thm: Every finite extensive-form game with perfectinformation has a subgame perfect equilibrium.
� Finite: The set of sequences H is finite.
Find A SPE: Backward Induction
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
Find A SPE: Backward Induction
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
(2,10)
Find A SPE: Backward Induction
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
(2,10)
(2,10)
Find A SPE: Backward Induction
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
(2,10)
(2,10)(3,8)
Find A SPE: Backward Induction
Alice
Bob Bob
Alice
A B
C D E F
J K
(3,8) (8,3) (5,5)
(2,10) (1,0)
(2,10)
(2,10)(3,8)
(3,8)
Note on Computational Complexity
� Finding NE for general normal-form games requires timeexponential in the size of the normal form.
� The induced normal form of an extensive-form game isexponentially larger than the original representation.
� Algorithm of backward induction requires time linear inthe size of the extensive-form game. (Depth-firsttransverse)
� For zero-sum extensive-form games, we can slightlyimprove the running time.
A Bargaining Game: Split-the-Pie
� Two players trying to split a desirable pie. The set of allpossible agreements X is the set of all divisions of the pie,
X = {(x1, x2) : xi ≥ 0 for i = 1, 2 and x1 + x2 = 1}.
� The first move of the game occurs in period 0, whenplayer 1 makes a proposal x
0 ∈ X , then player 2 eitheraccepts or rejects. Acceptance ends the game whilerejection leads to period 1, in which player 2 makes aproposal x
1 ∈ X , which player 1 has to accept or reject.Again, acceptance ends the game; rejection leads toperiod 2, in which it is once again player 1’s turn to makea proposal. The game continues in this fashion so long asno offer has been accepted.
� ui(x , t) = δtxi if proposal x has been accepted in period
t, δ ∈ (0, 1).� ui = 0 if no agreement has reached.
Split-the-Pie as A Perfect-Information Extensive-FormGame
1
2
x0
(x01 , x0
2 )
2
1
x1
R A
(δx11 , δx1
2 )R A
Nash Equilibria of the Split-the-Pie Game
� The set of NEs is very large. For example, forany x ∈ X there is a NE in which the playersimmediately agree on x .
E.g. Player 1 always propose (0.99, 0.01) andonly accepts a proposal (0.99, 0.01).
SPE of the Split-the-Pie Game
The unique SPE of the game is
� Player 1 always proposes ( 1
1+δ ,δ
1+δ) and accepts
proposals that has x1 ≥ δ1+δ .
� Player 2 always proposes ( δ1+δ ,
1
1+δ) and accepts
proposals that has x2 ≥ δ1+δ .