Embed Size (px)
Citation preview
Game Theory
7 – Repeated Games
Review of lecture six
• Definitions of imperfect information
• Graphical convention for dynamic games
• Subgames, information sets, and strategies with imperfect information
• Searching NE and SPNE (from the tree to the matrix … and back to the tree)
REPEATED GAMES
2
Examples of repeated interactions
• Many interactions in politics, economy, and society happen in repetitive way
• Colleagues, friends, businessmen have routine contacts in offices, houses, and markets
• Parties in democratic systems compete regularly in electoral contests
• Elected representatives have almost daily contacts with each other along the time of one or more mandates
• Governments have continuing relations in the international setting
REPEATED GAMES
3
Repetitiveness and cooperation
• In these conditions phenomena such as reputation, threats, promises, conventions, agreements may have a role in the analysis
• The theory of repeated games tries to deal with these phenomena
REPEATED GAMES
4
Repeated games as supergames
• Repeated games are a particular kind of dynamic games
• Mostly they consist of dynamic games of imperfect information where the same moves of the players are repeated two or more times
• The basic game that is going to be repeated is called stage game
• The repetition of the stage game is also called supergame
• Given a game G, the same game repeated t times (t=2, 3, …) can be indicated as Gt
REPEATED GAMES
5
Payoffs and strategies
• In all dynamic games (of imperfect information) the outcome of subsequent stages depends on that of former stages
• In a repeated game players observe and get their payoffs at the end of each stage
• The final payoffs are given by the sum of payoffs taken by players at each stage of the game
• The strategies of a supergame are plans of action that take into account all repetitions of the game
REPEATED GAMES
6
The stage game of a PD
REPEATED GAMES
7
2
c d
1 C 3,3 1,4
D 4,1 2,2
Example: PD2 payoffs
REPEATED GAMES
8
A
B
B
cooperate
A
6,6 4,7
7,4
5,5
B c
c
c d
d
d
A
4,7 2,8
5,5
3,6
B c
c
c d
d
d
A
7,4 5,5
8,2
6,3
B c
c
c d
d
d
A
5,5 3,6
6,3
4,4
B c
c
c d
d
d
Solving PD2 by backward induction
• The second stage is made by four proper subgames
• In each proper subgame cooperation is a dominated strategy…
• …so that the game in the first stage becomes the following
REPEATED GAMES
9
Example: PD2 payoffs
REPEATED GAMES
10
A
B
B
cooperate
5,5
3,6
6,3
4, 4
Solving PD2 by backward induction
• As a result…the SPNE is (defect2,defect2) and the outcome is (4,4)
• The result of repeated mutual defection of PD2 can be generalized to PDt (t being any finite integer number known to the players)
• The only resulting SPNE of PDt is still (defectt, defectt)
REPEATED GAMES
11
PD repeated a known number of times
• This is intuitively inferred from backwards induction
• At the last stage each player knows that the game is ending
• Then each player’s strategy takes the move “defect” at the last information set (“cooperation” is a dominated strategy!)
– Having pruned the last stage, the same reasoning applies to the last but one stage, and the same conclusion arises
– This reasoning can be reiterated t times, and …
• … the final result is that players anticipate to choose always the “defect” strategy of the stage game
REPEATED GAMES
12
Supergames repeated t times
• The same result can be extended to all supergames whose stage game has only one NE (as PD)
• In a supergame where: – the number of repetition is finite and known to all
players
– the stage game has only one NE, so that the NE of various stages are independent of each other
…Only one SPNE exists, which is the repetition t times of the NE of the stage game
REPEATED GAMES
13
More than one NE
• When the stage game has more than one NE, the equilibria of various stages may depend on each other
• Then it may happen that supergames have NE different from the repetition of the Nash equilibria of the stage game
• Here we will not discuss any further about this possibility
• Rather…
REPEATED GAMES
14
A different situation …what happens when people do not know whether the repeated game is ending or not?
• That is common in the real world: – In society: interactions among peers, or in hierarchical
organizations – In economy: among businessmen, or between firm and
consumer – In politics: among citizens and bureaucrats, or in party
coalitions
• In that circumstance the last stage is eliminated, and backward induction cannot be applied
• Then an important cause of defection is ruled out, and opportunities for cooperation might arise
REPEATED GAMES
15
If the future of interaction is not sure • As in the case of uncertain choice, we suppose that people may
conceive of reasonable ideas about the continuation of the game
• The simplest way to do so is assuming that players know the probability of the continuation
• We assume that future payoffs are discounted by that same probability
• That means that if b is the payoff of the first round of the game, bp is the expected value at the first repetition, bp2 the expected value at the second one, etc.
• Then the total expected benefit is:
REPEATED GAMES
16
2 nb b p b p ... b p ...
Temporal preferences, discounted payoffs and discount rate
• A complementary way: patient or impatient players? Better an egg today or a chicken tomorrow? A discount rate models how highly a person values the future as compared to the present
• The discount rate at time t of player i: dti, where 0< dt
i <1, tells us the present value (PV) of i’s payoff to be made at t periods from the present (where t can refer to days, weeks, years, etc.)
• A person with a discount rate close to 0 only cares about today. A person with a discount rate equal to 1 would be just happy to receive a million dollars in the future as she would be to receive it today
A complementary way Of course we can combine the two approaches, i.e., a person can discount the future and be uncertain about the continuation of the game, i.e. z=d*p
• Then the present value of the total expected benefit simply becomes:
a+az+az2+ az3+…+ azn, where z=d*p, z2= d2 *p2
How to compute an infinite series •Four main scenarios (that we will employ…)
• First: PV(present value) = a+az+az2+ az3+…+ azn is equal to: PV=a/(1-z)
•Second: PV(present value) = az+ az2+ az3+…+ azn is equal to: PV=az/(1-z)
•Third: PV(present value) = a+ az2+ az4+…+ azn is equal to: PV=a/(1-z2)
•Fourth: PV(present value) = az+ az3+ az5+…+ azn+1 is equal to: PV=az/(1-z2)
• For sake of simplicity let’s focus for now just on p
The general idea • Beyond temporal preferences, continual interactions
bring a whole new dimension to the problem. Which one?
• Conditional strategies become possible!
• In iterated games there exists information about the history of the game. Strategy sets employed by each player may therefore expanded to include actions in one particular PD game-stage that are conditional upon prior PD game-stages
REPEATED GAMES
20
The general idea • The main aim of repeated games analysis is to find out if in
strategic settings among rational actors the promises of future rewards or the threats of future punishments …
… may in certain present circumstances induce incentives for cooperative behavior in a PD
• In order this to happen it must be:
(value of being unfair) NOW <
(value of the reward of being fair − (dis)value of the punishment of being unfair) LATER ON
• Moreover, promises of rewards and threats of punishment tomorrow must be credible (in a game with a finite number of stages, in the last one any promise or threat are not credible!!!)
REPEATED GAMES
21
PD∞
Let us consider the PD game indefinitely repeated
REPEATED GAMES
22
B
c d
A C 4,4 1,5
D 5,1 2,2
and let us suppose that players follow these instructions as a conditional strategy: • At first stage play “cooperate” • Then play “cooperate” until the competitor does the same • If the competitor, at any stage, defects play “defect” from then
on
This set of instructions is called grim trigger strategy (GTS)
Grim trigger strategy (1)
• This is a strategy as it instructs about what to do at any stage that, in the case of repeated PD, coincide with information sets
• We want to see if the strategy profile (GTS,GTS) is a (SP)NE of the PD repeated an indefinite number of times
• The total utility of cooperation when playing this strategy is
REPEATED GAMES
23
The defecting player will receive instead
5+ 2 × p+2 × p2 +...+ 2 × pn +... = 5+ 2 × pt = 5+2 × p
1- pt=0
¥
å
4+ 4 × p+ 4 × p2 +...+ 4 × pn +... = 4 × pt =4
1- pt=0
¥
å
Grim trigger strategy (2)
• With some algebra it is easy to see that
REPEATED GAMES
24
4 2 p5 p 1/ 3
1 p 1 p
• That means that a profile of grim trigger strategies is a NE of PD∞ if and only if, at each stage, players expect that the continuation of the game is more probable than 1/3 (with these payoffs)
• This is also a SPNE because, when the game is repeated indefinitely, there are no proper subgames: just one subgame which coincides with the entire game, so that it has the same NE
More generally
• The opportunities of cooperation among rational actors grow with the probability that the interaction continues
• The more the shadow of the future impinges on our present actions the more we are induced to adopt a fair behavior in social relations
• This conclusion does not mean that people change their basic attitudes, but that the environmental conditions of interaction may give opportunities to gain by “good” social behavior
REPEATED GAMES
25
Not so drastic strategies • Grim trigger strategy in PD∞ illustrates the capacity of players
to resist the temptation of defecting today for the expectation of better payoffs tomorrow
• However it express a drastic attitude toward the opponent that does not describe appropriately the usual repeated interactions among social players. Besides it does not allow any mistake!
• This is why theorists have tried to formalize in the strategic framework more realistic models of repeated social interaction
• An idea of retaliation for defecting that is in a sense opposite to grim trigger strategy is embedded in the so called strategy of tit-for-tat tit-for-tat
REPEATED GAMES
26
Tit-for-Tat • A Conditional strategy:
• The Hinduist reincarnation approach? If you behave bad, next time poor you! But if next time you behave good, then lucky you!
Tit-for-Tat
• As PD∞ is concerned, its instructions are the following:
– play C at the first stage of the game
– then play C if (C,C) or a (D,C) profile occur in the preceding stage, and
– play D otherwise
• That is…a player begins by cooperating and then matches the opponents’ previous-period play. With a tit-for-tat strategy, defection is immediately punished; this punishment is applied until a cooperative response is evoked
REPEATED GAMES
28
Tit-for-Tat
• Does the use of this strategy by both players generate a NE of mutual cooperation?
• If yes, under what conditions, as to the probability of continuation of the game?
REPEATED GAMES
29
• Let’s identify with F the free-riding payoff, with C the cooperation payoff, with N the mutual defection payoff and with L (“loser”) the payoff of the lone cooperator
• We have a PD iff F>C>N>L
Tit-for-Tat
A PD Player B
Cooperate Defect
Player A
Cooperate
C, C
L, F
Defect
F, L
N , N
F > C > N > L
Tit-for-Tat (first condition) • Playing cooperatively produces a payoff equal to:
REPEATED GAMES
32
2 n t
t 0
CC C p C p ... C p ... C p
1 p
• Defecting from cooperation produces a payoff equal to:
2 n N pF N p N p ... N p ... F
1 p
• For (C,C) being a NE the first payoff must be greater than the second, and that brings about the condition, that is: p > (F-C)/(F-N)
Tit-for-Tat (second condition) • Alternatively, a player could decide to start to defect
at the first stage, then to cooperate at the next stage, then defect again, and so no. In this case the total payoff is
REPEATED GAMES
33
2 3 n
2 2
F L pF L p F p L p ... F p ...
1 p 1 p
• For (C,C) being a NE the first payoff must be greater than also the third payoff, and that brings about the second condition: p > (F-C)/(C-L)
Tit-for-Tat • For (C,C) being a NE the first payoff must be greater
than the largest alternative payoffs (p > (F-C)/(F-N) and p > (F-C)/(C-L))
• It can be shown that if the stringent of this condition is satisfied, TfT-TfT remains a NE against any other possible strategy (such as defecting for 2 stages, then cooperating for 3 stages, etc. etc.)
• Generally, we can say that, the probability to cooperate increases when: 1) p ↑; 2) C ↑; 3) N ↓; 4) L ↓; 5) F ↓
REPEATED GAMES
34
Linking the theory with the empirical facts: Elinor Ostrom (Nobel Prize 2009) in her book Governing the Commons discusses several examples of resolution of collective action problems • Most of these involve taking advantage of features
specific to the local context in order to set up systems of defection and punishment
• The most striking feature of Ostrom’s range of cases is their immense variety: some success, some failures
• Despite this variety, we can identify several common features that make it easier to solve cooperator’s dilemmas
Endogenous solutions? Yes, we can!
1. The benefits of cooperation have to be large enough to make it worth paying the costs of monitoring and enforcing the rules of cooperation
2. It is essential to have an identifiable and stable group of potential participants
Why do those factors matter? Now you know why!
Which factors matter
3. It is also very important that the members of the group can communicate with one another and spread information about each other behaviors!
Why? An example
Which factors matter
Imagine a group of m individuals that interact in a infinite way (or equivalently a group that is uncertain about the end of the game)
Let’s suppose that in each round of the game, every agent interacts with another agent
Every agent knows about the history (reputation) of the other player with probability 𝜌
With probability (1-𝜌) she cannot know if the other agent behaved cooperative or not in the previous game with some other agent in the group
Which factors matter
• Let’s further assume: 1) a grim strategy for sake of simplicity; 2) the probability of being discovered while cheating is strictly increasing in m (in the size of the group): if m increases, for example, the ability to get information about the strategy played by other players decreases considerably…
• Let’s model such assumption in the following way: for every stage of the game a fixed amount of players M<m remember the behavior of the players. Accordingly the probability that a player that did not cooperate in the previous round and a player that knows about that meet each other is 𝜌 𝑚 = 𝑀/𝑚 , a probability decreasing in m as we have assumed
The size of the group
If you Cooperate against a player adopting such conditional strategy, you get:
𝐶 + 𝑝 𝜌(𝑚)𝐶 + 1 − 𝜌(𝑚) 𝐶 = 𝐶/(1 − 𝑝)
∞
𝑡=1
Which factors matter
If you Defect, you get:
𝐹 + 𝑝 𝜌(𝑚)𝑁 + 1 − 𝜌(𝑚) 𝐹 = 𝐹 +𝑝
1 − 𝑝(𝜌(𝑚)𝑁 + 1 − 𝜌(𝑚) 𝐹)
∞
𝑡=1
Therefore, you cooperate if:
𝑝𝜌(𝑚) ≥𝐹 − 𝐶
𝐹 − 𝑁
According there is a level of m* above which the cooperation within the group cannot be sustainable
Implicitly m* is defined as the following:
𝑝𝜌 𝑚∗ ≥𝐹 − 𝐶
𝐹 − 𝑁
Remember that 𝜌 𝑚 = 𝑀/𝑚 , therefore:
𝑚∗ = 𝑝𝑀𝐹−𝑁
𝐹−𝐶 ,
where m* can increase when 1) p ↑; 2) M (good communication!) ↑; 3) C ↑); 4) F ↓; 5) N ↓
The size of the group
Parallel games • Players do not only interact over time. Sometimes they play more games at the same moment
• Linking two (or more) games: parallel games
Parallel games • The example of the Japanese firms. Why Japanese firms use (used) to subsidize the social activities of their workers outside of the working place?
• We have two games now. The work-game and the social-game. Does it matter?
• The role of regimes in IR: why creating a dense network of IO should matter? Why did embedded liberalism work for such a long time?
Parallel games
• In this scenario, the conditional strategy is not only affected of what is happening in one single game, but also to what is happening in the other game
• A typical conditional strategy now requires to link the behaviour of one player to what the other player is doing in both games through a grim trigger strategy
Parallel games • Let’s assume that in the second game (the social one) cooperation is feasible, that is : p>(F-C)/(F-N), while in the first game (the working one) it is not so, that is: : p<(F-C)/(F-N)
• Let’s call Bs the net benefits from cooperation in the second game (Bs = C/(1-p)-[F+pN/(1-p)]) and Ps the net costs of cooperation in the first game (Ps = [F+pN/(1-p)]-C/(1-p))
• If Bs>Ps, then if we have connected the two games, cooperation in the second game allows to produce cooperation also in the first game
• But be careful! If Bs<Ps, then connecting two games can destroy the cooperation also in the second game
Conclusion • Cooperation in the repeated PD game can evolve on
a purely egoistic basis: players will employ an initially suboptimal policy (a dominated strategy) in the long run because such a policy trades smaller short-term gains for larger long-term ones
• Rationality involves time preferences in addition to expectations of what others will do, that is, consideration of the short and long term
• However the possibility of cooperation does not imply its inevitability, even if the game is repeated! The Folk Theorem
REPEATED GAMES
46
