Inefficiency of Equilibria: POA, POS, SPOA

Based on Slides by Amir Epstein and by Svetlana Olonetsky

Modified/Corrupted by Michal Feldman and Amos Fiat

Inefficiency of equilibria

• Outcome of rational behavior might be inefficient• How to measure inefficiency?

– E.g., prisoner’s dilemma

• Define an objective function– Social welfare (= sum of players’ payoffs): utilitarian– Maximize mini ui (egalitarian)– …

3,3 0,5

5,0 1,1

Inefficiency of equilibria

• To measure inefficiency we need to specify: – Objective function– Definition of approximately optimal– Definition of an equilibrium– If multiple equilibria exist, which one do we

consider?

4

More Equilibrium Concepts

pureNash

mixed Nash

correlated eq ???

no regret ???

best-responsedynamics

Strong Nash

Common measures

• Price of anarchy (POA)=cost of worst NE / cost of OPT

• Price of stability (POS)=cost of best NE / cost of OPT

• Approximation ratio: Measures price of limited computational resources

• Competitive ratio: Measures price of not knowing future

• Price of anarchy: Measures price of lack of coordination

Pure

Pure

Pure

Pure

/Mixed

/Mixed

/Mixed

/Mixed

/Strong/Strong

/Strong /Strong

Corr POA ≤ Pure POA ≤ Mixed POA ≤ Strong POA

Corr POS ≥ Pure POS ≥ Mixed POA ≥ Strong POA

Price of anarchy

• Example: in prisoner’s dilemma, POA = POA = 3– But can be as large as desired

• Wish to find games in which POS or POA are bounded– NE “approximates” OPT– Might explains Internet efficiency.

• Suppose we define POA and POS w.r.t. NE in pure strategies– we first need to prove existence of pure NE

3,3 0,5

5,0 1,1

Prisoner’s dilemma

Max-cut game

• Given undirected graph G = (V,E)

• players are nodes v in V

• An edge (u,v) means u “hates” v (and vice versa)

• Strategy of node i: si {Black,White}

• Utility of node i: # neighbors of different color

• Lemma: for every graph G, corresponding game has a pure NE

Proof 1• Claim: OPT of max-cut defines a NE• Proof:

– Define strategies of players by cut (i.e., one side is Black, other side is White)

– Suppose a player i wishes to switch strategies: i’s benefit from switching = improvement in value of the cut

– Contradicting optimality of cut

ui=1 ui=2

Proof 2

• Algorithm greedy-find-cut (GFC):– Start with arbitrary partition of nodes into two sets– If exists node with more neighbors in other side, move

it to other side (repeat until no such node exists)• Claim 1: GFC provides 2-approx. to max-cut, and

runs in polynomial time• Proof:

– Poly time: GFC terminates within at most |E| steps (since every step improves the value of the solution in at least 1, and |E| is a trivial upper bound to solution)

– 2-approx.: Each node ends up with more neighbors in other side than in own side, so at least |E|/2 edges are in cut (since #edges in cut > #edges not in cut)

Proof 2 (cont’d)

• Claim 2: cut obtained by GFC defines a NE

• Proof: obvious, as each player stops only if his strategy is the best response to the other players’ strategies

• Conclusion: max-cut game admits a NE in pure strategies

11

Wardrop Equilibria: Traffic Flow: the Mathematical Model

• a directed graph G = (V,E)

• k source-destination pairs (s1 ,t1), …, (sk ,tk)

• a rate (amount) ri of traffic from si to ti

• for each edge e, a cost function ce(•)

s1 t1

c(x)=x Flow = ½

Flow = ½c(x)=1

Example: (k,r=1)

12

Routings of Traffic

Traffic and Flows:

• fP = amount of traffic routed on si-ti path P

• flow vector f routing of traffic

Selfish routing: what are the equilibria?

s t

13

Wardrop Flows

Special case, assumptions:• agents small relative to network (nonatomic game)• want to minimize cost of their path

Def: A flow is at [Pure] Nash equilibrium (or is a Nash flow) if all flow is routed on min-cost paths [given current edge congestion]

xs t

1Flow = .5

Flow = .5

s t1

Flow = 0

Flow = 1

x

Example:

14

History + Generalizations

• model, defn of Nash flows by [Wardrop 52]

• Nash flows exist, are (essentially) unique– due to [Beckmann et al. 56]– general nonatomic games: [Schmeidler 73]

• congestion game (payoffs fn of # of players)– defined for atomic games by [Rosenthal 73]– previous focus: Nash eq in pure strategies exist

• potential game (equilibria as optima)– defined by [Monderer/Shapley 96]

15

The Cost of a Flow

Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate)

s t

x

1

½

½

Cost = ½•½ +½•1 = ¾

16

The Cost of a Flow

Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate)

Formally: if cP(f) = sum of costs of edges of P (w.r.t. the flow f), then:

C(f) = P fP • cP(f)

s ts t

x

1

½

½

Cost = ½•½ +½•1 = ¾

17

Inefficiency of Nash FlowsNote: Nash flows do not minimize the cost • observed informally by [Pigou 1920]

• Cost of Nash flow = 1•1 + 0•1 = 1• Cost of optimal (min-cost) flow = ½•½ +½•1 = ¾• Price of anarchy := Nash/OPT ratio = 4/3

s t

x

1

0

1 ½

½

18

Braess’s Paradox

Initial Network:

s tx 1

½

x1½

½

½

cost = 1.5

19

Braess’s Paradox

Initial Network: Augmented Network:

s tx 1

½

x1½

½

½

cost = 1.5

s tx 1

½

x1½

½

½0

Now what?

20

Braess’s Paradox

Initial Network: Augmented Network:

s tx 1

½

x1½

½

½

cost = 1.5 cost = 2

s t

x 1

x1

0

21

Braess’s Paradox

Initial Network: Augmented Network:

All traffic incurs more cost! [Braess 68]

• see also [Cohen/Horowitz 91], [Roughgarden 01]

s tx 1

½

x1½

½

½

cost = 1.5 cost = 2

s t

x 1

x1

0

Special Case of routing: Equal Machine Load Balancing = Parallel

Links• Two nodes

• m parallel (related) links

• n jobs (communication requests)

• User cost (delay) is proportional to link load

• Global cost (maximum delay) is the maximum link load

Price of Anarchy

• Price of Anarchy:

The worst possible ratio between: - Objective function in Nash Equilibrium and- Optimal Objective function

• Objective function: total user cost, total user utility, maximal/minimal cost, utility, etc., etc.

Identical machines

• Main results (objective function – maximum load)- For m identical links, identical jobs (pure) R=1- For m identical links (pure) R=2-1/(m+1)- For m identical links (mixed)

m

mR

loglog

log

Lower bound – easy : uniformly choose machine with prob. 1/mUpper bound – assume opt = 1, opt = max expected ≤ 2 in NE (otherwise not NE,

NE = expected max ≤ log m / loglog m due to Hoeffding concentration inequality

Related Work (Cont’)

• Main results

- For 2 related links R=1.618

- For m related links (pure)

- For m related links (mixed)

- For m links restricted assignment (pure)

- For m links restricted assignment (mixed)

m

mR

loglog

log

m

mR

logloglog

log

m

mR

loglog

log

m

mR

logloglog

log

2 2

1

Identical machines

Highest load machine (#1), lowest weight job on #1 (1)

Lowest weight job on highest load machine ≤ ½ HL (3)

Every other machine has load ≥ ½ HL

2 1.5

• m (=3) machines• n (=4) jobs

• vi – speed of machine i

• wj – weight of job j

v1 = 4 v2 = 2 v3 = 1

1 (4) 2 (4) 2 (2)

1 (2)

L1 = 1 L2 = 3 L3 = 2

Related machines

• Li – load on machine i

Price of Anarchy: Lower Bound

k! / (k-i)!

Gi

k-i

k !1

k

Gk

k

k-1

k(k-1)

k-2

G0 G1 G2

v=2k-i v=1v=2k

w=2k-iw=2k

v=2w=2

v=2k-1

w=2k-1

Price of Anarchy: Lower Bound

Gi

k-i

k !1

k

Gk

k

k-1

k(k-1)

k-2

G0 G1 G2k! ~ m

k ~ log m / log log m

k! / (k-i)!

v=2k-i v=1v=2k

w=2k-iw=2k

v=2w=2

v=2k-1

w=2k-1

11

Its a Nash Equilibrium

Gi

k-i

k !1

k

Gk

k

k-1

k(k-1)

k-2

G0 G1 G2

k! / (k-i)!

2

v=2k-i v=1v=2k

w=2k-iw=2k

v=2w=2

v=2k-1

w=2k-1

1

Its a Nash Equilibrium

Gi

k-i

k !1

k

Gk

k

k-1

k(k-1)

k-2

G0 G1 G2

k! / (k-i)!

2 4

v=2k-i v=1v=2k

w=2k-iw=2k

v=2w=2

v=2k-1

w=2k-1

1

The social optimum

k! / (k-i)!

Gi

k-i

k !1

k

Gk

k

k-1

k(k-1)

k-2

G0 G1 G2

21

v=2k-i v=1v=2k

w=2k-iw=2k

v=2w=2

v=2k-1

w=2k-1

The social optimum

k! / (k-i)!

Gi

k-i

k !1

k

k

k-1

k(k-1)

k-2

G0 G1 G2

2

v=2k-i v=1v=2k

w=2k-iw=2k

v=2w=2

v=2k-1

w=2k-1

1

Gk

12

The social optimum

k! / (k-i)!

Gi

k-i

k !1

k

k

k-1

k(k-1)

k-2

G0 G1 G2

2

2 22 2

v=2k-i v=1v=2k

w=2k-iw=2k

v=2w=2

v=2k-1

w=2k-1

Gk

Related Machines: Price of Anarchy upper bound

• Normalize so that Opt = 1

• Sort machines by speed

• The fastest machine (#1) has load Z, no machine has load greater than Z+1 (otherwise some job would jump to machine #1)

• We want to give an upper bound on Z

Related Machines: Price of Anarchy upper bound

• Normalize so that Opt = 1• The fastest machine (#1) has load Z, but

Opt is 1, consider all the machines that Opt uses to run these jobs.

• These machines must have load ≥ Z-1 (otherwise job would jump from #1 to this machine)

• There must be at least Z such machines, as they need to do work ≥ Z

Related Machines: Price of Anarchy upper bound

• Take the set of all machines up to the last machine that opt uses to service the jobs on machine #1.

• The jobs on this set of machines have to use Z(Z-1) other machines under opt.

• Continue, the bottom line is that n ≥ Z!, or that Z ≤ log m / log log m

Restricted Assignment to Machines

m0 m0 m0 m0 m0 m1m0 m1 m1 m1 m1 m1 m2 m2 m2 m3

NASH

Group 1

m0 m0 m0 m0 m0 m1m0 m1 m1 m1 m1 m1 m2 m2 m2 m3

Group 2 Group 3

Group 1

Group 2

Group 3

OPT

l=3

Restricted Assignment to Machines

m0 m0 m0 m0 m0 m1m0 m1 m1 m1 m1 m1 m2 m2 m2 m3

NASH

Group 1

m0 m0 m0 m0 m0 m1m0 m1 m1 m1 m1 m1 m2 m2 m2 m3

Group 2 Group 3

Group 1

Group 2

Group 3

OPT

l=3

Price of anarchy for unrelated machines

• POA for unrelated machines is unbounded

1

1

Job 1

Job 2

Machine 1 Machine 2

1 1

Machine 2Machine 1Machine 2

Machine 1

Social optimum Nash equilibrium

makespan= makespan=

PoA=1/

Allowing Coordination in Equilibrium

• Strong Equilibrium [Aumann’59]– No coalition can deviate and strictly improve

the utility of all of its members• very robust concept• may be a better prediction of rational

behavior• most games do not admit Strong Eq.

– usually applied to pure Eq with pure deviations

Example 1: Prisoner’s Dilemma

0,5

5,0

cooperate

cooperate

defect

defect

Unique Nash Eq.

Strong Eq. ?

Prisoner’s dilemma does not admit any Strong Eq.

Strong Price of Anarchy

• Determining SPoA requires two parts:– Proving existence of Strong Eq– Bounding the worst ratio

• SE NE SPoA ≤ PoA

Price of Anarchy (PoA) [KP00]:

optimum social

mequilibriuNash worst PoA

Strong Price of Anarchy (SPoA):

optimum social

mequilibriu Strongworst SPoA

k-Strong Equilibrium

• A joint action sS is not resilient to a pure deviation of a coalition if there is a pure action profile of such that ci(s-,)<ci(s) for any i – e.g., (defect,defect) in Prisoner’s dilemma

• A pure Nash Eq sS is resilient to pure deviation of coalitions of size k if there is no coalition of size at most k such that s is not resilient to a pure deviation by

• A k-Strong Equilibrium is a pure Nash Eq that is resilient to pure deviation of coalitions of size at most k

S=S1x…xSn

Strong Equilibrium Hierarchy

1-SE

2-SE

n-SE

= NE

=SE [Aumann]

Related Work

• Existence of Strong Equilibrium– monotone decreasing congestion games [Holzman+Lev-

tov 1997, 2003]

– monotone increasing congestion games + correlated SE [Rosenfeld+Tennenholtz 2006]

• Related solution concepts– Coalition-proof Eq. [Bernheim 1987]

– Group-strategyproof mechanisms [Moulin+Shenker 2001]

– Coalitions with transferable utilities [Hayrapetyan et al 2006]

SE

CPE

NE

Existence of Strong Equilibrium in load balancing games

• Is every Nash Eq. on identical machines also a Strong Eq ?– NO ! (for m ≥ 3)

5

5

4 4

3 3

10 7 7

s

55

4 43

3

6 99

s’Coalition: 5,5,3,3

Strong Eq. Existence

• Theorem: in any load balancing game, the lex. minimal joint action s is a k-SE for any k

Recall Lexicographic Order

• Definition: a vector (l1,…lm) is smaller than (l1’,…,lm’) lexicographically if for some i, li < li’ and lk = lk’ for all k<I

• Definition: A joint action s is smaller than s’ lex. (ss’) if the vector of machine loads L(s), sorted in non-decreasing order, is smaller lex. than L(s’)

s

s’ss’

Proof of SE Existence• Suppose in contradiction that s (lex. minimal) is not a

SE, and let be the smallest coalition (deviating to s’).• Claim: the same set of machines are chosen by in s

and in s’ (denote it M())– If a job migrates TO some

machine, another jobmigrates FROM it

• else contradicting s is NE – If a job migrates FROM some

machine, another jobmigrates TO it

• else contradicting minimality of • Since all jobs in must benefit, all loads of M() in s’

must be smaller than max load of M() in s – Contradicting minimality of s

Price of Anarchy (PoA)• Recall: for unrelated machines, PoA may be

unbounded

1

1

Job 1

Job 2

Machine 1 Machine 2

Objective: min makespan

Social optimumNash equilibrium Nash equilibrium

PoA=1/

1 1

M1

makespan=

M2

makespan=

M1 M2

Strong equilibrium Strong equilibrium

SPoA=1

Strong Price of Anarchy

• Theorem: for any job scheduling game with m unrelated machines and n jobs, SPoA ≤ m

Proof for SpoA ≤ m

• Claim 1: L1(s) ≤ OPT– else: coalition of all jobs to OPT

M1Mm Mi Mi-1 M1Mm Mi Mi-1

OPT

L1(s)

OPT

L1(s)

Proof for SpoA ≤ m

• Claim 1: L1(s) ≤ OPT– else: coalition of all jobs to OPT

• Claim 2: i Li(s)-Li-1(s) ≤ OPT – else: consider s’, where all jobs on machines i..m go to OPT.

For all J • cJ(s) > Li-1(s) + OPT• cJ(s’) ≤ Li-1(s) + OPT (since all J together add at most OPT)

M1Mm Mi Mi-1 M1Mm Mi Mi-1

> OPT OPT

Lm(s) ≤ m OPT

Li-1(s) L1(s)

Li(s)

Lower Bound (m machines)• Theorem: there exists a job scheduling game with m

unrelated machines for which SPoA ≥ m

• Proof:

M1 M2 M3 M4 Mm

J1 1 1

J2 1 2

J3 1 3

J4 1 4

Jm 1 m

OPT = 1

makespan=mSE

Identical Machines

• Theorem: there exists a job scheduling game with m identical machines and n jobs, such that

m

SPoA1

1

2

1 2 m-1 m

J1

Jm

Jm+1

J2m

1

1/m1 m-2 m-1 m

OPT

SE

1+1/m

2

Results - machines

• Objective function – maximum load- For m identical links, identical jobs (pure) R=1- For m identical links (pure) POA= SPOA = 2-1/(m+1),

- For m related links (pure)

- For m links restricted assignment (pure)

- For m unrelated machines,

m

mPOA

loglog

log

m

mPOA

loglog

log

Homework assignment

• Fill in tables:

– (pure POA, pure SPOA, pure POS, pure SPOS) * (identical, related, unrelated)

– (mixed POA, mixed SPOA, mixed POS, mixed SPOS) * (identical, related, unrelated)

• Why?

• Send me e-mail with pptx presentation