Upload
emery-holt
View
223
Download
0
Tags:
Embed Size (px)
Citation preview
Potential Functions and the Inefficiency of Equilibria
Tim RoughgardenStanford University
2
Pigou's Example
Example: one unit of traffic wants to go from s to t
Question: what will selfish network users do?• assume everyone wants smallest-possible cost• [Pigou 1920]
s t
c(x)=x
c(x)=1
cost depends on congestion
no congestion effects
3
Motivating Example
Claim: all traffic will take the top link.
Reason:• Є > 0 traffic on bottom is envious• Є = 0 equilibrium
– all traffic incurs one unit of cost
s t
c(x)=x
c(x)=1
Flow = 1-Є
Flow = Єthis flow is envious!
4
Can We Do Better?
Consider instead: traffic split equally
Improvement:• half of traffic has cost 1 (same as before)• half of traffic has cost ½ (much improved!)
s t
c(x)=x
c(x)=1
Flow = ½
Flow = ½
5
Braess’s Paradox
Initial Network:
s tx 1
½
x1½
½
½
Cost = 1.5
6
Braess’s Paradox
Initial Network: Augmented Network:
s tx 1
½
x1½
½
½
Cost = 1.5
s tx 1
½
x1½
½
½0
Now what?
7
Braess’s Paradox
Initial Network: Augmented Network:
s tx 1
½
x1½
½
½
Cost = 1.5 Cost = 2
s t
x 1
x10
8
Braess’s Paradox
Initial Network: Augmented Network:
All traffic incurs more cost! [Braess 68]
• also has physical analogs [Cohen/Horowitz 91]
s tx 1
½
x1½
½
½
Cost = 1.5 Cost = 2
s t
x 1
x10
9
High-Level Overview
Motivation: equilibria of noncooperative network games typically inefficient
• e.g., Pigou's example + Braess's Paradox• don't optimize natural objective functions
Price of anarchy: quantify inefficiency w.r.t some objective function
Our goal: when is the price of anarchy small?– when does competition approximate cooperation?– benefit of centralized control is small
10
Selfish Routing Games
• directed graph G = (V,E)
• source-destination pairs (s1,t1), …, (sk,tk)
• ri = amount of traffic going from si to ti
• for each edge e, a cost function ce(•)– assumed continuous and nondecreasing
Examples: (r,k=1)
s1 t1
c(x)=x c(x)=1
c(x)=xc(x)=1
½
½s1 t1
c(x)=x
c(x)=1
½
½c(x)=0
11
Outcomes = Network Flows
Possible outcomes of a selfish routing game:
• fP = amount of traffic choosing si-ti path P
• outcomes of game flow vectors f– flow vector: nonnegative and total flow fP
on si-ti paths equals traffic rate ri (for all i)s t
12
Outcomes = Network Flows
Possible outcomes of a selfish routing game:
• fP = amount of traffic choosing si-ti path P
• outcomes of game flow vectors f– flow vector: nonnegative and total flow fP on si-ti
paths equals traffic rate ri (for all i)
Question: What are the equilibria (natural selfish outcomes) of this game?
s t
13
Nash Flows
Def: [Wardrop 52] A flow is at Nash equilibrium (or is a Nash flow) if no one can switch to a path of smaller cost. I.e., all flow is routed on min-cost paths. [given current edge congestion]
xs t
1s t
1
x
Examples:½
½
1
s tx 1
x10 1s t
x 1
x10
½
½
14
Our Objective Function
Definition of social cost: total cost C(f) incurred by the traffic in a flow f.
Formally: if cP(f) = sum of costs of edges of P (w.r.t. flow f), then:
C(f) = P fP • cP(f)
s t
15
Our Objective Function
Definition of social cost: total cost C(f) incurred by the traffic in a flow f.
Formally: if cP(f) = sum of costs of edges of P (w.r.t. flow f), then:
C(f) = P fP • cP(f)
Example:
s t
s tx
1½½
Cost = ½•½ +½•1 = ¾
16
The Price of Anarchy
Defn:
– definition from [Koutsoupias/Papadimitriou 99]
price ofanarchy of a game
=obj fn value of selfish outcome
optimal obj fn value
xs t
1s t
1
x
Example: POA = 4/3 in Pigou's example½
½
1
Cost = 1 Cost = 3/4
17
A Nonlinear Pigou Network
Bad Example: (d large)
equilibrium has cost 1, min cost 0
s t
xd
10
1 1-Є
Є
18
A Nonlinear Pigou Network
Bad Example: (d large)
equilibrium has cost 1, min cost 0
price of anarchy unbounded as d -> infinity
Goal: weakest-possible conditions under which P.O.A. is small.
s t
xd
10
1 1-Є
Є
19
When Is the Price of Anarchy Bounded?
Examples so far:
Hope: imposing additional structure on the cost functions helps– worry: bad things happen in larger networks
s tx
1s t
xd
1s t
x 1
x10
20
Polynomial Cost Functions
Def: linear cost fn is of form ce(x)=aex+be
Theorem: [Roughgarden/Tardos 00] for every network with linear cost functions:
≤ 4/3 × cost of Nash flow
cost of opt flow
s tx
1
21
Polynomial Cost Functions
Def: linear cost fn is of form ce(x)=aex+be
Theorem: [Roughgarden/Tardos 00] for every network with linear cost functions:
≤ 4/3 ×
Bounded-deg polys: (w/nonneg coeffs) replace 4/3 by Θ(d/log d)
cost of Nash flow
cost of opt flow
s txd
1
tightexample
s tx
1
22
A General Theorem
Thm: [Roughgarden 02], [Correa/Schulz/Stier
Moses 03] fix any set of cost fns. Then, a Pigou-like example 2 nodes, 2 links, 1 link w/constant cost fn) achieves worst POA
s txd
1
tightexample
23
Interpretation
Bad news: inefficiency of selfish routing grows as cost functions become "more nonlinear".– think of "nonlinear" as "heavily congested"– recall nonlinear Pigou's example
Good news: inefficiency does not grow with network size or # of source-destination pairs.– in lightly loaded networks, no matter how
large, selfish routing is nearly optimals txd
1
tightexample
24
Benefit of Overprovisioning
Suppose: network is overprovisioned by β > 0 (β fraction of each edge unused).
Then: Price of anarchy is at most ½(1+1/√β).
• arbitrary network size/topology, traffic matrix
Moral: Even modest (10%) over-provisioning sufficient for near-optimal routing.
25
Potential Functions
• potential games: equilibria are actually optima of a related optimization problem
– has immediate consequences for existence, uniqueness, and inefficiency of equilibria
– see [Beckmann/McGuire/Winsten 56], [Rosenthal 73], [Monderer/Shapley 96], for original references
– see [Roughgarden ICM 06] for survey
26
The Potential Function
Key fact: [BMV 56] Nash flows minimize “potential function” e ∫f
ce(x)dx (over all flows).
ce(fe)
00 fe
0e
27
The Potential Function
Key fact: [BMV 56] Nash flows minimize “potential function” e ∫f
ce(x)dx (over all flows).
Lemma 1: locally optimal solutions are precisely the Nash flows (derivative test).
Lemma 2: all locally optimal solutions are also globally optimal (convexity).
Corollary: Nash flows exist, are unique.
ce(fe)
00 fe
0e
28
Consequences for the Price of Anarchy
Example: linear cost functions.
Compare cost + potential function:
C(f) = e fe • ce(fe) = e [ae fe + be fe]
PF(f) = e ∫f ce(x)dx = e [(ae fe)/2 + be fe]
2
0e
2
29
Consequences for the Price of Anarchy
Example: linear cost functions.
Compare cost + potential function:
C(f) = e fe • ce(fe) = e [ae fe + be fe]
PF(f) = e ∫f ce(x)dx = e [(ae fe)/2 + be fe]
• cost, potential fn differ by factor of ≤ 2• gives upper bound of 2 on price on anarchy
– C(f) ≤ 2×PF(f) ≤ 2×PF(f*) ≤ 2×C(f*)
2
0e
2
30
Better Bounds?
Similarly: proves bound of d+1 for degree-d polynomials (w/nonnegative coefficients).
• not tight, but qualitatively accurate – e.g., price of anarchy goes to infinity with
degree bound, but only linearly
• to get tight bounds, need "variational inequalities"– see my ICM survey for details
31
Variational Inequality
Claim: • if f is a Nash flow and f* is feasible, then
e fe • ce(fe) ≤ e f* • ce(fe)
• proof: use that Nash flow routes flow on shortest paths (w.r.t. costs ce(fe))
e
32
Pigou Bound
Recall goal: want to show Pigou-like examples are always worst cases.
Pigou bound: given set of cost functions (e.g., degree-d polys), largest POA in a network:
• two nodes, two links• one function in given set• one constant function
– constant = cost of fully congested top edge
s txd
1
33
Pigou Bound (Formally)
Let S = a set of cost functions.– e.g., polynomials with degree at most d,
nonnegative coefficients
Definition: the Pigou bound α(S) for S is:
max
• max is over all choices of cost fns c in S, traffic rate r 0, flow y 0
s txd
1
r • c(r)
y • c(y) + (r-y) • c(r)
34
Pigou Bound (Example)
Let S = { c : c(x) = ax +b } [linear functions]
Recall: the Pigou bound α(S) for S is:
max
• max is over all choices of cost fns c in S, traffic rate r 0, flow y 0
• choose c(x) = x; r = 1; y = 1/2 get 4/3• calculus: α(S) = 4/3 [d/ln d for deg-d polynomials]
s tx
1
r • c(r)
y • c(y) + (r-y) • c(r)
35
Main Theorem (Formally)
Theorem: [Roughgarden 02, Correa/Schulz/Stier Moses 03]: For every set S, for every selfish routing network G with cost functions in C, the POA in G is at most α(S).– POA always maximized by Pigou-like examples
That is, if f and f* are Nash + optimal flows in G, then C(f)/C(f*) ≤ α(S).
– example: POA ≤ 4/3 if G has affine cost fns
36
Proof of General Thm
Let f and f* are Nash + optimal flows in G.
37
Proof of General Thm
Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound
with c = ce, y = f*, r = fe:
α(S) fe • ce(fe)/[f* • ce(f*) + (fe -f*
) • ce(fe)]eee
e
38
Proof of General Thm
Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound
with c = ce, y = f*, r = fe:
α(S) fe • ce(fe)/[f* • ce(f*) + (fe -f*
) • ce(fe)]
Step 2: rearrange and sum over e: C(f*) = e f*
• ce(f*)
eee
e e
e
39
Proof of General Thm
Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound with
c = ce, y = f*, r = fe:
α(S) fe • ce(fe)/[f* • ce(f*) + (fe -f*
) • ce(fe)]
Step 2: rearrange and sum over e: C(f*) = e f*
• ce(f*) [e fe • ce(fe)]/α(S) + [e (f* - fe) • ce(fe)]
eee
e e
e
e
40
Proof of General Thm
Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound with c
= ce, y = f*, r = fe:
α(S) fe • ce(fe)/[f* • ce(f*) + (fe -f*
) • ce(fe)]
Step 2: rearrange and sum over e: C(f*) = e f*
• ce(f*) [e fe • ce(fe)]/α(S) + [e (f* - fe) • ce(fe)]
Step 3: apply VI
eee
e
0
e e
e
41
Proof of General Thm
Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound with
c = ce, y = f*, r = fe:
α(S) fe • ce(fe)/[f* • ce(f*) + (fe -f*
) • ce(fe)]
Step 2: rearrange and sum over e: C(f*) = e f*
• ce(f*) [e fe • ce(fe)]/α(S)
Step 3: apply VI, done!
eee
=C(f)
e e
e
42
Recap
• selfish routing: simple, basic routing game– inefficient equilibria: Pigou + Braess examples
• price of anarchy: ratio of objective fn values of selfish + optimal outcomes
• potential functions: equilibria actually solving a related optimization problem– immediate consequence for existence,
uniqueness, and inefficiency of equilibria
43
Recap
• variational inequality: inequality based on "first-order condition" satisfied by equilibria
• Pigou bound: given a set of cost functions, largest POA in a Pigou-like example
• main result: for every set of cost fns, Pigou bound is tight (all multicommodity networks)– POA depends only on complexity of cost functions,
not on complexity of network structure
44
Outline
Part I: The Price of Anarchy in Selfish Routing Games
Part II: The Price of Stability in Network Connectivity Games
45
Selfish Network Design
Given: G = (V,E), fixed costs ce for all e є E,
k vertex pairs (si,ti)
Each player wants to build a network in which its nodes are connected.
Player strategy: select a path connecting si to ti.
• [Anshelevich et al 04]
46
Shapley Cost Sharing
How should multiple players on a single edge split costs?
Natural choice is fair sharing, or Shapley cost sharing:
Players using e pay for it evenly: ci(P) = Σ ce/ke
Each player tries to minimize its cost.
e є P
47
Comparison to Selfish Routing
Note: like selfish routing, except:• finite number of outcomes
– in selfish routing, outcomes = fractional flows
• positive (not negative) externalities– cost function (per player) = ce/ke
Objective: C = Σi ci(Pi) = Σ ce
• where S = union of Pi's
e є S
48
What's the POA?
Example:
t
s
1+ k
t1, t2, … tk
s1, s2, … sk
49
What's the POA?
Example:
t
s
1+ k
t1, t2, … tk
s1, s2, … sk
t
s
1+ k
OPT(also Nash eq)
50
What's the POA?
Example:
t
s
1+ k
t1, t2, … tk
s1, s2, … sk
t
s
1+ k
OPT(also Nash eq)
t
s
1+ k
anotherNash eq
51
Multiple Equilibria
Moral: in Shapley network design games, different Nash eq can have different costs.
Recall:
Note: not well defined if Nash eq not unique.• which one do we look at?
POA of a game
=obj fn value of selfish outcomeoptimal obj fn value
52
The Price of Stability
General definition of POA: [KP99]
• POA = k in last example, uninteresting
Price of Anarchy = cost(worst NE)
cost(OPT)
53
The Price of Stability
General definition of POA: [KP99]
• POA = k in last example, uninteresting
Alternative:
• POS = 1 in last example
Price of Anarchy = cost(worst NE)
cost(OPT)
Price of Stability = cost(best NE)
cost(OPT)
54
The Price of Stability
Note: small price of stability only guarantees that some Nash eq has low cost.
• much weaker guarantee than small POA
Interpretation: best solution consistent with self-interested players
• natural outcome for centralized planner to suggest [e.g., network protocol designer]
55
Example: High Price of Stability
1 1k
12
13
1 2 3 k
t
0 0 0 0
1+ . . . k-1
0
1k-1
56
Example: High Price of Stability
1 1k
12
13
1 2 3 k
t
0 0 0 0
1+ . . . k-1
0
1k-1
cost(OPT) = 1+ε
57
Example: High Price of Stability
1 1k
12
13
1 2 3 k
t
0 0 0 0
1+ . . . k-1
0
1k-1
cost(OPT) = 1+ε
…but not a NE:
player k
pays (1+ε)/k,
could pay 1/k
58
Example: High Price of Stability
1 1k
12
13
1 2 3 k
t
0 0 0 0
1+ . . . k-1
0
1k-1
so player k
would deviate
59
Example: High Price of Stability
1 1k
12
13
1 2 3 k
t
0 0 0 0
1+ . . . k-1
0
1k-1
now player k-1
pays (1+ε)/(k-1),
could pay 1/(k-1)
60
Example: High Price of Stability
1 1k
12
13
1 2 3 k
t
0 0 0 0
1+ . . . k-1
0
1k-1
so player k-1
deviates too
61
Example: High Price of Stability
1 1k
12
13
1 2 3 k
t
0 0 0 0
1+ . . . k-1
0
1k-1
Continuing this process, all players defect.
This is a NE!
(the only Nash)
cost = 1 + + … +
Price of Stability is Hk = Θ(log k)!
1 12 k
62
The Price of Stability of Selfish Network Design
Thus: the price of stability of selfish network design can be as high as ln k. [k = # players]
Our goals: in all such games,• there is at least one pure-strategy Nash eq• one of them has cost ≤ ln k • OPT
– i.e. price of stability always ≤ ln k– [Anshelevich et al 04]
Technique: potential function method.
63
Potential Functions
Recall: potential function Փ of a game = function optimized by selfish players– not necessarily a natural objective function
Defn: Փ (fn from outcomes to reals) is a potential function if for all outcomes S, players i, and deviations by i from S:
ΔՓ = Δci
64
Potential Functions
So: potential fn tracks deviations by players
Thus: equilibria of game = local optima of Փ• so finite potential games have pure-strategy
Nash equilibria (proof: just do "best-response dynamics") [Monderer/Shapley 96]– precursors: [Rosenthal 73], [Beckmann et al 56]
65
Potential Functions
So: potential fn tracks deviations by players
Thus: equilibria of game = local optima of Փ• so finite potential games have pure-strategy
Nash equilibria (proof: just do "best-response dynamics") [Monderer/Shapley 96]– precursors: [Rosenthal 73], [Beckmann et al 56]
Claim: every Shapley network design game has a potential function.
66
Proof of Potential Function
Define Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke]
where ke is # players using e in S. Hk
Let Ф(S) = Σ Фe(S)
Consider some solution S (a path for each player).
Suppose player i is unhappy and decides to deviate.
What happens to Ф(S)?
e є S
e
67
Proof of Potential Function
Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke]
Suppose player i’s new path includes e.i pays ce/(ke+1) to use e.
Фe(S) increases by the same amount.
If player i leaves an edge e ’, Фe ’(S) exactly reflects the
change in i’s payment.
e
e’
ce[1+ 1/2 +… +1/ke]
ce’[1+ 1/2 +… +1/ke’]
i
68
Proof of Potential Function
e
e’
ce[1+ 1/2 +… +1/ke]+ce/(ke+1)
ce’[1+ 1/2 +… +1/ke’] -ce’/ke’
i
Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke]
Suppose player i’s new path includes e.i pays ce/(ke+1) to use e.
Фe(S) increases by the same amount.
If player i leaves an edge e ’, Фe ’(S) exactly reflects the
change in i’s payment.
69
Bound on Price of StabilityCompare cost + potential function:
C(S) = e ce
PF(S) = e ce[1+ 1/2 + 1/3 + … 1/ke]
• cost, potential fn differ by factor of ≤ Hk
• gives upper bound of Hk on price on stability– let S = min-potential soln [note: also a Nash eq]– let S* = opt solution
C(S) ≤ PF(S) ≤ PF(S*) ≤ Hk • C(S*)
70
Undirected Networks
Open Question: what is the POS in undirected graphs?
• best known lower bound = 12/7• [Fiat et al 06]: O(log log k) for special case
1 1k12 13
= =
t
0 0 0 0
1+ . . .0
1k-1
71
Shapley Cost-Sharing
Summary: with Shapley cost sharing,• POA = k, even in undirected graphs
• POS = Hk in directed graphs– (unknown in undirected graphs)
Question #1: can we do better?
Question #2: subject to what?
72
In Defense of Shapley
Essential properties: (non-negotiable)• "budget-balanced" (total cost shares = cost)• "local" (cost shares computed edge-by-edge)• pure-strategy Nash equilibria exist
Bonus good properties: (negotiable)• "uniform" (same definition for all networks)• "fair" (characterizes Shapley)
73
Other Cost Shares?
Theorem: [Chen/Roughgarden/Valiant 07] Shapley minimizes POS among all uniform protocols in directed graphs.– Shapley justified on efficiency grounds!– non-uniform schemes not well understood
74
Other Cost Shares?
Theorem: [Chen/Roughgarden/Valiant 07] Shapley minimizes POS among all uniform protocols in directed graphs.– Shapley justified on efficiency grounds!– non-uniform schemes not well understood
Theorem: [Chen/Roughgarden/Valiant 07] Can do much better in undirected graphs.– can get POA = O(log2 k)– better for special cases or non-uniform protocols
75
Wrap-Up
• network games arise in many CS applications
• price of anarchy/stability/etc a flexible tool to measure inefficiency of selfish behavior– future direction: inform protocol design
• potential functions are an easy-to-use, versatile techniques to bound POA/POS
• many open questions...– looking forward to future theorems from you!