Selfish Routing

8/20/2019 Selfish Routing

1/80

A Glimpse of AGT:

Selfish Routing

Guido SchäferCWI Amsterdam / VU University Amsterdam

[email protected]

Course:

Combinatorial OptimizationVU University Amsterdam

March 12 & 14, 2013

http://find/http://goback/


2/80

Motivation

http://find/


3/80

Situations of strategic interaction

Viewpoint: many real-world problems are complex and

distributed in nature:1 involve several independent decision makers

(lack of coordination)

2 each decision maker attempts to achieve his own goals

(strategic behavior)

3 individual outcome depends on decisions made by others

(interdependent)

Examples:

• routing in networks• Internet applications

• auctions

• ...

Guido Schäfer A Glimpse of Algorithmic Game Theory: Selfish Routing 3

http://goforward/http://find/http://goback/


4/80

Network routing

Network routing: (think of urban road traffic)• large number of commuters want to travel from their origins

to their destinations in a given network

• commuters are autonomous and choose their routes so as to

minimize their individual travel times

• travel time along a network link is affected by congestion• individual travel time depends on the choices made by others

Applications: road traffic, public transportation, Internet

routing, etc.

Observation: selfish route choices lead to inefficient outcomes




5/80

Inefficient outcomes


http://find/


6/80

Consequences

Negative consequences:

• environmental pollution• waste of natural resources, time

and money

• stress on the traffic participants

• ...

“In 2010, congestion caused urban Americans to travel 4.8 billion hours more and to purchase an extra 1.9 billion gallons of fuel for a congestion cost of $101 billion.”

[Texas Transportation Institute, 2011 Urban Mobility Report]

Need: gain fundamental understanding of the effect of strategic

interactions in such applications


http://find/


7/80

Approach

Game theory: provides mathematical toolbox to study

situations of strategic interaction

• different models of games

• several solution concepts for the prediction of “rational

outcomes”

Algorithmic game theory:• use game-theoretical models and solution concepts• take computational/algorithmic issues into account

Goals:• study the effect of strategic behavior

• analyze the efficiency loss due to lack of coordination

• provide algorithmic means to reduce the inefficiency


http://find/


8/80

Today’s goals

Goals for today:

• get a glimpse of the research in algorithmic game theory• running example: network routing

• get an idea of the phenomena and questions that arise

• familiarize with used approaches and techniques• approach the state-of-the-art of recent studies (perhaps)

Algorithmic game theory ...

... is a young, challenging, interdisciplinary research field

... addresses many aspects of practical relevance

... can have an impact on real-world applications

... unites ideas from game theory, algorithms, combinatorial

optimization, complexity theory, etc.

... is fun!




9/80

AGT book

N. Nisan, T. Roughgarden, E. Tardos,

and V. Vazirani (Eds.)

Algorithmic Game Theory Cambridge University Press, 2007.

Available online (free!) at:

http://www.cambridge.org/us/9780521872829


http://www.cambridge.org/us/9780521872829http://www.cambridge.org/us/9780521872829http://find/


10/80

Roadmap

1 Examples of Some Phenomena

2 Wardrop Model and the Price of Anarchy

3 Coping with the Braess Paradox

4 Stackelberg Routing

5 Network Tolls


http://find/


11/80

Examples of Some Phenomena

http://find/


12/80

Cost minimization games

A cost minimization game G is given by

• set of players N = {1, . . . , n }• set of strategies S i for every player i ∈ N • cost function C i : S 1 × · · ·× S n → R

S = S 1 × · · ·× S n is called the set of strategy profiles.

Interpretation:

• every player i ∈ N chooses a strategy s i ∈ S i so as tominimize his individual cost C i (s 1, . . . , s n )

• one-shot, simultaneous-move: players choose their

strategies once and at the same time

• full-information: every player knows the strategies and cost

function of every other player


http://find/


13/80

Example: Congestion game

n = 4

S i = {e 1, e 2}

s t

e 1 (x ) = x

e 2 (x ) = 4


E l C i

http://find/


14/80


n = 4

S i = {e 1, e 2}

s t

e 1 (x ) = x

e 2 (x ) = 4

4


E l C ti

http://find/


15/80


n = 4

S i = {e 1, e 2}

s t

e 1 (x ) = x

1

e 2 (x ) = 4

3


E l C ti



16/80


n = 4

S i = {e 1, e 2}

s t

e 1 (x ) = x

2

e 2 (x ) = 4

2


E l C ti

http://find/


17/80


n = 4

S i = {e 1, e 2}

s t

e 1 (x ) = x

3

e 2 (x ) = 4

1


Nash equilibrium and social cost

http://find/


18/80

Nash equilibrium and social cost

Nash equilibrium: strategy profile s = (s 1, . . . , s n ) ∈ S is apure Nash equilibrium (PNE) if no player has an incentive to

unilaterally deviate

∀i ∈ N : C i (s i , s −i ) ≤ C i (s

i , s −i ) ∀s

i ∈ S i

(s −i refers to (s 1, . . . , s i −1, s i +1, . . . , s n ))

Social cost of strategy profile s = (s 1, . . . , s n ) ∈ S is

C (s ) = i ∈N

C i (s )

A strategy profile s ∗ that minimizes the social cost function C (·)is called a social optimum.



http://find/


19/80


n = 4

s t

x

4



http://find/


20/80


n = 4

s t

x

4

4

Nash equilibrium: C (s ) = 4 · 4 = 16



http://find/


21/80


n = 4

s t

x

2

4

2

social optimum: C (s ∗) = 4 + 8 = 12



http://find/


22/80


n = 4

s t

x

2

4

2

inefficiency: C (s )C (s ∗) =

1612 =

43



http://find/


23/80


n = 4

s t

x

4



http://find/


24/80


n = 4

s t

x

3

4

1

Nash equilibrium: C (s ) = 9 + 4 = 13



http://find/


25/80


n = 4

s t

x

3

4

1

inefficiency: C (s )C (s ∗) =

1312 ≈ 1.08


Inefficiency of equilibria

http://find/


26/80


Price of anarchy: worst-case inefficiency of equilibria

POA(G ) = maxs ∈PNE (G )

C (s )C (s ∗)

[Koutsoupias, Papadimitriou, STACS ’99]

Price of stability: best-case inefficiency of equilibria

POS (G ) = mins ∈PNE (G )

C (s )

C (s ∗)

[Schulz, Moses, SODA ’03]

Define the price of anarchy/stability of a class of games G as

POA(G ) = maxG ∈G

POA(G ) and POS (G ) = maxG ∈G

POS (G )


Example: Braess Paradox

http://find/


27/80


n =

4

s t

x 4

4 x



http://find/


28/80


n =

4

s t

x 2

4

42

x

Nash equilibrium: C (s ) = 24



http://find/


29/80

a p e aess a ado

n =

4

s t

x 4

4 x



http://find/


30/80

p

n = 4

s t

x 4

4 x

0



http://find/


31/80

p

n = 4

s t 0

4

4

x

40

x

Nash equilibrium: C (s ) = 32



http://find/


32/80

p

n = 4

s t 0

4

4

x

40

x

cost increase: 3224 = 43


http://find/


33/80

[New York Times Article]

Example: Network tolls

http://find/


34/80

n = 4

s t

x

4



http://find/


35/80

n = 4

s t

x +2

4



http://find/


36/80

n = 4

s t

x +2

4

2

2

Nash equilibrium = social optimum


http://find/


37/80

Wardrop Model and the Price of

Anarchy

Wardrop model

http://find/


38/80

x

a (x )

Given:

• directed network G = (V , A)

• k commodities (s 1, t 1), . . . , (s k , t k )

• demand r i for commodity i ∈ [k ]

• flow-dependent latency function a (·) for every arc a ∈ A(non-decreasing, differentiable, semi-convex)

Interpretation:

• send r i units of flow from s i to t i , corresponding to aninfinitely large population of non-atomic players

• latency experienced on arc a ∈ A is a (x ), where x is theamount of flow on arc a

• selfishness: players choose minimum-latency paths

• one-shot, simultaneous-move, full information game


Optimal flows

http://find/


39/80

Notation:

• P i = set of all (simple) directed s i , t i -paths

• P = ∪i ∈[k ]P i = set of all relevant paths

• flow f is a function f : P → R+

• f is a feasible flow if

P ∈P i f P = r i for all i

• flow on arc a : f a

:= P ∈P :a ∈P

f P

• latency of path P : P (f ) :=

a ∈P a (f a )

Social cost of flow:

C (f ) =i ∈[k ]

P ∈P i

f P P (f ) =a ∈A

f a a (f a )

Optimal flow: feasible flow f ∗ that minimizes C (·)


Nash flows

http://find/


40/80

Nash flow: feasible flow f that satisfies for every i ∈ [k ]

∀P 1

,

P 2 ∈

P i with

f P 1 > 0, ∀δ ∈

(0,

f P 1 ] :

P 1 (f )≤

P 2 (

˜

f ),

where

f̃ P :=

f P − δ if P = P 1

f P + δ if P = P 2

f P otherwise.

Exploiting continuity of a (·) and letting δ → 0, we obtain thefollowing equivalent definition:

Wardrop flow: feasible flow f that satisfies for every i ∈ [k ]

∀P 1, P 2 ∈ P i with f P 1 > 0 : P 1 (f ) ≤ P 2 (f )



http://find/


41/80



http://find/


42/80

Nash flow f : C (f ) = 1



http://find/


43/80

Optimal flow f

∗

: C (f

∗

) =

1

2 ·

1

2 +

1

2 · 1 =

3

4


Existence and computation of optimal flows

http://find/


44/80

min a ∈A

a (f a )f a

s.t.

P ∈P i

f P = r i ∀i ∈ [k ]

f a =

P ∈P :a ∈P

f P ∀a ∈ A

f P ≥ 0 ∀P ∈ P

Observations:

• minimize a continuous function (social cost) over a compact

set (feasible flows)

• optimal flow must exist (extreme value theorem)

• optimal flow can be computed efficiently (convex program)


KKT optimality conditions

http://find/


45/80

mina ∈A

h a (f a )

s.t.

P ∈P i

f P = r i ∀i ∈ [k ]

f a =

P ∈P :a ∈P

f P ∀a ∈ A

f P ≥ 0 ∀P ∈ P

(CP)

Theorem (KKT optimality conditions)

Suppose (h a )a ∈A are continuously differentiable and convex. f is

an optimal solution for (CP) if and only if for every i ∈ [k ]

∀P 1, P 2 ∈ P i , f P 1 > 0 :

a ∈P 1

h a (f a ) ≤

a ∈P 2

h a (f a )


Existence and computation of Nash flows

http://find/


46/80

Idea: Can we determine functions (h a )a ∈A such that KKToptimality conditions reduce to Nash flow conditions?

mina ∈A

f a

0

a (x )dx

s.t.

P ∈P i

f P = r i ∀i ∈ [k ]

f a = P ∈P :a ∈P

f P ∀a ∈ A

f P ≥ 0 ∀P ∈ P

Observations:

• optimal solutions of (CP) above coincides with Nash flows• Nash flow must exist (extreme value theorem)

• similar argument: cost of Nash flow is unique

• Nash flow can be computed efficiently (convex program)


Characterization of optimal flows

http://find/


47/80

min a ∈A

a (f a )f a

s.t.

P ∈P i

f P = r i ∀i ∈ [k ]

f a =

P ∈P :a ∈P

f P ∀a ∈ A

f P ≥ 0 ∀P ∈ P

Applying KKT optimality conditions yields:

Theorem

f is an optimal flow with respect to (a )a ∈A if and only if f is a Nash flow with respect to (∗a )a ∈A, where

∗

a (x ) = (x · a (x )).


Price of anarchy in selfish routing



48/80

Price of anarchy: given an instance I = (G , (s i , t i ), (r i ), (a ))

POA(I ) = maxf Nash flow

C (f )

C (f ∗)

POA for selfish routing is independent of the network topology,

but depends on the class of latency functions:

• POA = 43 for linear latency functions

• POA = Θ( p ln p ) for polynomial latency functions of degree p

• POA is unbounded in general[Roughgarden, Tardos, JACM ’02]


http://find/


49/80

[Upper bound on POA for linear latency functions]

POA for polynomial latency functions

http://find/


50/80

p 1 2 3 . . .

POA ≈ 1.333 ≈ 1.626 ≈ 1.896

Table: POA for polynomial latency functions of degree p

Theorem

Let (a )a ∈A be polynomial latency functions of degree at most p.

POA ≤

1 −

p

(p + 1)(p +1)/p −1

=Θ p

ln p

[Roughgarden, Tardos, JACM ’02]


Example: Unbounded POA

http://find/


51/80



http://find/


52/80

Nash flow f : C (f ) = 1



http://find/


53/80

Optimal flow f ∗: C (f ∗

) = (1−

)

1+p

+

· 1 → 0



http://find/


54/80

POA →∞ as p →∞


Reducing inefficiency

http://find/


55/80

Means to reduce inefficiency:

• infrastructure improvement (expensive, Braess paradox)

• centralized routing (difficult to implement)

• Stackelberg routing (control part of the traffic)• traffic regulation (taxes on fuel, pay-as-you-go tax)

• network tolls


http://find/


56/80

Coping with the Braess Paradox

Braess subgraph problem

http://find/


57/80

Consider a single-commodity instance I = (G , (s , t ), r , (a )) withlinear latency functions. Define

d (H ) = common latency of flow carrying paths in aNash flow for H ⊆ G

Braess subgraph problem: Given (G , (s , t ), r , (a )), find a

subgraph H ⊆ G that minimizes d (H ).

Algorithm TRIVIAL: simply return the original graph G

TheoremTRIVIAL is a 43 -approximation algorithm for the Braess subgraph problem.


Braess subgraph problem

Th

http://find/


58/80

Theorem

TRIVIAL is a 43 -approximation algorithm for the Braess subgraph

problem.

Proof:

1 Algorithm computes a feasible solution in polynomial time.

2 Consider an arbitrary subgraph H of G . Let

f = Nash flow for G and h = Nash flow for H

The bound of 43 on the price of anarchy yields

C (f ) ≤ 43 · C (f ∗) ≤ 4

3 · C (h )

r · d (G ) = C (f ) ≤ 4

3 · C (f ∗) ≤

4

3 · C (h ) =

4

3 · r · d (H )


As good as it gets...

http://find/


59/80

Theorem

Assuming P = NP, for every ε > 0 there is no approximation algorithm for the Braess subgraph problem achieving an approximation guarantee of ( 43 − ε).

Proof idea: A ( 43 − ε)-approximation algorithm can be used todetermine whether there exist two disjoint paths between (s 1, t 1)and (s 2, t 2) in a directed graph. The latter problem is

NP-complete.


http://find/


60/80

Network Tolls

Reducing inefficiency

M t d i ffi i

http://find/


61/80

Means to reduce inefficiency:

• infrastructure improvement (expensive, Braess paradox)

• centralized routing (difficult to implement)

• Stackelberg routing (control part of the traffic)

• traffic regulation (taxes on fuel, pay-as-you-go tax)

• network tolls

Advantages of network tolls:

• fine-grained means: impose “local” taxes

• new technologies facilitate electronic collection of tolls

• dynamic pricing schemes have potential to changeparticipants’ behavior

• . . .


Dynamic pricing: ERP in Singapore

http://find/


62/80


Road pricing

http://find/


63/80

Network tolls: impose non-negative tolls τ := (τ a )a ∈A on arcs

• dynamic: flow-dependent function τ a (x ) (non-decreasing,continuous)

• static: flow-independent constant τ a

Interpretation: by traversing an arc a ∈ A with flow value x , aplayer experiences a delay of a (x ) and has to pay a toll of τ a (x )

Assumption: players are homogeneous

φa (x ) := a (x ) + α · τ a (x )

α specifies how players value time over money (w.l.o.g. α = 1)


Marginal cost tolls

Marginal cost tolls: τa(x) := x · (x) ∀a ∈ A

http://find/


64/80

Marginal cost tolls: τ a (x ) := x · a (x ) ∀a ∈ A

Theoremf is an optimal flow with respect to iff f is a Nash flow with respect to + τ .

[Beckman, McGuire and Winsten 1956]

Drawbacks:

1 potentially every arc of the network is tolled

2 imposed tolls can be arbitrarily large

Most previous studies: no restrictions on tolls that can be

imposed on the arcs of the network

Question: What can be done if such restrictions are given

exogenously?


Effective road pricing in Singapore

http://find/


65/80


http://find/


66/80

Restricted Network Toll Problem

Restricted Network Toll Problem

http://find/


67/80

Our setting: suppose we are given threshold functions

θ := (θa )a ∈A (dynamic or static)

Tolls τ = (τ a )a ∈A are θ-restricted if

∀a ∈ A : 0 ≤ τ a (x ) ≤ θa (x ) ∀x ≥ 0

Special cases:

1 taxing subnetworks: allow tolls only on arcs T ⊆ A

2 restrict the toll on each arc a ∈ A by a (flow-independent)threshold value θa

3 toll on each arc a ∈ A does not exceed a certain fraction ofthe latency of that arc, e.g., θa (x ) = εa (x ) for some ε > 0


Efficiency measures

Given θ-restricted tolls τ , let f τ denote a Nash flow that is

http://find/


68/80

,

induced by τ , i.e., f τ is a Nash flow with respect to + τ .

θ-restricted tolls τ are ρ-approximate if for all Nash flows f τ̄

induced by θ-restricted tolls τ̄

C (f τ ) ≤ ρ · C (f τ̄ )

The efficiency of θ-restricted tolls is

minθ-restricted tolls τ

C (f τ )

C (f ∗)

Note: interested in the cost of the induced Nash flow (system

performance) rather than the total disutility of the players


Results in a nutshell

http://find/


69/80

1 Derive an algorithm to compute optimal θ-restricted tolls for

parallel-arc networks and affine latency/threshold functions• algorithm works for general latency/threshold functions• can guarantee polynomial running time only for affine case• key: characterize inducible flows for single-commodity networks

2 Provide bounds on the efficiency of θ-restricted tolls formulti-commodity networks and polynomial latency functions

• derive a pricing scheme that realizes respective efficiency• bounds are tight, even for parallel-arc networks• approach based on (λ, µ)-smoothness technique• insight: impose tolls on arcs that are sensitive to flow changes


Related work

i l t i b t k

http://find/


70/80

1 special case: taxing subnetworks

• NP-hard to compute optimal tolls for two-commodity networks

and affine latency functions• algorithm to compute optimal tolls for parallel-arc networks and

affine latency functions[Hoefer, Olbrich, Skopalik, WINE 08]

2

existence/computation of optimal-inducing tolls forheterogenous players

• single-commodity, non-atomic [Cole, Dodis, Roughgarden, STOC 03]• multi-commodity, non-atomic [Fleischer, Jain, Mahdian, FOCS 04]

[Karakostas, Kolliopoulos, FOCS 04]

• multi-commodity, atomic [Swamy, SODA 07]

3 price of anarchy and price of stability of ε-Nash flows[Christodoulou, Koutsoupias, Spirakis, ESA 09]


http://find/


71/80

Computing Optimal Tolls for

Parallel-Arc Networks

Characterization of inducible flows

Question: Given an arbitrary flow f , can we determine whetherf i i d ibl b θ t i t d t ll ?

http://find/


72/80

f is inducible by θ-restricted tolls?

Note: If f is given, then a (f a ) and θa (f a ) are constants.

Nash conditions: f is a Nash flow with respect to + τ if

∀P 1, P 2 ∈ P with f P 1 > 0 : ( + τ )P 1 (f ) ≤ ( + τ )P 2 (f )

Can describe the set of θ-restricted tolls that induce f as

{ (τ a )a ∈A | δ v ≤ δ u + a + τ a ∀a = (u , v ) ∈ A \ A+

δ v = δ u + a + τ a ∀a = (u , v ) ∈ A+

τ a ≥ 0 ∀a ∈ Aτ a ≤ θa ∀a ∈ A },

where A+ := {a ∈ A : f a > 0}.


Characterization of inducible flows

Auxiliary graph: Ĝ (f ) = (V , Â) with arc-costs c : Â → R:

http://find/


73/80

• for every arc (u , v ) ∈ A: introduce a forward arc (u , v ) ∈ Â

with c (u ,v ) := (u ,v ) + θ(u ,v )• for every arc (u , v ) ∈ A+: introduce a backward arc

(v , u ) ∈ Â with c (v ,u ) := −(u ,v )

Theoremf is inducible by θ-restricted tolls if and only if Ĝ (f ) does not contain a cycle of negative cost.

[Bonifaci, Mahyar, Schäfer, SAGT ’11]

Remark: can also extract respective θ-restricted tolls from Ĝ (f )

Computing optimal tolls: compute a minimum cost flow f suchthat Ĝ (f ) does not contain a negative cycle


Algorithm for parallel-arc networks

Goal: compute a minimum cost flow f such that

http://find/


74/80

Goa co pu e a u cos o suc a

∀a ∈ A, f a > 0 : a (f a ) ≤ a

(f a

) + θa

(f a

) ∀a

∈ A.

s t ...




http://find/


75/80

p

∀a ∈ A, f a > 0 : a (f a ) ≤ a

(f a

) + θa

(f a

) ∀a

∈ A.

s t ...ordered bydecreasinga (0) + θa (0)




http://find/


76/80

p

∀a ∈ A, f a > 0 : a (f a ) ≤ a

(f a

) + θa

(f a

) ∀a

∈ A.

s t

a

...ordered bydecreasinga (0) + θa (0)

suppose we knew the arc a with f a = 0 and a (0) + θa (0) minimum




http://find/


77/80

p

∀a ∈ A, f a > 0 : a (f a ) ≤ a

(f a

) + θa

(f a

) ∀a

∈ A.

s t

0

a






http://find/


78/80

∀a ∈ A, f a > 0 : a (f a ) ≤ a

(f a

) + θa

(f a

) ∀a

∈ A.

s t

0

a






http://find/


79/80

∀a ∈ A, f a > 0 : a (f a ) ≤ a (f a ) + θa (f a ) ∀a ∈ A. (1)

Algorithm:

1 Guess the minimum value z = a (0) + θa (0) of a zero-flowarc a in a minimum cost flow satisfying (1).

2 Set f a = 0 for every arc a ∈ A with a (0) > z .3 Let A = {a ∈ A | a (0) ≤ z } be the remaining arcs and solve

min

a ∈A f a a (f a )s.t. a ∈A f a = r

f a ≥ 0 ∀a ∈ A

a (f a ) ≤ z ∀a ∈ A

a (f a ) ≤ a (f a ) + θa (f a ) ∀a , a ∈ A


Main result

Theorem

http://find/


80/80

The algorithm computes optimal θ-restricted tolls for parallel-arc

networks in polynomial time if all latency and threshold functions are affine .

[Bonifaci, Mahyar, Schäfer, SAGT ’11]

Remark: [Hoefer et al., WINE 08] derived a similar result for the

special case that restrictions are of the form θa ∈ {0,∞} forevery arc a ∈ A.

Recall: problem is NP-hard for two-commodity networks and

affine latency functions

Open problem: Is the restricted network toll problem NP-hard

for single-commodity networks and affine latency functions?

http://find/

Documents

Selfish Routing