COLOR TESTCOLOR TESTCOLOR TESTCOLOR TEST
Dueling Algorithms
NICOLE IMMORLICA, NORTHWESTERN UNIVERSITY
WITH A. TAUMAN KALAI, B. LUCIER, A. MOITRA, A. POSTLEWAITE, AND M. TENNENHOLTZ
Social Contexts
Normal-form games:Players choose strategies to maximize expected von Neumann-Morgenstern utility.
Social context games [AKT’08]:Players choose strategies to achieve particular social status among peers.
Social Contexts
Ranking games [BFHS’08]:Players choose strategies to achieve particular payoff rank among peers.
Two-Player Ranking Games
GAlice
Bob
RG payoff of Alice:
1 Alice beats Bob in G
Alice and Bob play game:
½ Alice ties Bob in G
0 Alice loses to Bob in G
Implicit Representations
Succinct games [FIKU’08]:Payoff matrix represented by boolean circuit. NE hard to solve or approximate.
Blotto games [B’21, GW’50, R’06, H’08]:Distribute armies to battlefields.
Implicit Representations
Optimization duels [this work]:Underlying game is optimization problem. Goal is to optimize better than opponent.
Ranking Duel
A search engine is an algorithm that inputs• set Ω = {1, 2, …, n} of items• probabilities p1 + … + pn = 1 of eachand outputs a permutation π of Ω.
Monopolist objective: minimize Ei~p[π(i)].
Ranking Duel
Competitive objective: Let the expected score of a ranking π versus a ranking π’ be
Pri~p[ π(i) < π’(i) ] + (½) Pri~p[ π(i) = π’(i) ].
Then objective is to output a π that maximizes expected score given algorithm of opponent.
Optimizing a Search Engine
?
User searches for object drawn according to known probability dist.
0.19 0.16 0.27 0.07 0.22 0.09
Search: pretty shape
1. (27%)
2. (22%)
3. (19%)
4. (16%)
5. (09%)
6. (07%)
Greedy is optimal.
Choosing a Search Engine
1. Search for “pretty shape”.2. See which search engine ranks
my favorite shape higher.3. Thereafter, use that one.
0.19 0.16 0.27 0.07 0.22 0.09
Search: pretty shape
1. (27%)
2. (22%)
3. (19%)
4. (16%)
5. (09%)
6. (07%)
Search: pretty shape
6. (27%)
1. (22%)
2. (19%)
3. (16%)
4. (09%)
5. (07%)
Questions
Can we efficiently compute an equilibrium of a ranking duel?
How poorly does greedy perform in a competitive setting?
What consequences does the duel have for the searcher?
Optimization Problems as DuelsRanking Binary Search Routing
ParkingCompressionHiring
Start
Finish
?
??
??
?
?
Duel Framework
Finite feasible set X of strategies.Prob. distribution p over states of nature Ω.Objective cost c: Ω × X R.
Monopolist: choose x to minimize Eω~p[cω(x)].
Duel Framework
1. Players select strategies x, x’ from X.2. Nature selects state ω from Ω according to p.3. Payoffs v(x,x’), (1-v(x,x’)) are realized.
1 if cω(x) < cω(x’)
0 if cω(x) > cω(x’)
½ if cω(x) = cω(x’)
v(x,x’) = Eω~p
Results: Computation
An LP-based technique to compute exact equilibria, A low-regret learning technique to compute approximate equilibria,
… and a demonstration of these techniques in our sample settings
Computing Exact Equilibria
Formulate game as bilinear duel:1. Efficiently map strategies to points X in Rn.2. Define constraints describing K=convex-hull(X).3. Define payoff matrix M that computes values.4. Maps points in K back to strategies in original
setting.
Bilinear Duels
If feasible strategies X are points in Rn, and payoff v(x, x’) is xtMx’ for some M in Rnxn, then
maxv,x v s.t. xtMx’ ≥ v for all x’ in X x is in K (=convex-hull(X))
Exponential, but equivalent poly-sized LP.
Ranking Duel
Formulate game as bilinear duel:1. Efficiently map strategies to points X in Rn.
X = set of permutation matrices(entry xij indicates item i placed in position j)
2. Define constraints describing K=convex-hull(X).K = set of doubly stochastic matrices(entry yij = prob. item i placed in position j)
Ranking Duel
Formulate game as bilinear duel:4. Design “rounding alg.” that maps points in K back
to strategies in original setting.Birkhoff–von Neumann Theorem: Can efficiently construct permutation basis for doubly stochastic matrix (e.g., via matching).
Ranking Duel
Formulate game as bilinear duel:3. Define payoff matrix M that computes values.
Ep,y,y’[v(x,x’)] = ∑i p(i) ( ½ Pry,y’ [x i = x’i ] + Pry,y’ [x i > x’i ])= ∑i p(i) (∑i yij ( ½ y’ij + ∑k>j y’ik ))
which is bilinear in y,y’ and so can be written ytMy’.
Ranking Duel
Result: Can reduce computation time to poly(n) versus poly(n!) with standard LP approach.
Technique also applies to hiring duel and binary search duel.
Compression Duel
data
Goal: smaller compression (i.e., lower depth in tree).
(each with prob. p(.))
Classical Algorithm
Huffman coding:Repeatedly pair nodes with
lowest probability.
Compression Duel
Formulate game as bilinear duel:1. Efficiently map strategies to points X in Rn.
X = subset of zero-one matrices*(entry xij indicates item i placed at depth j)
2. Define constraints describing K=convex-hull(X).K = subset of row-stochastic matrices*(entry yij = prob. item i placed at depth j)
* Must correspond to depth profile of some binary tree!
Compression Duel
Formulate game as bilinear duel:3. Define payoff matrix M that computes values.
Ep,y,y’[v(x,x’)] = ∑i p(i) (∑i yij ( ½ y’ij + ∑k>j y’ik ))
which is bilinear in y,y’ and so can be written ytMy’.
Compression Duel
Bilinear Form:maxv,x v s.t. xtMx’ ≥ v for all x’ in X x is in K (=convex-hull(X))
Problems:1. How to round points in K back to a random binary tree with right depth profile?2. How to succinctly express constraints describing K?
Approximate Minimax
Defn. For any ε > 0, an approximate minimax strategy guarantees payoff not worse than best possible value minus ε.
Defn. For any ε > 0, an approximate best response has payoff not worse than payoff of best response minus ε.
Best-Response Oracle
Idea. Use approximate best-response oracle to get approximate minimax strategies.
1. Low-regret learning: if x1,…,xT and x’1,…,x’T have low regret, then ave. is approx minimax.2. Follow expected leader: on round t+1, play best-response to x1,…,xt to get low-regret.
Compression Best-Response
Given lists of items with values and weights, pick one from each list with max total value and total weight at most one.
Multiple-choice Knapsack:
Compression Best-Response
Depth: 1 2 3 4
Compression Best-Response
(each with prob. p(.)) x’ in K
For j from 1..n, list of depth j:
v( ) = Pr[win at depth j | x’ ]w( ) = 2-j
… Kraft inequality
Other Duels
1. Hiring duel: constraints defining Euclidean subspace correspond to hiring probabilities.
2. Binary search duel: similar to hiring duel, but constraints defining Euclidean subspace more complex (must correspond to search trees).
3. Racing duel: seems computationally hard, even though single-player problem easy.
Conclusion
• Every optimization problem has a duel.• Classic solutions (and all deterministic
algorithms) can usually be badly beaten.• Duel can be easier or harder to solve, and can
lead to inefficiencies.
OPEN QUESTION: effect of duel on the solution to the optimization problem?