Ryan O’Donnell (CMU, IAS)

joint work with

Ankur Moitra (MIT)

Binary optimization, linear objective

F ⊆ {0,1}n, “feasible solutions”

max

Binary optimization, linear objectives

F ⊆ {0,1}n, “feasible solutions”

“max”

Pareto optima

def: x ∈ F is Pareto optimal if not dominated on every objective by some y ∈ F.

Obj1

Obj2

xy

Pareto optima

Obj1

Obj2

xy

def: x ∈ F is Pareto optimal if not dominated on every objective by some y ∈ F.

Obj1

Obj2

thm:

If d+1 linear objectives are “semi-random”

(in the Smoothed Analysis sense)

then for any F ⊆ {0,1}n,

E[# Pareto optima in F] ≤ O(n2d).

Application: 0-1 Knapsack

[Nemhauser−Ullmann69]:

For i=1…n, compute Pareto optimal

weight,value pairs achievable by items 1…i.⟨ ⟩

[Beier−Vöcking03]:

With 2 semi-rand objs, E[# PO’s] ≤ O(n4).

⇒ O(n5)-time alg for 0-1 Knapsack in

Smoothed Analysis model!

Model [BV03]

F ⊆ {0,1}n arbitrary

d+1 linear objs

x

=

Obj(x)

ϕ ≥ 1 a parameter

Each is a r.v. on [0,1] with pdf bdd by ϕ.

All independent.

Model [BV03]

Each is a r.v. on [0,1] with pdf bdd by ϕ.

All independent.

1/ϕ

ϕ

Prior work

[BV03]: d = 1, E[# PO’s] ≤ O(ϕn4)

[B04]: Same for arbit. F. Conj: ≤ O(nf(d))

[BRV07]: d = 1, E[# PO’s] ≤ O(ϕn2)

[RT09]: E[# PO’s] ≤ roughly

[MO11]: E[# PO’s] ≤ 2(4ϕd)d(d+1)/2 n2d

Remark: All bounds here hold even assuming Obj0 adversarial, nonlinear, arbitrary.

for n ≥ exp(exp(O(d2 log d))

(F = {0,1}n)

[BV03]: Ω(n2) when Obj0 adversarial, ϕ = 1

[BR11]: Ω(ϕn2), d = 1; Ω(ϕn)d-logd roughly, d > 1

Warmup:

Alternate proof (sketch) for

[BRV07]’s d = 1 bound

F ⊆ {0,1}nObj0(x) arbitrary

Obj1(x) ϕ-semi-random E[# PO’s] ≤ O(ϕn2) ?

Obj0 01011

10100

01101

01111

01000

11011

00111

11110

Obj1

.8.4

Obj0 01011

10100

01101

01111

01000

11011

00111

11110

.2 .9

Obj0(x) arbitrary

Obj1(x) ϕ-semi-random

Obj1

F ⊆ {0,1}n

E[# PO’s] ≤ O(ϕn2) ?

Goal: For each ϵ-strip S,

E[# PO’s in S] ≈ Pr[∃ a PO in S] ≤ O(n) ϕ ϵ

Obj0 01011

10100

01101

01111

01000

11011

00111

11110

ϵ

ϵ2

Obj1

Union-bound over (n/ϵ) strips ⇒ E[# PO’s] ≤ O(ϕn2)

S

Obj0 01011

10100

01111

01000

11011

00111

11110

ϵ

Obj1

Given x, Pr[ Vx ∈ S ] ?

S

01101

Obj0 01011

10100

01111

01000

11011

00111

11110

ϵ

Obj1

Given x, Pr[ Vx ∈ S ] ?

S

Take any j s.t. xj = 1.

01101

Obj0 01011

10100

01101

01111

01000

11011

00111

11110

ϵ

Obj1

Given x, Pr[ Vx ∈ S ] ?

S

Take any j s.t. xj = 1.

.8.4.2 .9

= Pr[ Vj ∈ certain width-ϵ interval] ≤ ϕϵ.

Obj0 01011

10100

01101

01111

01000

11011

00111

11110

ϵ

Given x, Pr[ Vx ∈ S ] ?

S

Take any j s.t. xj = 1.

= Pr[ Vj ∈ certain width-ϵ interval] ≤ ϕϵ.

Boundedness Lemma: Pr[ Vx ∈ S] ≤ O(ϵ),

even just using the randomness of Vj.

Obj0 01011

10100

01101

01111

01000

11011

00111

11110

ϵ

Obj1

S

event Tx,S = “x is PO and Vx ∈ S”

.8.4.2 .9

Obj0 01011

10100

01101

01111

01000

11011

00111

11110

ϵ

Obj1

S

event Tx,S = “x is PO and Vx ∈ S”

.8.4.2 .9

Fantasy: Now a unique x s.t. Tx,S may yet occur

more complicated event:

Tx,j,S = “x is PO and xj = 1 and Vx ∈ S

and first PO to x’s left, call it y, has yj = 0”

Uniqueness Lemma:

Draw all Vi’s except Vj.

Then ∃ at most 1 x s.t. Tx,j,S may still occur.

Tx,j,S = “x is PO and xj = 1 and Vx ∈ S

and first PO to x’s left, call it y, has yj = 0”

Obj0 01011

10100

01101

01111

01000

11011

00111

11110

ϵ

Obj1

S

.8.4.2 .9

Remainder of the d=1 proof sketch

Tx,j,S = “x is PO and xj = 1 and Vx ∈ S and …”

Uniqueness Lemma: Draw all Vi’s except Vj.

Then ∃ at most 1 x s.t. Tx,j,S may still occur.Bddness Lemma: For that x, Pr[Vx ∈ S] ≤ ϕϵ.

Union-bound over all j, S:

E[ # PO x s.t. first PO to x’s left, y, has yj ≠ 1 = xj for some j ] ≤ n(n/ϵ)ϕϵ = n2ϕ.

For each PO x, ∃ j s.t. yj ≠ c = xj. Maybe c = 0…

Union-bound over c ∈ {0,1}… + another trick.

Our result: Larger d

Obj0 arbitrary; Obj1, Obj2 ϕ-semi-random

01011

10100

01101

01111

01000

11011

00111

11110

d=2:

Obj0

Obj1

Obj2

Obj0 01011

10100

01101

01111

01000

11011

00111

11110

Obj1

Obj2

V =

Obj0 01011

10100

01101

01111

01000

11011

00111

11110

Obj2

ϵS

ϵ

Obj1

V =

more complicated event:

Tx,j,S = “x is PO and xj = 1 and Vx ∈ S

and first PO to x’s left, call it y, has yj = 0”

more complicated event:

Tx,j,c,S = “x is PO and xj = c and Vx − ∈ S

and first PO to x’s left, call it y, has yj ≠ c”

more complicated event:

Tx,J,C,S = “x is PO and…”

still

J: list of d coordinates

C: d×d matrix of bits w/ certain pattern

S: d-dimensional strip, plus “nearby” d′-dim. strips for all d′ < d

more complicated event:

J: list of d coordinates

C: d×d matrix of bits w/ certain pattern

S: d-dimensional strip, plus “nearby” d’-dim. strips for all d’ < d

B: “blueprint” [AMR09]

+

=

Tx,B = “x is PO and conditions involving V, x, B.”

still

Given B, entries of V are partitioned into

two sets, V[fewB] and V[mostB].

V =

Tx,B = “x is PO and conditions involving V, x, B.”

Boundedness Lemma: For any V[mostB] outcome,

Pr [condits involving V, x, B] ≤ ϕd(d+1)/2 ϵd(d+1)/2.

Uniqueness Lemma: Draw V[mostB].

Then ∃ at most 1 x s.t. Tx,B may still

occur.

Tx,B = “x is PO and conditions involving V, x, B.”

V[fewB]

Counting Lemma: O(d/ϵ)d(d+1)/2 n2d possible B.

Union bound ⇒ E[# PO’s] ≤ O(dϕ)d(d+1)/2 n2d.

Given B, entries of V are partitioned into

two sets, V[fewB] and V[mostB].

Tx,j,c,S = “x is PO and xj = c and Vx − ∈ S

and first PO to x’s left, call it y, has yj ≠ c”

J: list of d coordinates

C: d×d matrix of bits w/ certain pattern

S: d-dimensional strip, plus “nearby” d′-dim. strips for all d′ < d

B: “blueprint”

+

=

Tx,B = “x is PO and conditions involving V, x, B.”

Tx,B = “x is PO and SKETCH(x,V) = B”

where SKETCH() is a certain

deterministic algorithm.

SKETCH(x,V):

Bddness Lemma: For any V[mostB] outcome,

Pr [SKETCH(x,V) = B] ≤ ϕd(d+1)/2 ϵd(d+1)/2.

Uniqueness Lemma: Draw V[mostB].

Then ∃ at most 1 x s.t. Tx,B may still occur.

V[fewB]

Tx,B = “x is PO and SKETCH(x,V) = B”

Uniqueness Lemma: Draw V[mostB].

Then ∃ at most 1 x s.t. Tx,B may still occur.

Tx,B = “x is PO and SKETCH(x,V) = B”

Equivalent Lemma:

Suppose x, V are such that x is PO.

Assume SKETCH(x,V) = B.

Then RECONSTRUCT(B,V[mostB]) = x.

RECONSTRUCT(B,V[most]):

Bddness Lemma: For any V[mostB] outcome,

Pr [SKETCH(x,V) = B] ≤ ϕd(d+1)/2 ϵd(d+1)/2.V[fewB]

Uniqueness Lemma:

Suppose x, V are such that x is PO.

Assume SKETCH(x,V) = B.

Then RECONSTRUCT(B,V[mostB]) = x.

Counting Lemma: O(d/ϵ)d(d+1)/2 n2d possible B.

A sketch of SKETCH(x,V) for d = 2

SKETCH(x,V):

1. Let F0 = F

2. Let D1 = { z ∈ F0 : V1z > V1x and V2z > V2x }

3. Let y1 = argmax{ Obj0(y) : y ∈ D1

}

4. Let j1 = least index s.t.

5. Let F1 = { z ∈ F0 : Obj0(z) > Obj0(y1) and }

6. Let D2 = { z ∈ F1 : V2z > V2x }

7. Let y2 = argmax{ Obj1(y) : y ∈ D2 }

8. Let j2 = least index s.t.

9. Let F2 = { z ∈ F1 : Obj1(z) > Obj1(y1) and }

// if Vx is PO then it must be argmax{ Obj2(z) : z ∈ F2 }

10. Output: J = (j1, j2), C =

S = ( strip of Obj1,2(x) − diag(Vj1,j2 C), strip of Obj2(x) − )

SKETCH(x,V):

• Close gap of nd vs. n2d.

• Lower bounds when all objs semirandom?

• F ⊆ {0,1,2,…,k}n ?

• Bounds on Var[ # POs ] ?

• More Smoothed Analysis via “blueprints”?

Future directions?