Lecture: ApproximationAlgorithms · Basicinformation Lectures: Mondays,12:15-13:45...

Jannik Matuschke

October 24, 2016

Lecture: Approximation AlgorithmsLecture: Approximation Algorithms

Basic information

Lectures: Mondays, 12:15-13:45

Exercise sessions: Tuesdays (bi-weekly) 16:00-18:00(given by Marcus Kaiser)

Book: The Design of Approximation Algorithmsby D. Williamson and D. Shmoys

More info on the website.

Introduction toApproximation Algorithms

Motivation

Algorithmic wishlist1 fast (run in polynomial time)

2 universal (work for any instance)

3 optimal (find best solution)

Choose two.(unless P = NP)

Motivation

Algorithmic wishlist1 fast (run in polynomial time)

2 universal (work for any instance)

3 optimal (find best solution)

Choose two.(unless P = NP)

Motivation

Algorithmic wishlist1 fast (run in polynomial time)

2 universal (work for any instance)

3 approximately optimal (find provably good solution)

⇓Approximation Algorithms

Approximation Algorithms

Definition An α-approximation algorithm for an optimizationproblem is an algorithm that

I runs in polynomial time andI computes for any instance of the problem a solution,I whose value is within a factor of α of the optimal solution.

ALG: value of solution computed by algorithmOPT: value of optimal solution

for maximization problems:

ALG ≥ α · OPT

(α ≤ 1)

for minimization problems:

ALG ≤ α · OPT

(α ≥ 1)

We call α approximation factor or performance guarantee.

Approximation Algorithms

Definition An α-approximation algorithm for an optimizationproblem is an algorithm that

I runs in polynomial time andI computes for any instance of the problem a solution,I whose value is within a factor of α of the optimal solution.

ALG: value of solution computed by algorithmOPT: value of optimal solution

for maximization problems:

ALG ≥ α · OPT

(α ≤ 1)

for minimization problems:

ALG ≤ α · OPT

(α ≥ 1)

We call α approximation factor or performance guarantee.

Approximation Algorithms

Definition An α-approximation algorithm for an optimizationproblem is an algorithm that

I runs in polynomial time andI computes for any instance of the problem a solution,I whose value is within a factor of α of the optimal solution.

ALG: value of solution computed by algorithmOPT: value of optimal solution

for maximization problems:

ALG ≥ α · OPT

(α ≤ 1)

for minimization problems:

ALG ≤ α · OPT

(α ≥ 1)

We call α approximation factor or performance guarantee.

Example: Set Cover

The Set Cover problem

Input: elements E , sets S ⊆ 2E , weights w : S → R+

Task: find S ′ ⊆ S with⋃

S∈S′ S = Eminimizing

∑S∈S′ w(S)

32

8

4

Special case: Vertex Cover

The Set Cover problem

Input: elements E , sets S ⊆ 2E , weights w : S → R+

Task: find S ′ ⊆ S with⋃

S∈S′ S = Eminimizing

∑S∈S′ w(S)

32

8

4

Special case: Vertex Cover

The Set Cover problem

Input: elements E , sets S ⊆ 2E , weights w : S → R+

Task: find S ′ ⊆ S with⋃

S∈S′ S = Eminimizing

∑S∈S′ w(S)

32

8

4

Special case: Vertex Cover

How to design an approximation algorithm?

We don’t know OPT,but we can get lower bounds.

How to design an approximation algorithm?

We don’t know OPT,but we can get lower bounds.

IP formulation

min∑S∈S

w(S)x(S)

s.t.∑

S:e∈Sx(S) ≥ 1 ∀ e ∈ E

x(S) ∈ {0, 1} ∀S ∈ S

LP relaxation

Z ∗ :=

min∑S∈S

w(S)x(S)

s.t.∑

S:e∈Sx(S) ≥ 1 ∀ e ∈ E

x(S) ≥ 0 ∀S ∈ S

LP value is lower bound:

Z ∗ ≤ OPT

LP relaxation

Z ∗ :=

min∑S∈S

w(S)x(S)

s.t.∑

S:e∈Sx(S) ≥ 1 ∀ e ∈ E

x(S) ≥ 0 ∀S ∈ S

LP value is lower bound:

Z ∗ ≤ OPT

LP relaxation

Z ∗ := min∑S∈S

w(S)x(S)

s.t.∑

S:e∈Sx(S) ≥ 1 ∀ e ∈ E

x(S) ≥ 0 ∀S ∈ S

LP value is lower bound:

Z ∗ ≤ OPT

(Deterministic) LP Rounding

LP rounding

Idea: Select S if x(S) ≥ 1f . f := maxe∈E |{S : e ∈ S}|

TheoremLP rounding is an f -approximation algorithm for Set Cover.

Proof. S ′ :={

S : x(S) ≥ 1f

}I Is every element covered?∑

S∈S:e∈Sx(S) ≥ 1 ⇒ ∃ S ∈ S : x(S) ≥ 1

f

I Is the approximation factor fulfilled?∑S∈S′

w(S) ≤∑

S∈S′

w(S)·f ·x(S) ≤ f ·∑S∈S

w(S)x(S) = f ·Z ∗

LP rounding

Idea: Select S if x(S) ≥ 1f . f := maxe∈E |{S : e ∈ S}|

TheoremLP rounding is an f -approximation algorithm for Set Cover.

Proof. S ′ :={

S : x(S) ≥ 1f

}I Is every element covered?∑

S∈S:e∈Sx(S) ≥ 1 ⇒ ∃ S ∈ S : x(S) ≥ 1

f

I Is the approximation factor fulfilled?∑S∈S′

w(S) ≤∑

S∈S′

w(S)·f ·x(S) ≤ f ·∑S∈S

w(S)x(S) = f ·Z ∗

LP rounding

Idea: Select S if x(S) ≥ 1f . f := maxe∈E |{S : e ∈ S}|

TheoremLP rounding is an f -approximation algorithm for Set Cover.

Proof. S ′ :={

S : x(S) ≥ 1f

}

I Is every element covered?∑S∈S:e∈S

x(S) ≥ 1 ⇒ ∃ S ∈ S : x(S) ≥ 1f

I Is the approximation factor fulfilled?∑S∈S′

w(S) ≤∑

S∈S′

w(S)·f ·x(S) ≤ f ·∑S∈S

w(S)x(S) = f ·Z ∗

LP rounding

Idea: Select S if x(S) ≥ 1f . f := maxe∈E |{S : e ∈ S}|

TheoremLP rounding is an f -approximation algorithm for Set Cover.

Proof. S ′ :={

S : x(S) ≥ 1f

}I Is every element covered?

∑S∈S:e∈S

x(S) ≥ 1 ⇒ ∃ S ∈ S : x(S) ≥ 1f

I Is the approximation factor fulfilled?∑S∈S′

w(S) ≤∑

S∈S′

w(S)·f ·x(S) ≤ f ·∑S∈S

w(S)x(S) = f ·Z ∗

LP rounding

Idea: Select S if x(S) ≥ 1f . f := maxe∈E |{S : e ∈ S}|

TheoremLP rounding is an f -approximation algorithm for Set Cover.

Proof. S ′ :={

S : x(S) ≥ 1f

}I Is every element covered?∑

S∈S:e∈Sx(S) ≥ 1 ⇒ ∃ S ∈ S : x(S) ≥ 1

f

I Is the approximation factor fulfilled?∑S∈S′

w(S) ≤∑

S∈S′

w(S)·f ·x(S) ≤ f ·∑S∈S

w(S)x(S) = f ·Z ∗

LP rounding

Idea: Select S if x(S) ≥ 1f . f := maxe∈E |{S : e ∈ S}|

TheoremLP rounding is an f -approximation algorithm for Set Cover.

Proof. S ′ :={

S : x(S) ≥ 1f

}I Is every element covered?∑

S∈S:e∈Sx(S) ≥ 1 ⇒ ∃ S ∈ S : x(S) ≥ 1

f

I Is the approximation factor fulfilled?

∑S∈S′

w(S) ≤∑

S∈S′

w(S)·f ·x(S) ≤ f ·∑S∈S

w(S)x(S) = f ·Z ∗

LP rounding

Idea: Select S if x(S) ≥ 1f . f := maxe∈E |{S : e ∈ S}|

TheoremLP rounding is an f -approximation algorithm for Set Cover.

Proof. S ′ :={

S : x(S) ≥ 1f

}I Is every element covered?∑

S∈S:e∈Sx(S) ≥ 1 ⇒ ∃ S ∈ S : x(S) ≥ 1

f

I Is the approximation factor fulfilled?∑S∈S′

w(S) ≤∑

S∈S′

w(S)·f ·x(S)

≤ f ·∑S∈S

w(S)x(S) = f ·Z ∗

LP rounding

Idea: Select S if x(S) ≥ 1f . f := maxe∈E |{S : e ∈ S}|

TheoremLP rounding is an f -approximation algorithm for Set Cover.

Proof. S ′ :={

S : x(S) ≥ 1f

}I Is every element covered?∑

S∈S:e∈Sx(S) ≥ 1 ⇒ ∃ S ∈ S : x(S) ≥ 1

f

I Is the approximation factor fulfilled?∑S∈S′

w(S) ≤∑

S∈S′

w(S)·f ·x(S) ≤ f ·∑S∈S

w(S)x(S) = f ·Z ∗

Consequences

A priori vs. a fortioriI The LP rounding analysis gives us an a priori guarantee:

ALG ≤ f · OPT for any instance of Set Cover.I For a concrete run of the algorithm, we get an a fortiori

guarantee: If ALG /Z ∗ ≤ α, we know that ALG ≤ αOPT.

Integrality gap

The integrality gap of an LP is the ratio Z ∗OPT.

The LP rounding algorithm implies that the integrality gap of theSet Cover LP is bounded by f .

Consequences

A priori vs. a fortioriI The LP rounding analysis gives us an a priori guarantee:

ALG ≤ f · OPT for any instance of Set Cover.I For a concrete run of the algorithm, we get an a fortiori

guarantee: If ALG /Z ∗ ≤ α, we know that ALG ≤ αOPT.

Integrality gap

The integrality gap of an LP is the ratio Z ∗OPT.

The LP rounding algorithm implies that the integrality gap of theSet Cover LP is bounded by f .

The Primal-Dual Method

Primal-dual method

max∑e∈E

y(e)

s.t.∑e∈S

y(e) ≤ w(S) ∀ S ∈ S

y(e) ≥ 0 ∀e ∈ E

Algorithm:1 Initialize y(e) = 0 for all e ∈ E .2 while (∃ uncovered element e′)

Increase y(e) until a set S with e ∈ S becomes tight.Add S to S ′. (

∑e∈S y(e) = w(S))

3 Return S ′.

TheoremPrimal-dual is an f -approximation algorithm for Set Cover.

Primal-dual method

max∑e∈E

y(e)

s.t.∑e∈S

y(e) ≤ w(S) ∀ S ∈ S

y(e) ≥ 0 ∀e ∈ E

Algorithm:1 Initialize y(e) = 0 for all e ∈ E .2 while (∃ uncovered element e′)

Increase y(e) until a set S with e ∈ S becomes tight.Add S to S ′. (

∑e∈S y(e) = w(S))

3 Return S ′.

TheoremPrimal-dual is an f -approximation algorithm for Set Cover.

Primal-dual method

max∑e∈E

y(e)

s.t.∑e∈S

y(e) ≤ w(S) ∀ S ∈ S

y(e) ≥ 0 ∀e ∈ E

Algorithm:1 Initialize y(e) = 0 for all e ∈ E .2 while (∃ uncovered element e′)

Increase y(e) until a set S with e ∈ S becomes tight.Add S to S ′. (

∑e∈S y(e) = w(S))

3 Return S ′.

TheoremPrimal-dual is an f -approximation algorithm for Set Cover.

Greedy algorithm

Greedy algorithmAlgorithm:

1 while (∃ uncovered element)Choose S ′ minimizing w(S′)

|S′\⋃

S∈S′ S|Add S ′ to S ′.

2 Return S ′.

n := |E |, Hn :=∑n

i=11i

TheoremThe Greedy Algorithm is an Hn-approximation for Set Cover.

Si : set selected in iteration ini : uncovered elements at start of iteration i

Lemma

For every iteration i : w(Si) ≤ni − ni+1

niOPT .

Greedy algorithmAlgorithm:

1 while (∃ uncovered element)Choose S ′ minimizing w(S′)

|S′\⋃

S∈S′ S|Add S ′ to S ′.

2 Return S ′. n := |E |, Hn :=∑n

i=11i

TheoremThe Greedy Algorithm is an Hn-approximation for Set Cover.

Si : set selected in iteration ini : uncovered elements at start of iteration i

Lemma

For every iteration i : w(Si) ≤ni − ni+1

niOPT .

Greedy algorithmAlgorithm:

1 while (∃ uncovered element)Choose S ′ minimizing w(S′)

|S′\⋃

S∈S′ S|Add S ′ to S ′.

2 Return S ′. n := |E |, Hn :=∑n

i=11i

TheoremThe Greedy Algorithm is an Hn-approximation for Set Cover.

Si : set selected in iteration ini : uncovered elements at start of iteration i

Lemma

For every iteration i : w(Si) ≤ni − ni+1

niOPT .

Greedy algorithmAlgorithm:

1 while (∃ uncovered element)Choose S ′ minimizing w(S′)

|S′\⋃

S∈S′ S|Add S ′ to S ′.

2 Return S ′. n := |E |, Hn :=∑n

i=11i

TheoremThe Greedy Algorithm is an Hn-approximation for Set Cover.

Si : set selected in iteration ini : uncovered elements at start of iteration i

Lemma

For every iteration i : w(Si) ≤ni − ni+1

niOPT .

Greedy algorithmAlgorithm:

1 while (∃ uncovered element)Choose S ′ minimizing w(S′)

|S′\⋃

S∈S′ S|Add S ′ to S ′.

2 Return S ′.

Theorem∑S∈S′ w(S) ≤ Hg · Z ∗, where g := maxS∈S |S|.