Introduction to Algorithms - University of Florida · Introduction to Algorithms ... Algorithms ....

Introduction to Algorithms

NP-Completeness and Approximation

Algorithms

My T. Thai

mythai@cise.ufl.edu

2

Why Approximation Algorithms

Problems that we cannot find an optimal

solution in a polynomial time

Eg: Set Cover, Bin Packing

Need to find a near-optimal solution:

Heuristic

Approximation algorithms:

This gives us a guarantee approximation ratio

My T. Thai

mythai@cise.ufl.edu

3

Why important

Your advisers/bosses give you a

computationally hard problem. Here are two

scenarios:

No knowledge about approximation:

Spend a few months looking for an optimal solution

Come to their office and confess that you cannot do it

Get fired

Knowledge about approximation:

My T. Thai

mythai@cise.ufl.edu

4

Knowledge about approximation

Show your boss that this is a NP-complete (NP-

hard) problem

There does not exist any polynomial time algorithm

to find an exact solution

Propose a good algorithm (either heuristic or

approximation) to find a near-optimal solution

Better yet, prove the approximation ratio

My T. Thai

mythai@cise.ufl.edu

5

Techniques

A variety of techniques to design and analyze

many approximation algorithms for

computationally hard problems:

Combinatorial algorithms:

Greedy Techniques, Independent System, Submodular

Function

Linear Programming based algorithms

Semidefinite Programming based algorithms

In this class, we just introduce a few examples

My T. Thai

mythai@cise.ufl.edu

6

Combinatorial Optimization

The study of finding the “best” object from

within some finite space of objects, eg:

Shortest path: Given a graph with edge costs and a

pair of nodes, find the shortest path (least costs)

between them

Traveling salesman: Given a complete graph with

nonnegative edge costs, find a minimum cost cycle

visiting every vertex exactly once

Maximum Network Lifetime: Given a wireless

sensor networks and a set of targets, find a schedule

of these sensors to maximize network lifetime

My T. Thai

mythai@cise.ufl.edu

7

In P or not in P?

Informal Definitions:

The class P consists of those problems that are

solvable in polynomial time, i.e. O(nk) for some

constant k where n is the size of the input.

The class NP consists of those problems that

are “verifiable” in polynomial time:

Given a certificate of a solution, then we can verify

that the certificate is correct in polynomial time

My T. Thai

mythai@cise.ufl.edu

8

In P or not in P: Examples

In P:

Shortest path

Minimum Spanning Tree

Not in P (NP):

Vertex Cover

Traveling salesman

Minimum Connected Dominating Set

My T. Thai

mythai@cise.ufl.edu

9

NP-completeness (NPC)

A problem is in the class NPC if it is in NP and

is as “hard” as any problem in NP

My T. Thai

mythai@cise.ufl.edu

10

What is “hard”

Decision Problems: Only consider YES-NO

Decision version of TSP: Given n cities, is there a

TSP tour of length at most L?

Why Decision Problem? What relation between

the optimization problem and its decision?

Decision is not “harder” than that of its

optimization problem

If the decision is “hard”, then the optimization

problem should be “hard”

My T. Thai

mythai@cise.ufl.edu

11

NP-complete and NP-hard

A language L is NP-complete if:

1. L is in NP, and

2. For every L’ in NP, L’ can be polynomial-time

reducible to L

My T. Thai

mythai@cise.ufl.edu

12

Approximation Algorithms

An algorithm that returns near-optimal

solutions in polynomial time

Approximation Ratio ρ(n):

Define: C* as a optimal solution and C is the

solution produced by the approximation algorithm

max (C/C*, C*/C) <= ρ(n)

Maximization problem: 0 < C <= C*, thus C*/C

shows that C* is larger than C by ρ(n)

Minimization problem: 0 < C* <= C, thus C/C*

shows that C is larger than C* by ρ(n)

My T. Thai

mythai@cise.ufl.edu

13

Approximation Algorithms (cont)

PTAS (Polynomial Time Approximation

Scheme): A (1 + ε)-approximation algorithm

for a NP-hard optimization П where its running

time is bounded by a polynomial in the size of

instance I

FPTAS (Fully PTAS): The same as above +

time is bounded by a polynomial in both the

size of instance I and 1/ε

My T. Thai

mythai@cise.ufl.edu

14

A Dilemma!

We cannot find C*, how can we compare C to

C*?

How can we design an algorithm so that we can

compare C to C*

It is the objective of this course!!!

My T. Thai

mythai@cise.ufl.edu

15

Vertex Cover

Definition:

An Example

'at incident endpoint one

leastat has , edgeevery for iff ofcover vertex a

called is 'subset a ,),(graph undirectedan Given

V

eEeG

VVEVG

My T. Thai

mythai@cise.ufl.edu

16

Vertex Cover Problem

Definition:

Given an undirected graph G=(V,E), find a vertex

cover with minimum size (has the least number of

vertices)

This is sometimes called cardinality vertex cover

More generalization:

Given an undirected graph G=(V,E), and a cost

function on vertices c: V → Q+

Find a minimum cost vertex cover

My T. Thai

mythai@cise.ufl.edu

17

How to solve it

Matching:

A set M of edges in a graph G is called a matching

of G if no two edges in set M have an endpoint in

common

Example:

My T. Thai

mythai@cise.ufl.edu

18

How to solve it (cont)

Maximum Matching:

A matching of G with the greatest number of edges

Maximal Matching:

A matching which is not contained in any larger

matching

Note: Any maximum matching is certainly

maximal, but not the reverse

My T. Thai

mythai@cise.ufl.edu

19

Main Observation

No vertex can cover two edges of a matching

The size of every vertex cover is at least the

size of every matching: |M| ≤ |C|

|M| = |C| indicates the optimality

Possible Solution: Using Maximal matching to

find Minimum vertex cover

My T. Thai

mythai@cise.ufl.edu

20

An Algorithm

Alg 1: Find a maximal matching in G and output the

set of matched vertices

Theorem: The algorithm 1 is a factor 2-approximation

algorithm.

Proof:

optC

MCoptM

Copt

2|| Therefore,

||2|| and ||

:have We1. algorithm from obtainedcover vertex

ofset a be Let solution. optimalan of size a be Let

My T. Thai

mythai@cise.ufl.edu

21

Can Alg1 be improved?

Q1: Can the approximation ratio of Alg 1 be

improved by a better analysis?

Q2: Can we design a better approximation

algorithm using the similar technique (maximal

matching)?

Q3: Is there any other lower bounding method

that can lead to a better approximation

algorithm?

My T. Thai

mythai@cise.ufl.edu

22

Answers

A1: No by considering the complete bipartite

graphs Kn,n

A2: No by considering the complete graph Kn

where n is odd.

|M| = (n-1)/2 whereas opt = n -1

My T. Thai

mythai@cise.ufl.edu

23

Answers (cont)

A3:

Currently a central open problem

Yes, we can obtain a better one by using the

semidefinite programming

Generalization vertex Cover

Can we still able to design a 2-approximation

algorithm?

Homework Excersice

My T. Thai

mythai@cise.ufl.edu

24

Set Cover

Definition: Given a universe U of n elements, a collection of subsets of U, S = {S1, …, Sm}, and a cost function c: S → Q+, find a minimum cost subcollection C of S that covers all elements of U.

Example:

U = {1, 2, 3, 4, 5}

S1 = {1, 2, 3}, S2 = {2,3}, S3 = {4, 5}, S4 = {1, 2, 4}

c1 = c2 = c3 = c4 = 1

Solution C = {S1, S3}

If the cost is uniform, then the set cover problem asks us to find a subcollection covering all elements of U with the minimum size.

My T. Thai

mythai@cise.ufl.edu

25

An Example

My T. Thai

mythai@cise.ufl.edu

26

INSTANCE: Given a universe U of n elements, a collection of subsets of U, S = {S1, …, Sm}, and a positive integer b

QUESTION: Is there a , |C| ≤ b, such that

(Note: The subcollection {Si | } satisfying the above condition is called a set cover of U

NP-completeness

Theorem: Set Cover (SC) is NP-complete

Proof:

My T. Thai

mythai@cise.ufl.edu

27

Proof (cont)

First we need to show that SC is in NP. Given a collection of sets C, it is easy to verify that if |C| ≤ b and the union of all sets listed in C does include all elements in U.

To complete the proof, we need to show that Vertex Cover (VC) ≤p Set Cover (SC)

Given an instance C of VC (an undirected graph G=(V,E) and a positive integer j), we need to construct an instance C’ of SC in polynomial time such that C is satisfiable iff C’ is satisfiable.

My T. Thai

mythai@cise.ufl.edu

28

Proof (cont)

Construction: Let U = E. We will define n elements of U and a collection S as follows:

Note: Each edge corresponds to each element in U and each vertex corresponds to each set in S.

Label all vertices in V from 1 to n. Let Si be the set of all edges that incident to vertex i. Finally, let b = j. This construction is in poly-time with respect to the size of VC instance.

My T. Thai

mythai@cise.ufl.edu

29

VERTEX-COVER p SET-COVER

one set for every vertex, containing the edges it covers

VC

one element for every edge

VC SC

My T. Thai

mythai@cise.ufl.edu

30

Proof (cont)

Now, we need to prove that C is satisfiable iff C’ is.

That is, we need to show that if the original instance

of VC is a yes instance iff the constructed instance of

SC is a yes instance.

(→) Suppose G has a vertex cover of size at most j, called

C. By our construction, C corresponds to a collection C’ of

subsets of U. Since b = j, |C’| ≤ b. Plus, C’ covers all

elements in U since C “covers” all edges in G. To see this,

consider any element of U. Such an element is an edge in

G. Since C is a vertex cover, at least endpoint of this edge is

in C.

My T. Thai

mythai@cise.ufl.edu

31

(←) Suppose there is a set cover C’ of size at most b in our constructed instance. Since each set in C’ is associated with a vertex in G, let C be the set of these vertices. Then |C| = |C’| ≤ b = j. Plus, C is a vertex cover of G since C’ is a set cover. To see this, consider any edge e. Since e is in U, C’ must contain at least one set that includes e. By our construction, the only set that include e correspond to nodes that are endpoints of e. Thus C must contain at least one endpoints of e.

My T. Thai

mythai@cise.ufl.edu

32

Solutions

Algorithm 1: (in the case of uniform cost)

1: C = empty

2: while U is not empty

3: pick a set Si such that Si covers the most

elements in U

4: remove the new covered elements from U

5: C = C union Si

6: return C

My T. Thai

mythai@cise.ufl.edu

33

Solutions (cont)

In the case of non-uniform cost

Similar method. At each iteration, instead of picking a

set Si such that Si covers the most uncovered elements,

we will pick a set Si whose cost-effectiveness α is

smallest, where α is defined as:

Questions: Why choose smallest α? Why define α as

above

||/)( USSc ii

My T. Thai

mythai@cise.ufl.edu

34

Solutions (cont)

Algorithm 2: (in the case of non-uniform cost)

1: C = empty

2: while U is not empty

3: pick a set Si such that Si has the smallest α

4: for each new covered elements e in U

5: set price(e) = α

6: remove the new covered elements from U

7: C = C union Si

8: return C

My T. Thai

mythai@cise.ufl.edu

35

Approximation Ratio Analysis

Let ek, k = 1…n, be the elements of U in the order in which

they were covered by Alg 2. We have:

Lemma 1:

Proof: Let Uj be the set the remaining set U at iteration j.

That is, Uj contains all the uncovered elements at

iteration j. At any iteration j, the leftover sets of the

optimal solution can cover the remaining elements at a

cost of at most opt. (Why?)

solution optimal theofcost theis where

)1/()(price },,...,1{each For

opt

knoptenk k

My T. Thai

mythai@cise.ufl.edu

36

Proof of Lemma 1 (cont)

Thus, among these leftover sets, there must exist

one set where its α value is at most opt/|Uj|. Let

ek be the element covered at this iteration, |Uj| ≥

n – k + 1. Since ek was covered by the most

cost-effective set in this iteration, we have:

price(ek) ≤ opt/|Uj| ≤ opt/(n-k+1)

My T. Thai

mythai@cise.ufl.edu

37

Approximation Ratio

Theorem 1: The set cover obtained form Alg 2

(also Alg 1) has a factor of Hn where Hn is a

harmonic series Hn = 1 + 1/2 + … + 1/n

Proof: It follows directly from Lemma 1

Examples of NP-complete problems

Independent set

- independent set: a set of vertices in the graph with no

edges between each pair of nodes.

- given a graph G=(V,E), find the largest independent set

- reduction from vertex cover:

smallest vertex cover S largest independent set V/S

Independent Set

Independent set

- if G has a vertex cover S, then V \ S is an independent set

proof: consider two nodes in V \ S:

if there is an edge connecting them, then one of them must be in

S, which means one of them is not in V \ S

- if G has an independent set I, then V \ I is a vertex cover

proof: consider one edge in G:

if it is not covered by any node in V \ I, then its two end vertices

must be both in I, which means I is not an independent set

Steiner Tree

Steiner tree

- given a graph G=(V,E), and a subset C of V

- find the minimum tree to connect each vertex in C

- reduction

Vertex cover for Graph G Steiner tree for graph G’

vertex

(u,v)

edge

(u,v)-u

edge

(u,v)-v

G’ (only edges with distance 1 are shown)

G

Steiner Tree

Construction

- G’ is a complete graph

- for every node u in G, create a node u in G’

- for every edge (u,v) in G, create a node (u,v) in G’

- in G’, every node (u,v) is connected to u and v with distance 1

- in G’, every node u and v is connected with distance 1

- other edges in G’ are of distance 2 vertex

(u,v)

edge

(u,v)-u

edge

(u,v)-v

G’ (only edges with distance 1 are shown)

G

Sketch of Proof:

- in the Steiner tree problem for G’, choose C to be the set of all

nodes (u,v)

- G’ has a minimum Steiner tree of cost m+k-1 iff G has a minimum

vertex cover of size k

vertex

(u,v)

edge

(u,v)-u

edge

(u,v)-v

G’ (only edges with distance 1 are shown)

G

Summary of some NPc problems

SAT

3SAT

Vertex cover

Independent set

Set cover

Graph coloring

Maximum clique size

Minimum Steiner tree

Hamiltonian cycle

Maximum cut

find more NP-complete problems in http://en.wikipedia.org/wiki/List_of_NP-complete_problems

Approximation Algorithms

Bin Packing

My T. Thai

mythai@cise.ufl.edu

45

Definition

My T. Thai

mythai@cise.ufl.edu

46

Theorem

My T. Thai

mythai@cise.ufl.edu

47

An Easy Solution: First Fit

My T. Thai

mythai@cise.ufl.edu

48

Asymptotic PTAS

My T. Thai

mythai@cise.ufl.edu

49

My T. Thai

mythai@cise.ufl.edu

50

Approximation Algorithms

k-Center

My T. Thai

mythai@cise.ufl.edu

52

Definition

My T. Thai

mythai@cise.ufl.edu

53

2-Approx

My T. Thai

mythai@cise.ufl.edu

54

My T. Thai

mythai@cise.ufl.edu

55

My T. Thai

mythai@cise.ufl.edu

56

Analysis

My T. Thai

mythai@cise.ufl.edu

57

Obtain Better Ratio?

My T. Thai

mythai@cise.ufl.edu

58

Weighted k-Center

My T. Thai

mythai@cise.ufl.edu

59

Notations

My T. Thai

mythai@cise.ufl.edu

60

Algorithm

My T. Thai

mythai@cise.ufl.edu

61

Algorithm 2

My T. Thai

mythai@cise.ufl.edu

62

Analysis

Theorem: Alg 2 has an approx ratio of 3.

Approximation Algorithms

Multiway Cut and k-Cut

My T. Thai

mythai@cise.ufl.edu

63

Definition

My T. Thai

mythai@cise.ufl.edu

64

Example

My T. Thai

mythai@cise.ufl.edu

65

My T. Thai

mythai@cise.ufl.edu

66

Can you prove it???

Approximation

My T. Thai

mythai@cise.ufl.edu

67

Analysis

My T. Thai

mythai@cise.ufl.edu

68

Minimum k-cut Problem

My T. Thai

mythai@cise.ufl.edu

69

Example

My T. Thai

mythai@cise.ufl.edu

70

Construct a Gomory-Hu Tree

My T. Thai

mythai@cise.ufl.edu

71

My T. Thai

mythai@cise.ufl.edu

72

Properties of Gomory-Hu Tree wrt

k-cut

My T. Thai

mythai@cise.ufl.edu

73

Approximation Algorithm

My T. Thai

mythai@cise.ufl.edu

74

Analysis

My T. Thai

mythai@cise.ufl.edu

75