NP Algoritm

7/30/2019 NP Algoritm

1/23

Chapters 11-12P, NP, NP-Complete Problems

and

Approximation Algorithms

Dr. Natarajan MeghanathanAssociate Professor of Computer Science

Jackson State UniversityJackson, MS 39217

E-mail: [email protected]


2/23

P, NP-Problems An algorithm is said to solve a problem in polynomial time if its worst-case time complexity is O(p(n)) where p(n) is a polynomial (includinglogarithmic time) of the problems input size n.

Problems that can be solved in polynomial time are called tractable,and those that cannot be solved in polynomial time are calledintractable.

A problem is a decision problem if it asks for an yes or no answer.

For example, the problem of determining whether a directedgraph is a DAG or not is a decision problem.

A problem is an optimization problem if it aims to find an optimal(minimum or maximum, depending on the context) solution.

For example, the minimum spanning tree problem, shortest pathproblem are all optimization problems.


3/23

P, NP-Problems A decision problem can often be transformed to an optimizationproblem and vice-versa.

For example, the graph coloring problem that asks what is theminimum number of colors needed for a graph such that no twoadjacent vertices have the same color (an optimization problem).This can be transformed to a decision problem by asking whetherthe graph is 2-colorable, 3-colorable, etc.

Note that there are many decision or optimization problems for which

no polynomial-time algorithm has been found; but, neither theimpossibility of such an algorithm been proved.

Example: Traveling Salesman problem, Hamiltonian Circuit problem

Hamiltonian Circuit Problem: Determine whether a given graph has a

Hamiltonian Circuit a path that starts and ends at the same vertexand passes through all the other vertices exactly once.

Traveling Salesman Problem: Find the shortest tour through n citieswith known positive integer distances between them (find the

shortest Hamiltonian Circuit in a complete graph with positive integerweights).


4/23

P, NP-Problems Class P (Polynomial) is a class of decision or optimization problemsthat can be solved in polynomial time by deterministic algorithms.

A deterministic algorithm is the one that produces the same outputfor a problem instance each time the algorithm is run.

A non-deterministic algorithm is a two-stage procedure that takes asits input an instance Iof a decision problem and does the following:

Stage 1: Non-deterministic (Guessing) Stage: An arbitrary string S isgenerated that can be thought of as a candidate solution to the given

instance I (could be sometimes, not a correct solution too). Stage 2: Deterministic (Verification) Stage: A deterministic algorithm

takes both I and S as its input and outputs yes if S represents a solutionto instance I. Note that we do not want a non-deterministic algorithm tooutput a yes answer on an instance for which the answer should be no.

A non-deterministic algorithm for the Traveling Salesman problemcould be the one that inputs a complete weighted graph of n vertices,then (stage 1): generates an arbitrary sequence of n vertices and(stage 2): verifies whether or not that each vertex, except the starting

vertex, is repeated only once in the sequence, and outputs a yes/noanswer accordingly.


5/23

P, NP-Problems Class NP (Non-deterministic Polynomial) is the class of decisionproblems that can be solved by non-deterministic polynomialalgorithms.

Note that all problems in Class P are in Class NP: We can replacethe non-deterministic guessing of Stage 1 with the deterministicalgorithm for the decision problem, and then in Stage 2,deterministically verify the correctness of the solution of thedeterministic algorithm. In other words, P NP. However, it is notclear whether P = NP or P NP.

Polynomial reduction: A problem Iis said to be reducible to another

problem Jin polynomial time (denoted IP J), if every instance of Ican be reduced to an instance of Jin polynomial time. So, if apolynomial time algorithm exists to solve an instance of problem J,then we can say that there exists a polynomial time algorithm for I.


6/23

NP-Complete Problems A problem Iis said to be NP-complete, if: Iis in NP

Every problem in class NP is polynomial-time reducible to I.

There are several decision problems that have been proven to beNP-complete.

If we could find a polynomial-time deterministic algorithm to solveany single NP-complete problem, then all problems in NP can be

considered to also be solvable deterministically in polynomial-time. Inthat case P = NP.

After proving that the problem they want to solve is an NP-completeproblem, researchers do not break their heads to find a polynomial-timedeterministic algorithm if they find one, they would surely win the

Turing award though!!! Instead, the focus will be on developing an approximate algorithm (called

a heuristic) that can yield a solution that will be bounded below or above

(depending on the context) by the optimal solution within a certain value,

that is as minimal as possible. That is, aim for an approximation ratio thatis closer to 1; but not equal to 1.


7/23

NP-Complete Problems How would you prove that a problem H is NP-complete?

Let L be a problem that has been already provento be NP-complete.

We will show the following: (1) H is polynomial time reducible to L: HP L

(2) H is NP-hard: A problem H in NP is said to be

NP-hard, if a NP-complete problem is polynomialtime reducible to H. In this case, we show that LP H.

All NP-Problems

except H

P

L H

Proof of NP-hard: P

P


8/23

Traveling Salesman Problem Given a complete weighted graph (i.e., weighted edges representingdistances in the Euclidean plane between any two vertices), find aminimum weight tour that starts from a particular vertex, goesthrough all the other vertices exactly once, and ends at the startingvertex. Possible Combinations and their Tour

Weights:

a b c d a 10

a b d c a 8 a c b d a 14

a c d b a 8

a d c b a 10

a d b c a 14 Similarly 18 more tours, can be written (6

each starting from b, c, or d).

The minimal tour weight is 8 and it could be

one of the tours of the correspondingweight , as listed above.

Traveling SalesmanProblem (TSP)


9/23

Traveling Salesman Problem Both the Hamiltonian Circuit problem and the Traveling Salesmanproblems are NP-Complete problems.

Polynomial Reduction: We will see the polynomial-time reduction of a Hamiltonian Circuit

problem to a Traveling Salesman problem.

Recollect the Hamiltonian Circuit problem is to find a tour of a given

unweighted graph that simply starts at one vertex and goes throughall the other vertices and ends at the starting vertex.

Note that the input graph G to a Hamiltonian Circuit problem neednot be a complete graph connecting all vertices.

For the reduction, we will construct a complete graph G* such thatthere exists an edge of weight 1 in G* if the edge exists in G;otherwise, a weight of 2 is assigned to an edge between two verticesin G* if the edge does not exist in G.


10/23

Traveling Salesman Problem This way, if we solve the complete graph G* for the TravelingSalesman problem, if there exists a Hamiltonian Circuit in the originalgraph G, the minimum weight tour in G* will involve only edges ofweight 1. All such edges in the minimal weight tour of G* will exist inan Hamiltonian Circuit of G.

The polynomial time reduction is involved in:

Given an weighted graph G for the Hamiltonian Circuit problem, theconstruction of the complete graph G* for the Traveling Salesman

problem:

For every edge (u, v) in G, include (u, v) in G* and weight (u, v) = 1

For every edge (u, v) not in G, include (u, v) in G* and weight(u, v) = 2


11/23

Heuristic 1: Nearest Neighbor (NN)

Heuristic for the TSP Problem Start the tour with a particular

vertex, and include it to the tour.

For every iteration, a vertex (fromthe set of vertices that are not yetpart of the tour) that is closest to thelast added vertex to the tour is

selected, and included to the tour. The above procedure is repeated

until all vertices are part of the tour.

Time Complexity: It takes O(V)

times to choose a vertex among thecandidate vertices for inclusion asthe next vertex on the tour. Thisprocedure is repeated for V-1 times.Hence, the time complexity of theheuristic is O(V2).


12/23

NN Heuristic Example (contd..)


13/23

NN Heuristic Example (contd..)v1 v3 v2

v6v4v5

3 5

2

59

6

Improvement to the NN Heuristic using 2-Change HeuristicPick two non-overlapping edges (with no common end vertices) and see if we can

swap for them using edges that connect the end vertices so that the connectivity of thetour is maintained and the tour cost can be further reduced.

Strategy: Pick the costliest edge and a non-overlapping edge (i.e., no common endvertices) that is the next costliestIn the above example, we can pick v5 v4 (edge wt: 9) and the next costliest non-

overlapping edge v3 v2 (edge wt: 5) and replace them with edges v5 v2 (wt: 3) andv4 v3 (wt: 5). The revised tour is v1 v3 v4 v6 v2 v5 v1; tour weight: 24

v1 v3 v2

v6v4v5

3

2

5

6 35


14/23

Heuristic # 2: Multi-fragment

Heuristic for the TSP Problem Step 1: Sort the edges in increasing

order of their weights (break the ties

arbitrarily). Initialize the set of touredges to be constructed to theempty set.

Step 2: Repeat this step |V| times,where |V| is the number of vertices

in the instance of the TSP problembeing solved:

Add the next edge on the sortededge list to the set of tour edges,

provided this addition does notcreate a vertex of degree 3 or acycle of length less than |V|;otherwise, skip the edge

Step 3: Return the set of tour edges

Complexity: O(E*logE) = O(V2*logV)


15/23


Heuristic for the TSP Problem

edge v2 v6 (wt: 2) OKedge v1 v3 (wt: 3) OK

edge v2 v5 (wt: 3) OKedge v1 v6 (wt: 4) OKedge v2 v4 (wt: 4) NOedge v1 v2 (wt: 5) NOedge v2 v3 (wt: 5) NO

edge v3 v4 (wt: 5) OKedge v4 v6 (wt: 5) NO

v2 v6 v4

v3v1v5edge v1 v5 (wt: 6) NOedge v3 v5 (wt: 6) NOedge v5 v6 (wt: 6) NOedge v3 v6 (wt: 7) NO

edge v1 v4 (wt: 8) NOedge v4 v5 (wt: 9) OK

Tour: v1 v3 v4 v5 v2 v6 v1

Tour Weight: 26


16/23


Heuristic for the TSP Problemv1 v3 v4

v5v2v6

Tour: v1 v3 v4 v6 v2 v5 v1 Tour Weight: 24

3 5

9

32

4

v1 v3 v4

v5v2v6

3 5

32

Tour from the Multi-fragment

Heuristic (prev. slide): wt 26

56

Improvement using 2-Change

v1 v3 v4

v6v2v5

3 5

5

32

6


17/23

Heuristic # 3 for the TSP Problem

Twice-around-the Tree Algorithm Step 1: Construct a Minimum Spanning

Tree of the graph corresponding to agiven instance of the TSP problem

Step 2: Starting at an arbitrary vertex,perform a walk around the minimumspanning tree recording all the verticespassed by. This can be done by a DFS

traversal.

Step 3: Scan the vertex list obtained in

Step 2 and eliminate from it all repeatedoccurrences of the same vertex exceptthe starting one at the end of the list.

The vertices remaining on the list willform a Hamiltonian Circuit, which is theoutput of the algorithm.

v1 v2 v3

v4v5v6

Step 1: MST of the Graph

Note: We will use the principle of Triangle

Inequality for Euclidean plane:

The sum of the two sides of a triangle isgreater than the third side of the triangle


18/23


Twice-around-the Tree Algorithm

v1 v2v3

v4v5v6

Step 2: DFS Traversal of the MST

MST (vertices rearranged) from Step 1

v1 v3 v1 v6 v2 v4 v2 v5 v2 v6 v1

v1 v2v3

v4v5v6

Step 3: Optimizing the DFS Walk

Tour from Step 2:

v1 v3 v1 v6 v2 v4 v2 v5 v2 v6 v1

Optimized Tour:v1 v3 v1 v6 v2 v4 v2 v5 v2 v6 v1

v1 v3 v6 v2 v4 v5 v1Tour Weight: 31

v1 v2v3

v4v5v6


19/23


Twice-around-the Tree AlgorithmTSP Tour of Twice-around-the-Tree Algorithm

v1 v3 v6 v2 v4 v5 v1Tour Weight: 31

v1 v3 v6

v2v4v5

3 7

2

49

6

Improved Tour Weight: 26 Improvement using 2-Change

v1 v3 v6

v2v4v5

3

5 2

46

6

v1 v3 v4

v2v6v5

3 5

4

26

6


20/23

Proof for the ApproximationRatio for Twice-around-the-Tree Let w(MST) be the weight of the MST generated from Step 1.

The weight of the DFS walk generated from Step 2 could be at most2*w(MST), as seen in the example.

In Step 3, we are trying to optimize the DFS walk and extract aHamiltonian Circuit of lower weight. Even if no optimization ispossible, the weight of the tour generated by the Twice-around-the-

Tree algorithm is at most twice the weight of the minimum spanningtree of the graph instance of the TSP problem.

Note that w(MST) of the graph has to be less than the weight of anoptimal tour, w(Optimal Tour). Otherwise, if w(Optimal Tour)

w(MST), then the so-called MST with V-1 edges is not a MST. W(Twice-around-the-Tree tour) 2*W(MST) < 2*w(Optimal Tour).

W(Twice-around-the-Tree tour) / W(Optimal Tour) < 2.

Hence, the approximation ratio of the Twice-around-the-Tree

algorithm is less than 2.


21/23


Christofides Algorithm Step 1: Construct a Minimum Spanning

Tree (MST) of the graph correspondingto a given instance of the TSP problem

Step 2: Find the set of vertices thathave an odd degree in the MST of Step1. There will always be an even numberof vertices with an odd degree in a

connected graph. Form a complete subgraph of only the vertices with odddegree in the MST.

Step 3: Run the Edmonds-Karpalgorithm (a slight variation of the Ford-

Fulkerson algorithm with the augmentingpaths determined in increasing order ofthe hop count using BFS) on the

complete graph and determine the set ofedges that form a minimum-weight pair

matching (MWPM).

Step 4: Superimpose the edges formingthe MWPM of Step 3 with the edgesforming the MST of Step 1. On the

MST-MWPM graph, do a DFS traversaland construct a walk as in Step 2 of theTwice-around-the-Tree algorithm.

Step 5: Scan the vertex list obtained in

Step 4 and eliminate from it all repeatedoccurrences of the same vertex exceptthe starting one at the end of the list.The vertices remaining on the list will

form a Hamiltonian Circuit, which is theoutput of the algorithm.

Note that instead of using Edmonds-Karp algorithm, we will use a greedystrategy of picking the edges

in the increasing order of their edge

weights


22/23


Christofides Algorithm

v1 v2 v3

v4v5v6

Step 1: MST of the Graph

Step 2: Complete Graph of theVertices with Odd Degree in the MST

v2, v3, v4 and v5 are the four vertices

with odd degree in the MST of Step 1.We now form a complete graph involving

these vertices and their edges

v2 v3

v4v5

3

5

5

9

46

Step 3: Min. Wt. Pair Matching (MWPM)

v2 v3

v4v5

3 5


23/23


Christofides Algorithm

v1 v2 v3

v4v5v6

Step 4: Superimposition ofthe MST and MWPM

Comparison of Tour Weights ofHeuristics before and after theApplication of 2-Change

Before AfterNearest Neighbor 30 24Multi-fragment 26 24

Twice-around-the-Tree 31 26

Christofides algorithm 25 25

Step 5: Optimizing the DFS Walk

List of vertices on a DFS Traversal

v1 v3 v4 v2 v5 v2 v6 v1

List of vertices on a Hamiltonian Circuit

v1 v3 v4 v2 v5 v6 v1

v1 v3 v4

v2v5v6

3 5

4

36

4

Tour Weight: 25No further improvement with 2-change

is possible.

Documents

NP Algoritm