View
420
Download
8
Category
Tags:
Preview:
DESCRIPTION
bridge question solved with 4 different algorithms, plus the complexity question with justification, Attention: This assignment got complete score.
Citation preview
ContentsProblem Introduction..................................................................................................................................2
Assumption..................................................................................................................................................2
Brute Force Algorithm.................................................................................................................................2
Pseudocode.............................................................................................................................................2
Brute Force Time Complexity..................................................................................................................3
Problem 1 (Brute force)...........................................................................................................................3
Problem 2 (Brute force)...........................................................................................................................5
Prim’s Algorithm..........................................................................................................................................7
Pseudocode.............................................................................................................................................7
Prim’s Time Complexity...........................................................................................................................7
Problem 1 (Prim’s)...................................................................................................................................8
Problem 2 (Prim’s).................................................................................................................................12
Kruskal’s Algorithm....................................................................................................................................14
Pseudocode...........................................................................................................................................14
Kruskal’s Time Complexity.....................................................................................................................14
Problem 1 (Kruskal’s).............................................................................................................................15
Problem 2 (Kruskal’s).............................................................................................................................18
Dijkstra Algorithm......................................................................................................................................20
Pseudocode...........................................................................................................................................20
Time complexity Dijkstra.......................................................................................................................21
Problem 1 (Dijkstra)...............................................................................................................................21
Problem 2 (Dijkstra)...............................................................................................................................22
Comparison of different algorithms for these problems...........................................................................23
Complexity Class........................................................................................................................................25
Complexity classes.................................................................................................................................25
Justification............................................................................................................................................27
Reference..................................................................................................................................................28
1
Problem Introduction
There are 8 islands which should be connected to each other with the minimum cost by 7
bridges. The cost of construction is related to the length of the bridges to be built. The problem
can be seen as a graph optimization problem and there are different solutions available for it.
Few algorithms will be applied to these problems and the efficiency of them will be compared
later.
Assumption
Since demonstrating the brute force solution is very difficult since the answer will be very long
and performing the algorithm completely will be very time consuming, Here only 2 examples of
routes that brute force will find is demonstrated.
Brute Force AlgorithmBrute force is a very simple but may not be very efficient solution for this type of problems. It is
not very efficient because in large number of nodes, brute force may take years to complete the
answer. But it can be used for small graphs.
Brute force lists all of the possible routes, evaluates their weight and then compares them to find
the smallest.
Pseudocode Generate first candidate C1 1
While candidate ≠ null C2 n!
If candidate is the solution C3 n!
Output the candidate C4 1
2
Generate next candidate C5 n!
Brute Force Time Complexity Running time for brute force is calculated as shows in below:
T(n) = C1 + n!(C2+C3) + C4 + n!(C5)
T(n) = C1 + C4 + n!(C2+C3+C5)
For example for the existing problem, the running time will be 8! = 40320.
Problem 1 (Brute force)Since the number of possible answers is very large, here 2 possible routes are drawn.
Route No 1:
3
1
2
3
4
5
6
7
8
240
265
260
160 360
175
305
Route No 2:
As it is shown, these are two random answers for the problem. When one connection is changed,
the total number of weight differs.
steps Total WeightRoute 1 1-2
2-33-44-55-66-77-8
1765
Route 2 1-22-33-44-55-66-77-8
1580
4
3
5
6
78
240
265
260
160360
175
120
Looking at the tables shows that in these 2 routes with only moving one connection, there is a
drastic change in total weight. So in order to find the best route all of the possible routes must be
compared.
Problem 2 (Brute force)For the second problem, 2 bridges are already built. So using brute force, we create random
graphs which include bridge between island 5&7 and 3&6.
First route:
5
120
160
260
265
240
8
6
5
3
1
400
350
And the second route:
steps Total Weight
Route 1 1-22-33-44-55-73-68-1
1795
Route 2 1-22-33-44-55-77-83-6
1980
6
160
260
265
240
8
6
5
3
1
400
350
305
Prim’s AlgorithmPrim’s is one of the greedy algorithms to find the minimum spanning tree in graphs. Prim’s
algorithm finds a subset of the graph which includes all of the nodes and ensures the total weigth
of edges is kept minimal.
Pseudocode(V = No of vertex)
Tree = NULL C1 1
E = sequence of all edges by weight C2 1
For i = 1 to V C3 V+1
For j = 1 to E C4 E+1
If ej NO cycle with edges & incident to a vertex C5 E-1
Add ej to tree C6 V-1
Break
Prim’s Time Complexity
T(V) = c1 + C2 + (V+1)C3 + V(E+1)C4 + V(E)(E-1)C5
T(v) = c1 + c2 + vC3 + C3 + VEC4 + VC4 + VE^2C5 – VEC5
The complexity for the algorithm is: O(|V||E|)
Minimum edge weight data structure Time complexity
Adjacency matrix, searching O(V*V)
Binary heap and adjacency list O((V+E)log(v)) = O(E+log(V))
Fibonacci heap and adjacency list O(E+Vlog(V))
7
1 8120
2
155
Problem 1 (Prim’s)Island 1 Island 2 Island 3 Island 4 Island 5 Island 6 Island 7 Island 8
Island 1 - 240 210 340 280 200 345 120
Island 2 - - 265 175 215 180 185 155
Island 3 - - - 260 115 350 435 195Island 4 - - - - 160 330 295 230
Island 5 - - - - - 360 400 170
Island 6 - - - - - - 175 205
Island 7 - - - - - - - 305
Island 8 - - - - - - - -
To find the minimum spanning tree using prim’s algorithm for this question, first we have to
choose a node to start. It is better to choose a node which is related to the smallest weight in
edges but since all the nodes will be incorporated in answer, really there is no different about
which node to start with. Thus, we will start by the island 1 as known as Node 1. Step 1
We need to find the smallest edge connected to node 1. From the first row we can see that the
edge between node 1 and 8 is the smallest. So:
Nodes Weight
1-8 120
Nodes Weight
1-8 120
Step 2
From node 8 or 1, we have to find an edge
with smallest weight:
Nodes Weight
8
1 8120
1-8 120
8-2 155
Step 3
Nodes Weight
1-8 120
8-2 155
8-5 170
Step 4
Nodes Weight
1-8 120
8-2 155
8-5 170
5-3 115
Step 5
9
1 8120
2
155
5170
1 8120
2
155
5170
3
115
1 8120
2
155
5170
3
115
4
160
6 180
1 8120
2
155
5170
3
115
4
160
Step 6
Nodes Weight
1-8 120
8-2 155
8-5 170
5-3 115
5-4 160
2-6 180
Step 7
10
Nodes Weight
1-8 120
8-2 155
8-5 170
5-3 115
5-4 160
1 8120
2
155
5170
3
115
4
160
6 180 7 175
Nodes Weight
1-8 120
8-2 155
8-5 170
5-3 115
5-4 160
2-6 180
6-7 175
Now, because all of the 8 islands are connected, there is no need for more edges and the solution
is complete. The total weight is: 1075
11
Problem 2 (Prim’s)
Step 1
Nodes Weight
5-7 400
3-6 350
5-3 115
Step 2
Nodes Weight
5-7 400
3-6 350
5-3 115
5-4 160
Step 3
Nodes Weight
5-7 400
3-6 350
5-3 115
5-4 160
5-8 170
Step 4
12
5 7
3 6
400
350
115
4 160
Nodes Weight
5-7 400
3-6 350
5-3 115
5-4 160
5-8 170
8-1 120
Step 5
Nodes Weight
5-7 400
3-6 350
5-3 115
5-4 160
5-8 170
8-1 120
8-2 155
Now all of the 8 islands are connected to each other by 7 bridges and no more bridge is needed.
Total weight is: 1470
13
5 7
3 6
400
350
115
4 160
8
170
1120
5 7
3 6
400
350
115
4 160
8
170
11202 155
Kruskal’s AlgorithmKruskal’s algorithm is another greedy algorithm based on generic minimum spanning tree
algorithm. It finds a safe edge to add the forest. At each step it adds the possible edge with least
weight.
Pseudocode
Define an empty tree T C1 1
For each vertex v C2 V+1
Define a cluster C(v) C3 V
Initialize a priority queue Q using weights as keys to conclude all edges C4 E
While edges in T<V-1 (V=No of vertices) C5 E
(u,v) <-Q.removeMin() C6 E logE
If C(v) ≠C(u) C7 E-1
Add edge (u,v) to T C8 V-1
Merge C(v) and C(u) into one cluster //union C9 V-1
Return T
Kruskal’s Time Complexity
T(v) = (C1+C10) + v(c3) + (v+1)(C2) + E(C4+c5) + (E log E)(C6) + (E-1)(C7) + (v-1)(C8+C9)
T(v) = (C1+C2+C10-C7-C8-C9) + v(C2+C3+C8+C9) + E(C4+C5+C7) + (E log E)(C6)
The complexity for the algorithm is O(|E|log|V|)
14
Problem 1 (Kruskal’s)Island 1 Island 2 Island 3 Island 4 Island 5 Island 6 Island 7 Island 8
Island 1 - 240 210 340 280 200 345 120
Island 2 - - 265 175 215 180 185 155
Island 3 - - - 260 115 350 435 195
Island 4 - - - - 160 330 295 230
Island 5 - - - - - 360 400 170
Island 6 - - - - - - 175 205
Island 7 - - - - - - - 305
Island 8 - - - - - - - -
Step 1
Nodes Weight
3-5 115
The edge between the nodes 3 and 5 hast the smallest weight.
Step 2
The next smallest weight in edges is:
Nodes Weight
3-5 115
8-1 120
15
1 8120 5
3
115
Step 3
Nodes Weight
3-5 115
8-1 120
8-2 155
Step 4
Nodes Weight
3-5 115
8-1 120
8-2 155
5-4 160
Step 5
Nodes Weight
3-5 115
8-1 120
8-2 155
5-4 160
8-5 170
16
1 8120
2
155
5
3
115
1 8120
2
155
5
3
115
4
160
1 8120
2
155
5170
3
115
4
160
Step 6
Nodes Weight
3-5 115
8-1 120
8-2 155
5-4 160
8-5 170
6-7 175
Step 7
Nodes Weight
3-5 115
8-1 120
8-2 155
5-4 160
8-5 170
6-7 175
6-2 180
We can see that all of the islands are connected to each other and no more edge is required, so
the solution is done.
The total weight is: 1075
The answer is similar to the answer when using prim’s algorithm which is obvious because we
are trying to find 1 thing using these 2 algorithms and the algorithms seek similar things and the
only difference between them is the steps.
17
1 8120
2
155
5170
3
115
4
160
67 175
1 8120
2
155
5170
3
115
4
160
6 180 7 175
Problem 2 (Kruskal’s)Step 1
Nodes Weight
5-7 400
3-6 350
3-5 115
Step 2
Nodes Weight
5-7 400
3-6 350
3-5 115
8-1 120
Step 3
Nodes Weight
5-7 400
3-6 350
3-5 115
8-1 120
8-2 155
18
5 7
3 6
400
350
115
5 7
3 6
400
350
115
8 1120
5 7
3 6
400
350
115
8 11202 155
Step 4
Nodes Weight
5-7 400
3-6 350
3-5 115
8-1 120
8-2 155
5-4 160
Step 5
Nodes Weight
5-7 400
3-6 350
3-5 115
8-1 120
8-2 155
5-4 160
5-8 170
Total weight is 1470.
19
5 7
3 6
400
350
115
4 160
8 11202 155
5 7
3 6
400
350
115
4 160
8
170
11202 155
Dijkstra Algorithm It solves the single source shortest path problem for graphs with weight and direction.
PseudocodeAssume Node A C1 1
For all nodes N adjacent to node A (m) C2 m
Store the edge between node N and node A C3 m
Link N to node A C4 m
For all nodes V C5 n
currDist(v) = ∞ C6 n
currDist(A) = 0 C7 1
ToBeChecked = all nodes C8 n
While ToBeChecked is not empty { C9 n+1
V = a node in ToBeChecked with minimal currDist(V) C10∑v=1
n
v
Remove V from ToBeChecked C11 n
For nodes U adjacent to V & in ToBeChecked{ C12∑v=1
n
t v
If currDist(U)>currDist(V) + weight(VU){ C13∑v=1
n
( tv−1 )
currDist(U)=currDist(V) + weight(UV) C14
predecessor(U) = V C15
}end if
}end for
20
}end while
Time complexity Dijkstra
T(n) = (C1 + C7) + m(C2 + C3 + C4) + n (C5 + C6 + C7 + C8 + C9 + C11 – C13) + ∑v=1
n
v (C10)
+ ∑v=1
n
t v(C12+C13) + x(C14+C15)
We know that V = a node in ToBeChecked with minimum currDist(V) will be executed
times. It will run n times at the beginning then repeats n-1 times and n-2 and..1 which is sum
from 1 to n meaning n^2. So its time complexity will be O(n^2).When there is one path between
each pair of nodes, tv will be equal to C so = ∑v=1
n
v and also time complexity = O(n^2). So
it can be said that when all nodes are connected to each other time complexity of dijkstra
algorithm will be O(n^2).
Problem 1 (Dijkstra)Dijsktra algorithm helps to find the shortest path in a graph. For this problem, I want to connect
island 1 to all of the other islands.
2 3 4 5 6 7 81 240 210 340 280 200 345 1208 155+120
240145+120
210230+120
340230+120
280205+120
200305+120
345-
6 180+200240
350+200210
330+200340
360+200280
- 175+200345
-
3 265+210240
- 260+210340
160+210280
- 435+210345
-
2 - - 175+240340
215+240280
- 185+240345
-
5 - - 160+280340
- - 400+280345
-
4 - - - - - 245+340345
-
21
n
v
v1
n
vvt
1
The table shows that the shortest path between island number 1 and all of the other islands is the
direct path between 1 and every other island.
1
8
7
6
5
4
3
2
120 240
210
340
280
200
345
Total weight for problem 1 using dijkstra algorithm is: 1735
Problem 2 (Dijkstra)Here we have to consider that 2 bridges are already built. Island 5 is selected for the first node.
1 2 3 4 6 7 85 280 215 115 160 360 400 1703 210+115
280265+115
215- 260+115
160350 - 195+115
1704 340+160
280175+160
215- - - - 230+160
1708 120+170
280155+170
215- - - - -
2 240+215280
- - - - - -
22
5
4
6
37
8 2
1
280
215
115
160
350
400
170
Total weight for problem 2 using dijkstra algorithm is:
Comparison of different algorithms for these problems
As said before, the main problem was to connect the 8 islands with 7 bridges and keep the cost as
low as possible. There were 4 different algorithms applied to solve this problem. But the
efficiency of the algorithms may differ according to the suitability of the algorithm for that
particular problem.
23
Here we can see the details of each algorithm:
Brute Force Prim’s Kruskal’s Dijkstra
Aim Any Problem Minimum
Spanning Tree
Minimum
Spanning Tree
Shortest Path
Running Time
Complexity
O(n!) O(|E||V|) O(ElogE) O(n^2)
Solution Yes Yes Yes Yes
Total
Weight
P 1 1765/1580 1075 1075 1735
P 2 1795/1980 1470 1470 1690
We know that between every node of this graph there is an edge. To find the total number of
edge we can use this formula: n(n-1)/2 so there are totally 28 edges in this graph and 8 nodes.
Running time for each algorithm will be:
Brute Force Prim’s Kruskal’s Dijkstra
Running Time
Complexity
O(n!) O(|E||V|) O(ElogE) O(n^2)
N= 8
= 40320
|E| = 28
|V| = 8
= 224
E = 28
Log28 =
1.4471580
= 40.520424
64
This table shows for this number of nodes and edges, dijkstra executes commands 64 times and it
is the most efficient algorithm.
24
Complexity Class
Each complexity class consists of problems of related resource based complexity. It usually uses
running time as complexity measure. Problems of same class can be solved by an abstract
machine M using O(f(n)) of resource R, where n is the number of input.
Notes:
Decision Problems are the kind of problems that the answer to them will be Yes or No or
0 or 1. The algorithm is applies on the input to find the answer.
Turing machine is a device that uses logics to find the answer. It executes a series of
discrete transitions as determined by its transition table and by the initial characters on
the tape.
Complexity classesP complexity class is class of decision problems that can be solved with a deterministic Turing
machine in polynomial time.
Class P contains problems that can be solved efficiently such as finding shortest paths in
networks. They can be solved by deterministic programs and solving their worst case takes
reasonable time. Class P consists of problems that can be solved in time O(nk) (n is the size of
input and k is constant). But p also includes problems that the best algorithms to solve them
have time complexity n10^500 although they are not computationally feasible.
NP complexity class is the class for decision problems that can be solved by non-deterministic
Turing machine in polynomial time.
This class is also defined by means other than nondeterministic Turing machines. NP is the class
of problems that the best solutions for them can be verified fast using deterministic machines in
polynomial time.
It should be mentioned that any problem in class P can be classified as NP also because when a
problem is P, it is possible to be solved without even needing verification.
25
Example: A decision problem where instances of the problem for which the answer is yes have
proofs that can be verified in polynomial time. This means that if someone gives us an instance
of the problem and a certificate (sometimes called a witness) to the answer being yes, we can
check that it is correct in polynomial time.
NP hard complexity class includes the NP problems that can be efficiently reduced to it. It
means if there is a polynomial time algorithm that translates any instance of NP problem into an
instance of the problem in question. When a decision version of a combinational optimization
problem is proved to belong to NP-complete problems, the optimization problem is NP hard.
Example: this problem: “is there a Hamiltonian cycle with length less than K” belong to NP
complete class. But this question “what is the shortest tour?” is NP hard since there is no easy
way to determine if a certificate is the shortest.
NP complete complexity class is the class for which answers can be checked for correctness
given a certificate by an algorithm that its run time is polynomial in size of the input. The
answers of a NP complete problem can be verified quickly and a quick algorithm to solve this
problem can be used to all other NP problems quickly.
Example: This is the problem wherein we are given a conjunction of 3-clause disjunctions (i.e.,
statements of the form
(x_v11 or x_v21 or x_v31) and
(x_v12 or x_v22 or x_v32) and
... and
(x_v1n or x_v2n or x_v3n)
where each x_vij is a boolean variable or the negation of a variable from a finite predefined
list(x_1, x_2, ... x_n). It can be shown that every NP problem can be reduced to 3-
SAT. The proof of this is technical and requires use of the technical definition of NP (based on
non-deterministic Turing machines and the like). This is known as Cook's theorem.
26
What makes NP-complete problems important is that if a deterministic polynomial time
algorithm can be found to solve one of them, every NP problem is solvable in polynomial time
(one problem to rule them all).
JustificationIn the definition of P class it was told that P contains familiar problems that can be solved
efficiently. Finding the shortest path in networks and graphs is also one of the P problems. The
problem in this assignment also can be solved with finding the shortest path and there is at least
one algorithm that can solve the problem in polynomial time. This problem in 8 nodes (limited
size of input) can be solved in polynomial time but in larger sizes of input cannot be solved in
polynomial time. Also, since it is possible to say that because the problem is P, it also can be NP
because there is no evidence that P problems do not belong to NP class. Greedy algorithms such
as Prim’s and Kruskal’s are P but Brute force is not greedy and in larger instances it can be NP.
27
Reference
Cormen Thomas H., Introduction to Algorithms, 2nd edn, McGraw-Hill, US
Eric Allender , Complexity Classes, (Online) Reached from:
http://ftp.cs.rutgers.edu/pub/allender/ALRch33.pdf [Visited on 14th 5 2012]
Unknown, NP vs NP-complete vs NP HARD, (Online) Reached from:
http://stackoverflow.com/questions/1857244/np-vs-np-complete-vs-np-hard-what-does-
it-all-mean [Visited on 19th 5 2012]
Scott Aaronson, Computational Complexity and the Anthropic Principle, 2006, (Online)
Reached from: http://www.scottaaronson.com/talks/anthropic.html [Visited on 20th 5
2012]
28
Recommended