Algorithm Design and Analysis

9/10/10A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

Adam Smith

Algorithm Design and Analysis

LECTURE 7Greedy Graph Algorithms• Topological sort• Shortest paths

A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

The (Algorithm) Design Process

1. Work out the answer for some examples

2. Look for a general principle– Does it work on *all* your examples?

3. Write pseudocode

4. Test your algorithm by hand or computer– Does it work on *all* your examples?

5. Prove your algorithm is always correct

6. Check running time

Be prepared to go back to step 1!

9/10/10


Writing algorithms

• Clear and unambiguous– Test: You should be able to hand it to any student in

the class, and have them convert it into working code.

• Homework pitfalls:– remember to specify data structures– writing recursive algorithms: don’t confuse the

recursive subroutine with the first call– label global variables clearly

9/10/10


Writing proofs

• Purpose– Determine for yourself that algorithm is correct– Convince reader

• Who is your audience?– Yourself– Your classmate– Not the TA/grader

• Main goal: Find your own mistakes

9/10/10


Exploring a graph

Classic problem: Given vertices s,t ∈ V, is there a path from s to t?

Idea: explore all vertices reachable from s

Two basic techniques:• Breadth-first search (BFS)

• Explore children in order of distance to start node

• Depth-first search (DFS)• Recursively explore vertex’s children before exploring

siblings

9/10/10

How to convert these descriptions to precise algorithms?


Adjacency-matrix representation

The adjacency matrix of a graph G = (V, E), where V = {1, 2, …, n}, is the matrix A[1 . . n, 1 . . n] given by

A[i, j] =1 if (i, j) E∈ ,0 if (i, j) E.

2 1

3 4

A 1 2 3 4

1

2

3

4

0 1 1 0

0 0 1 0

0 0 0 0

0 0 1 0

Storage: ϴ(V 2) Good for dense graphs.

• Lookup: O(1) time• List all neighbors: O(|V|)

9/10/10

Adjacency list representation

• An adjacency list of a vertex v ∈ V is the list Adj[v] of vertices adjacent to v.


2 1

3 4

Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}

For undirected graphs, | Adj[v] | = degree(v).For digraphs, | Adj[v] | = out-degree(v).

How many entries in lists? 2|E| (“handshaking lemma”)Total ϴ(V + E) storage — good for sparse graphs.

Typical notation:n = |V| = # verticesm = |E| = # edges


DFS pseudocode

• Maintain a global counter time• Maintain for each vertex v

– Two timestamps:• v.d = time first discovered• v.f = time when finished

– “color”: v.color • white = unexplored• gray = in process• black = finished

– Parent v.π in DFS tree

9/10/10


DFS pseudocode

• Note: recursive function different from first call…

• Exercise: change code to set “parent” pointer?

9/10/10


DFS example

• T = tree edge (drawn in red)• F = forward edge (to a descendant in DFS forest)• B= back edge (to an ancestor in DFS forest)• C = cross edge (goes to a vertex that is neither ancestor nor descendant)

9/10/10


DFS with adjacency lists

• Outer code runs once, takes time O(n) (not counting time to execute recursive calls)

• Recursive calls:– Run once per vertex– time = O(degree(v))

• Total: O(m+n)

9/10/10


Toplogical Sort

9/10/10

Directed Acyclic Graphs

Def. An DAG is a directed graph that contains no directed cycles.

Ex. Precedence constraints: edge (vi, vj) means vi must precede vj.

Def. A topological order of a directed graph G = (V, E) is an ordering of its nodes as v1, v2, …, vn so that for every edge (vi, vj)

we have i < j.

a DAG a topological ordering

v2 v3

v6 v5 v4

v7 v1

v1 v2 v3 v4 v5 v6 v7

Precedence Constraints

Precedence constraints. Edge (vi, vj) means task vi must occur before vj.

Applications. Course prerequisite graph: course vi must be taken before vj. Compilation: module vi must be compiled before vj. Pipeline of

computing jobs: output of job vi needed to determine input of job vj.

Getting dressed

underwearpants

jacket

shirt

boots

socks

mittens


Recall from book

• Every DAG has a topological order

• If G graph has a topological order, then G is a DAG.

9/10/10


Review

• Suppose your run DFS on a DAG G=(V,E)• True or false?

– Sorting by discovery time gives a topological order– Sorting by finish time gives a topological order

9/10/10


Shortest Paths

9/10/10

A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne9/10/10

Shortest Path Problem

• Input:– Directed graph G = (V, E).– Source node s, destination node t.– for each edge e, length (e) = length of e.– length path = sum of edge lengths

• Find: shortest directed path from s to t.

Length of path (s,2,3,5,t)is 9 + 23 + 2 + 16 = 50.

s

3

t

2

6

7

4

5

23

18

2

9

14

155

30

20

44

16

11

6

19

6


Rough algorithm (Dijkstra)

•Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known.•Initialize S = { s }, d(s) = 0.

•Repeatedly choose unexplored node v which minimizes

•add v to S, and set d(v) = (v).

,)()(min)(:),(

eudvSuvue

shortest path to some u in explored part, followed by a single edge (u, v)

s

v

u

d(u)

S

(e)


Rough algorithm (Dijkstra)

•Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known.•Initialize S = { s }, d(s) = 0.

•Repeatedly choose unexplored node v which minimizes

•add v to S, and set d(v) = (v). shortest path to some u in explored part, followed by a single edge (u, v)

s

v

u

d(u)

S

(e)BFS withweighted edges

,)()(min)(:),(

eudvSuvue

Invariant: d(u) is known for all vertices in S


Demo of Dijkstra’s Algorithm

A

B D

C E

10

3

1 4 7 98

2

2

Graph with nonnegative edge lengths:



A

B D

C E

10

3

1 4 7 98

2

2

Initialize:

A B C D EQ:0 ¥ ¥ ¥ ¥

S: {}

0

¥

¥ ¥

¥



A

B D

C E

10

3

1 4 7 98

2

2A B C D EQ:0 ¥ ¥ ¥ ¥

S: { A }

0

¥

¥ ¥

¥“A” EXTRACT-MIN(Q):



A

B D

C E

10

3

1 4 7 98

2

2A B C D EQ:0 ¥ ¥ ¥ ¥

S: { A }

0

10

3 ¥

¥

10 3

Explore edges leaving A:

¥ ¥



A

B D

C E

10

3

1 4 7 98

2

2A B C D EQ:0 ¥ ¥ ¥ ¥

S: { A, C }

0

10

3 ¥

¥

10 3

“C” EXTRACT-MIN(Q):

¥ ¥



A

B D

C E

10

3

1 4 7 98

2

2A B C D EQ:0 ¥ ¥ ¥ ¥

S: { A, C }

0

7

3 5

11

10 37 11 5

Explore edges leaving C:

¥ ¥



A

B D

C E

10

3

1 4 7 98

2

2A B C D EQ:0 ¥ ¥ ¥ ¥

S: { A, C, E }

0

7

3 5

11

10 37 11 5

“E” EXTRACT-MIN(Q):

¥ ¥



A

B D

C E

10

3

1 4 7 98

2

2A B C D EQ:0 ¥ ¥ ¥ ¥

S: { A, C, E }

0

7

3 5

11

10 3 ¥ ¥7 11 57 11

Explore edges leaving E:



A

B D

C E

10

3

1 4 7 98

2

2A B C D EQ:0 ¥ ¥ ¥ ¥

S: { A, C, E, B }

0

7

3 5

11

10 3 ¥ ¥7 11 57 11

“B” EXTRACT-MIN(Q):



A

B D

C E

10

3

1 4 7 98

2

2A B C D EQ:0 ¥ ¥ ¥ ¥

S: { A, C, E, B }

0

7

3 5

9

10 3 ¥ ¥7 11 57 11

Explore edges leaving B:

9



A

B D

C E

10

3

1 4 7 98

2

2A B C D EQ:0 ¥ ¥ ¥ ¥

S: { A, C, E, B, D }

0

7

3 5

9

10 3 ¥ ¥7 11 57 11

9

“D” EXTRACT-MIN(Q):


Proof of Correctness (Greedy Stays Ahead)

Invariant. For each node u S, d(u) is the length of the shortest path from s to u.

Proof: (by induction on |S|)•Base case: |S| = 1 is trivial.•Inductive hypothesis: Assume for |S| = k 1.– Let v be next node added to S, and let (u,v) be the chosen edge.– The shortest s-u path plus (u,v) is an s-v path of length (v).– Consider any s-v path P. We'll see that it's no shorter than (v).– Let (x,y) be the first edge in P that leaves S,

and let P' be the subpath to x.– P + (x,y) has length · d(x)+ (x,y)· (y)· (v)

S

s

y

v

x

P

u

P'


Review Question

• Is Dijsktra’s algorithm correct with negative edge weights?

9/10/10


Implementation

•For unexplored nodes, maintain–Next node to explore = node with minimum (v).–When exploring v, for each edge e = (v,w), update

•Efficient implementation: Maintain a priority queue of unexplored nodes, prioritized by (v).

).()(min)(:),(

eudvSuvue

.)}()( ),( {min )( evww

Priority Queue


Priority queues

• Maintain a set of items with priorities (= “keys”)– Example: jobs to be performed

• Operations:– Insert– Increase key– Decrease key– Extract-min: find and remove item with least key

• Common data structure: heap– Time: O(log n) per operation

9/10/10


Pseudocode for Dijkstra(G, )

d[s] 0for each v Î V – {s}

do d[v] ¥; p[v] ¥S Q V ⊳ Q is a priority queue maintaining V – S,

keyed on [v]while Q ¹

do u EXTRACT-MIN(Q)S S È {u}; d[u] p [u] for each v Î Adjacency-list[u]

do if [v] > [u] + (u, v)then [v] d[u] + (u, v)

explore edges

leaving vImplicit DECREASE-KEY


Analysis of Dijkstra

degree(u)times

n times

Handshaking Lemma ·m implicit DECREASE-KEY’s.

while Q ¹ do u EXTRACT-MIN(Q)

S S È {u}for each v Î Adj[u]

do if d[v] > d[u] + w(u, v)then d[v] d[u] + w(u, v)

† Individual ops are amortized bounds

PQ Operation

ExtractMin

DecreaseKey

Binary heap

log n

log n

Fib heap †

log n

1

Array

n

1Total m log n m + n log nn2

Dijkstra

n

m

d-way Heap

HW3

HW3m log m/n n

Documents

Algorithm Design and Analysis