Part8 Graphs

7/29/2019 Part8 Graphs

1/123

CSE 326: Data Structures

Part 8

Graphs

Henry KautzAutumn Quarter 2002


2/123

Outline

Graphs (TO DO: READ WEISS CH 9)

Graph Data Structures

Graph Properties

Topological Sort

Graph Traversals

Depth First Search

Breadth First Search Iterative Deepening Depth First

Shortest Path Problem

Dijkstras Algorithm


3/123

Graph ADT

Graphs are a formalism for representingrelationships between objects

a graph G is represented asG = (V, E)

Vis a set of vertices E is a set of edges

operations include:

iterating over vertices iterating over edges

iterating over vertices adjacent to a specific vertex

asking whether an edge exists connected two vertices

Han

Leia

Luke

V = {Han, Leia, Luke}

E = {(Luke, Leia),

(Han, Leia),

(Leia, Han)}


4/123

What Graph is THIS?


5/123

ReferralWeb

(co-authorship in scientific papers)


6/123

Biological Function Semantic Network


7/123

Graph Representation 1:

Adjacency MatrixA |V| x |V| array in which an element (u, v)

is true if and only if there is an edge from uto v

Han

Leia

Luke

Han Luke Leia

Han

Luke

LeiaRuntime:

iterate over vertices

iterate ever edges

iterate edges adj. to vertex

edge exists?Space requirements:


8/123

Graph Representation 2:

Adjacency ListA |V|-ary list (array) in which each entry stores a

list (linked list) of all adjacent vertices

Han

Leia

LukeHan

Luke

Leia

space requirements:

Runtime:

iterate over vertices

iterate ever edges

iterate edges adj. to vertex

edge exists?


9/123

Directed vs. Undirected Graphs

Han

Leia

Luke

Han

Leia

Luke

In directedgraphs, edges have a specific direction:

In undirectedgraphs, they dont (edges are two-way):

Vertices u and v are adjacentif(u, v) E


10/123

Graph Density

A sparse graph has O(|V|) edges

A dense graph has (|V|2) edges

Anything in between is either sparsish or densy depending on the context.


11/123

Weighted Graphs

20

30

35

60

Mukilteo

Edmonds

Seattle

Bremerton

Bainbridge

Kingston

Clinton

There may be more

information in the graph as well.

Each edge has an associated weight or cost.


12/123

Paths and CyclesApath is a list of vertices {v1, v2, , vn} such

that (vi, vi+1) E for all 0 i < n.

A cycle is a path that begins and ends at the samenode.

Seattle

San Francisco

Dallas

Chicago

Salt Lake City

p = {Seattle, Salt Lake City, Chicago, Dallas, San Francisco, Seattle}


13/123

Path Length and Cost

Path length: the number of edges in the pathPath cost: the sum of the costs of each edge

Seattle

San Francisco

Dallas

Chicago

Salt Lake City

3.5

2 2

2.5

3

22.5

2.5

length(p) = 5 cost(p) = 11.5


14/123

ConnectivityUndirected graphs are connectedif there is a path between

any two vertices

Directed graphs are strongly connectedif there is a path from

any one vertex to any other

Directed graphs are weakly connectedif there is a path

between any two vertices, ignoring direction

A complete graph has an edge between every pair of vertices


15/123

Trees as Graphs

Every tree is a graph with

some restrictions:

the tree is directed there are no cycles (directed

or undirected)

there is a directed path from

the root to every node

A

B

D E

C

F

HG

JI

BAD!


16/123

Directed Acyclic Graphs (DAGs)

DAGs are directed

graphs with no

cycles.

main()

add()

access()

mult()

read()

Trees DAGs Graphs

if program call

graph is a DAG,

then allprocedure calls

can be in-lined


17/123

Application of DAGs:

Representing Partial Orders

check in

airport

call

taxi

taxi to

airport

reserve

flight

pack

bagstake

flight

locate

gate


18/123

Topological Sort

Given a graph, G = (V, E), output all the verticesinVsuch that no vertex is output before any other

vertex with an edge to it.

check in

airportcall

taxi

taxi toairport

reserve

flight

pack

bags

take

flightlocate

gate


19/123

Topo-Sort Take One

Label each vertexs in-degree (# of inbound edges)

While there are vertices remaining

Pick a vertex with in-degree of zero and output it

Reduce the in-degree of all vertices adjacent to it

Remove it from the list of vertices

runtime:


20/123

Topo-Sort Take Two

Label each vertexs in-degree

Initialize a queue (or stack)to contain all in-degree zero

verticesWhile there are vertices remaining in the queue

Remove a vertex v with in-degree of zero and output it

Reduce the in-degree of all vertices adjacent to vPut any of these with new in-degree zero on the queue

runtime:


21/123

Recall: Tree Traversals

a

i

d

h j

b

f

k l

ec

g

a b f g k c d h i l j e


22/123

Depth-First Search Pre/Post/Inorder traversals are examples of

depth-first search

Nodes are visited deeply on the left-most branches

before any nodes are visited on the right-most branches Visiting the right branches deeply before the left would still be

depth-first! Crucial idea is go deep first!

Difference in pre/post/in-order is how some computation (e.g.

printing) is done at current node relative to the recursive calls

In DFS the nodes being worked on are kept ona stack


23/123

Iterative Version DFS

Pre-order Traversal

Push root on a Stack

Repeat until Stack is empty:Pop a node

Process it

Push its children on the Stack


24/123

Level-Order Tree Traversal

Consider task of traversing tree level by level fromtop to

bottom (alphabetic order)

Is this also DFS? a

i

d

h j

b

f

k l

ec

g


25/123

Breadth-First Search

No! Level-order traversal is an example ofBreadth-FirstSearch

BFS characteristics

Nodes being worked on maintained in a FIFO Queue, not a stack Iterative style procedures often easier to design than recursive

procedures

Put root in a Queue

Repeat until Queue is empty:

Dequeue a node

Process it

Add its children to queue


26/123

QUEUE

a

b c d e

c d e f g

d e f g

e f g h i j

f g h i j

g h i j

h i j k

i j kj k l

k l

l

a

i

d

h j

b

f

k l

ec

g


27/123

Graph Traversals

Depth first search and breadth first search also work forarbitrary (directed or undirected) graphs

Must mark visited vertices so you do not go into an infinite

loop!

Either can be used to determine connectivity:

Is there a path between two given vertices?

Is the graph (weakly) connected?

Important difference: Breadth-first search always findsa shortest path from the start vertex to any other (for

unweighted graphs)

Depth first search may not!


28/123

Demos on Web Page

DFSBFS


29/123

Is BFS the Hands Down Winner?

Depth-first search Simple to implement (implicit or explict stack)

Does not always find shortest paths

Must be careful to mark visited vertices, or you could

go into an infinite loop if there is a cycle Breadth-first search

Simple to implement (queue)

Always finds shortest paths

Marking visited nodes can improve efficiency, but even

without doing so search is guaranteed to terminate


30/123

Space Requirements

Consider space required by the stack or queue

Suppose

G is known to be at distance dfrom S

Each vertex n has kout-edges

There are no (undirected or directed) cycles

BFS queue will grow to size kd

Will simultaneously contain all nodes that are at

distance d(once last vertex at distance d-1 is expanded)

For k=10, d=15, size is 1,000,000,000,000,000


31/123

DFS Space Requirements

Consider DFS, where we limit the depth of the searchto d

Force a backtrack at d+1

When visiting a node n at depth d, stack will contain

(at most) k-1 siblings ofn parent ofn

siblings of parent ofn

grandparent ofn

siblings of grandparent ofn

DFS queue grows at most to size dk

For k=10, d=15, size is 150

Compare with BFS 1,000,000,000,000,000


32/123

Conclusion

For very large graphsDFS is hugely more

memory efficient, ifwe know the distance to the

goal vertex! But suppose we dont know d. What is the

(obvious) strategy?


33/123

Iterative Deepening DFS

IterativeDeepeningDFS(vertex s, g){

for (i=1;true;i++)

if DFS(i, s, g) return;

}

// Also need to keep track of path found

bool DFS(int limit, vertex s, g){

if (s==g) return true;

if (limit--


34/123

Analysis of Iterative Deepening

Even without marking nodes as visited,

iterative-deepening DFS never goes into an

infinite loop For very large graphs, memory cost of keeping track of

visited vertices may make marking prohibitive

Work performed with limit < actual distance to G

is wastedbut the wasted work is usually smallcompared to amount of work done during the last

iteration


35/123

Asymptotic Analysis

There are pathological graphs for which

iterative deepening is bad:

S G

n=d

2

Iterative Deepening DFS =

1 2 3 ... ( )

BFS = ( )

n O n

O n


36/123

A Better Case

Suppose each vertex n has kout-edges, no cycles

Bounded DFS to level i reaches ki vertices

Iterative Deepening DFS(d) =

1

( )

BFS = ( )

d

i d

i

d

k O k

O k

ignore low order terms!


37/123

(More) Conclusions

To find a shortest path between two nodes in a

unweighted graph, use either BFS or Iterated DFS

If the graph is large, Iterated DFS typically uses

much less memory

Later well learn about heuristic search algorithms,

which use additional knowledge about the problem

domain to reduce the number of vertices visited


38/123

Single Source, Shortest Path for

Weighted GraphsGiven a graph G = (V, E) with edge costs c(e),

and a vertex s V, find the shortest (lowest cost)

path from s to every vertex inV

Graph may be directed or undirected

Graph may or may not contain cycles

Weights may be all positive or not

What is the problem if graph contains cycleswhose total cost is negative?


39/123

The Trouble with

Negative Weighted Cycles

A B

C D

E

2

101-5

2


40/123

Edsger Wybe Dijkstra

(1930-2002)

Invented concepts of structured programming,synchronization, weakest precondition, and "semaphores"

for controlling computer processes. The Oxford EnglishDictionary cites his use of the words "vector" and "stack" ina computing context.

Believed programming should be taught without computers

1972 Turing Award

In their capacity as a tool, computers will be but a ripple onthe surface of our culture. In their capacity as intellectualchallenge, they are without precedent in the cultural historyof mankind.


41/123

Dijkstras Algorithm for

Single Source Shortest Path Classic algorithm for solving shortest path in

weighted graphs (with onlypositive edge weights)

Similar to breadth-first search, but uses a priorityqueue instead of a FIFO queue:

Always select (expand) the vertex that has a lowest-costpath to the start vertex

a kind of greedy algorithm

Correctly handles the case where the lowest-cost(shortest) path to a vertex is not the one withfewest edges


42/123

Pseudocode for Dijkstra

Initialize the cost of each vertex to cost[s] = 0;

heap.insert(s);

While (! heap.empty())

n = heap.deleteMin()For (each vertex a which is adjacent to n along edge e)

if (cost[n] + edge_cost[e] < cost[a]) then

cost [a] = cost[n] + edge_cost[e]

previous_on_path_to[a] = n;if (a is in the heap) then heap.decreaseKey(a)

else heap.insert(a)


43/123

Important Features

Once a vertex is removed from the head, the cost

of the shortest path to that node is known

While a vertex is still in the heap, another shorterpath to it might still be found

The shortest path itselffrom s to any node a can

be found by following the pointers stored in

previous_on_path_to[a]


44/123

Dijkstras Algorithm in Action

A

C

B

D

F H

G

E

2 2 3

21

1

410

8

11

94

2

7

vertex known cost

A

B

C

D

E

F

G

H


45/123

Demo

Dijkstras


46/123

Data Structures

for Dijkstras AlgorithmSelect the unknown node with the lowest cost

findMin/deleteMin

as cost = min(as old cost, )

decreaseKey

|V| times:

|E| times:

runtime: O(|E| log |V|)

O(log |V|)

O(log |V|)


47/123


Lecture 8.B

Heuristic Graph Search

Henry Kautz

Winter Quarter 2002


48/123

Homework Hint - Problem 4

You can turn in a final version of your answer toproblem 4 without penalty on Wednesday.

1

1 2 1 1

1

2

1 2

final mod in case sum is

Let ( ... ) be the interpretation of a bit string

as a bin

( ) mod ( mod

ary numbe

mod ) mod

( ( mod )) mod ( )

r. Then:

mod

( ... ) 2 ( .

k

k k k k k k

k k k

a b p a p b p p

c a p

p

b b b b

p ca p

b b b b b b b

1.. )b


49/123

Outline

Best First Search

A* Search

Example: Plan Synthesis

This material is NOT in Weiss, but is important

for both the programming project and the finalexam!


50/123

Huge Graphs

Consider some really hugegraphs

All cities and towns in the World Atlas

All stars in the Galaxy

All ways 10 blocks can be stacked

Huh???
http://images.google.com/imgres?imgurl=skyserver.fnal.gov/images/sdss-float-galaxy-1.gif&imgrefurl=http://skyserver.fnal.gov/&h=160&w=160&prev=/images%3Fq%3Dgalaxy%26svnum%3D10%26hl%3Denhttp://images.google.com/imgres?imgurl=www.sgi.com/sales/images/earth.jpg&imgrefurl=http://www.sgi.com/sales/solutions_finance/&h=148&w=150&prev=/images%3Fq%3Dearth%26svnum%3D10%26hl%3Den


51/123

Implicitly Generated Graphs

A huge graph may be implicitly specified by rules forgenerating it on-the-fly

Blocks world: vertex = relative positions of all blocks

edge = robot arm stacks one block

stack(blue,red)

stack(green,red)

stack(green,blue)

stack(blue,table)

stack(green,blue)


52/123

Blocks World

Source = initial state of the blocks

Goal = desired state of the blocks

Path source to goal = sequence of actions(program) for robot arm!

n blocks nn vertices

10 blocks

10 billion vertices!


53/123

Problem: Branching Factor

Cannot search such huge graphs exhaustively.

Suppose we know that goal is only dsteps away.

Dijkstras algorithm is basically breadth-firstsearch (modified to handle arc weights)

Breadth-first search (or for weighted graphs,

Dijkstras algorithm) If out-degree of each node

is 10, potentially visits 10dvertices 10 step plan = 10 billion vertices visited!


54/123

An Easier Case

Suppose you live in Manhattan; what do you do?

52nd St

51st St

50th St

10thAve

9thAve

8thAve

7thAve

6thAve

5thAve

4thAve

3rdAve

2ndAve

S

G


55/123

Best-First Search

TheManhattan distance ( x+ y) is an estimate

of the distance to the goal

a heuristic value

Best-First Search

Order nodes in priority to minimize estimated distance

to the goal h(n)

Compare: BFS / Dijkstra Order nodes in priority to minimize distance from the

start


56/123

Best First in Action

Suppose you live in Manhattan; what do you do?

52nd St

51st St

50th St

10thAve

9thAve

8thAve

7thAve

6thAve

5thAve

4thAve

3rdAve

2ndAve

S

G


57/123

Problem 1: Led Astray

Eventually will expand vertex to get back on the

right track

52nd St

51st St

50th St

10thAve

9thAve

8thAve

7thAve

6thAve

5thAve

4thAve

3rdAve

2ndAve

SG


58/123

Problem 2: Optimality

With Best-First Search, are you guaranteeda

shortest path is found when

goal is first seen?

when goal is removed from priority queue (as with

Dijkstra?)


59/123

Sub-Optimal Solution No! Goal is by definition at distance 0: will be

removed from priority queue immediately, even if

a shorter path exists!

52nd St

51st St

9thAve

8thAve

7thAve

6thAve

5thAve

4thAve

S

G

(5 blocks)

h=2

h=1h=4

h=5


60/123

Synergy?

Dijkstra / Breadth First guaranteed to find optimal

solution

Best First often visitsfar fewervertices, but maynot provide optimal solution

Can we get the best of both?


61/123

A* (A star)

Order vertices in priority queue to minimize

(distance from start) + (estimated distance to goal)

f(n) = g(n) + h(n)

f(n) = priority of a nodeg(n) = true distance from start

h(n) = heuristic distance to goal


62/123

Optimality Suppose the estimated distance (h) is

always less than or equal to the true distance to the

goal

heuristic is a lower bound on true distance

Then: when the goal is removed from the priority

queue, we are guaranteed to have found a shortest

path!


63/123

Problem 2 Revisited

52nd St

51st St

9thAve

8thAve

7thAve

6thAve

5thAve

4thAve

S

G

(5 blocks)

50th St

vertex g(n) h(n) f(n)

52nd & 9th 0 5 5


64/123

Problem 2 Revisited

52nd St

51st St

9thAve

8thAve

7thAve

6thAve

5thAve

4thAve

S

G

(5 blocks)

50th St


52nd & 4th 5 2 7

51st & 9th 1 4 5


65/123

Problem 2 Revisited

52nd St

51st St

9thAve

8thAve

7thAve

6thAve

5thAve

4thAve

S

G

(5 blocks)

50th St


52nd & 4th 5 2 7

51st & 8th 2 3 5

50th & 9th 2 5 7


66/123

Problem 2 Revisited

52nd St

51st St

9thAve

8thAve

7thAve

6thAve

5thAve

4thAve

S

G

(5 blocks)

50th St


52nd & 4th 5 2 7

51st & 7th 3 2 5

50th & 9th 2 5 7

50th & 8th 3 4 7


67/123

Problem 2 Revisited

52nd St

51st St

9thAve

8thAve

7thAve

6thAve

5thAve

4thAve

S

G

(5 blocks)

50th St


52nd & 4th 5 2 7

51st & 6th 4 1 5

50th & 9th 2 5 7

50th & 8th 3 4 7

50th

& 7th

4 3 7


68/123

Problem 2 Revisited

52nd St

51st St

9thAve

8thAve

7thAve

6thAve

5thAve

4thAve

S

G

(5 blocks)

50th St


52nd & 4th 5 2 7

51st & 5th 5 0 5

50th & 9th 2 5 7

50th & 8th 3 4 7

50th

& 7th

4 3 7


69/123

Problem 2 Revisited

52nd St

51st St

9thAve

8thAve

7thAve

6thAve

5thAve

4thAve

S

G

(5 blocks)

50th St


52nd & 4th 5 2 7

50th & 9th 2 5 7

50th & 8th 3 4 7

50th & 7th 4 3 7

DONE!


70/123

What Would Dijkstra Have

Done?

52nd St

51st St

9thAve

8thAv

e

7thAv

e

6thAv

e

5thAv

e

4thAv

e

S

G

(5 blocks)

50th St

49th St

48th St

47th St

Proof of A* Optimality


71/123

Proof of A* Optimality

A* terminates when G is popped from the heap.

Suppose G is popped but the path found isnt optimal:

priority(G) > optimal path length c

Let P be an optimal path from S to G, and let N be the lastvertex on that path that has been visited but not yet popped.

There must be such an N, otherwise the optimal path would have beenfound.

priority(N) = g(N) + h(N) c

So N should have popped before G can pop. Contradiction.

S

N

G

non-optimal path to G

portion of optimal

path found so far

undiscovered portion

of shortest path


72/123

What About Those Blocks?

Distance to goal is not always physical distance

Blocks world:

distance = number of stacks to perform heuristic lower bound = number of blocks out of place

# out of place = 2, true distance to goal = 3

3-Blocks State

S G h


73/123

Space GraphABC

h=2

C

AB

h=3

B

AC

h=2

A

BC

h=1

C

BA

h=3

A

CB

h=2

B

CA

h=1

B

C

A

h=3

C

B

A

h=3

C

A

B

h=3

A

C

B

h=3

B

A

C

h=2

A

B

C

h=0

start goal

3-BlocksBest First


74/123

Solution ABC

h=2

C

AB

h=3

B

AC

h=2

A

BC

h=1

C

BA

h=3

A

CB

h=2

B

CA

h=1

B

C

A

h=3

C

B

A

h=3

C

A

B

h=3

A

C

B

h=3

B

A

C

h=2

A

B

C

h=0

start goal

3-Blocks BFS

Solution


75/123

SolutionABC

h=2

C

AB

h=3

B

AC

h=2

A

BC

h=1

C

BA

h=3

A

CB

h=2

B

CA

h=1

B

C

A

h=3

C

B

A

h=3

C

A

B

h=3

A

C

B

h=3

B

A

C

h=2

A

B

C

h=0

expanded, but not

in solution

start goal

3-Blocks A*

Solution


76/123

SolutionABC

h=2

C

AB

h=3

B

AC

h=2

A

BC

h=1

C

BA

h=3

A

CB

h=2

B

CA

h=1

B

C

A

h=3

C

B

A

h=3

C

A

B

h=3

A

C

B

h=3

B

A

C

h=2

A

B

C

h=0

expanded, but not

in solution

start goal


77/123

Other Real-World Applications

Routing findingcomputer networks, airlineroute planning

VLSI layoutcell layout and channel routing

Production planningjust in time optimization

Protein sequence alignment

Many other NP-Hard problems

A class of problems for which no exact polynomialtime algorithms existso heuristic search is the bestwe can hope for


78/123

Coming Up

Other graph problems

Connected components

Spanning tree


79/123


Part 8.C

Spanning Trees and More

Henry Kautz

Autumn Quarter 2002


80/123

Today

Incremental hashing

MazeRunner project

Longest Path? Finding Connected Components

Application to machine vision

Finding Minimum Spanning Trees

Yet another use for union/find

Incremental Hashing


81/123

Incremental Hashing

11 1

2 1 1

2 2

11 1

1

1 1

11

1 11

1

1 1

1

2

1

( ... ) % %

( ..

%

%

% %

. ) %

n nn i n i

n i n i

i i

nn

n

nn i

nn i

n ii

nn n i

i

i

nn i

n

n

i

i

in

i

h a a c a p a c a p

a c a p

a c

h a a c a p

c a c a

ca p

a p c

a

c cp a

c

a

11 11

%

( ... )

%

%n

nn

p

a c a c a p

p

h a

20 15

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


82/123

Maze Runner+ + + + + + + + + + + + + + + + + + + + +

|* |

+ + + + + + + + + + + + + + + + + + + + +

| | | | | | | | | | | | | | | | | | | | |

+-+-+-+-+ +-+ +-+ +-+ +-+-+ +-+-+-+-+-+-+

| | | | | | | | | | | |

+-+-+-+-+-+-+-+-+-+-+-+ + + + + + + +-+ +

|X | | | | | |

+-+ + +-+-+ +-+-+-+ +-+ +-+ +-+-+-+-+-+-+

| | | | | | | | | | | | | | | | | |

+ + + + + + + + + +-+ + + +-+ + + +-+ +-+

| | | | | | | | | | | | | | |

+-+-+ + + + + + + + + + + + + +-+ + + + +

| | | | | | | | | |

+ + + +-+ + + + + + + + + + + + + + + + +

| | | | | | | | | | | | | | |

+ + + + + + + + + + + + + + +-+-+-+-+ +-+

| | | | | | | | | | | | | | | |

+ + + + + + + + + + +-+ +-+-+ + + +-+-+ +

| | | | | | | | | | | | | | | |

+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+

| | | | | | | | | | | | |

+ + + + + + + + + + +-+ +-+-+-+-+ +-+-+-+

| | | | | | | | | | | | | |

+ + + + +-+ +-+ + + + + +-+ + +-+ + + + +

| | | | | | | | | | | |

+ + +-+-+-+-+ +-+ +-+-+-+ +-+-+ +-+ +-+ +

| | | | | | | | | | | | | | | | | |

+ + + + + + + + + + + + + + + + + + +-+ +

| | | |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

DFS, iterated DFS, BFS,

best-first, A*

Crufty old C++ code from

fresh clean Java code

Win fame and glory by

writing a nice real-time maze

visualizer


83/123

Java Note

Java lacks enumerated constants

enum {DOG, CAT, MOUSE} animal;

animal a = DOG;

Static constants not type-safe

static final int DOG = 1;

static final int CAT = 2;

static final int BLUE = 1;

int favoriteColor = DOG;


84/123

Amazing Java Trick

public final class Animal {

private Animal() {}

public static final Animal DOG = new Animal();

public static final Animal CAT = new Animal();

}

public final class Color {

private Color() {}

public static final Animal BLUE = new Color();

}

Animal x = DOG;

Animal x = BLUE; // Gives compile-time error!


85/123

Longest Path Problem

Given a graph G=(V,E) and vertices s, t

Find a longest simple path (no repeating vertices)

from s to t. Does reverse Dijkstra work?

Dijkstra


86/123

Dijkstra

Initialize the cost of each vertex to

cost[s] = 0;

heap.insert(s);


n = heap.deleteMin()For (each vertex a which is adjacent to n along edge e)

if (cost[n] + edge_cost[e] < cost[a]) then


previous_on_path_to[a] = n;if (a is in the heap) then heap.decreaseKey(a)

else heap.insert(a)

Reverse Dijkstra


87/123

Reverse Dijkstra


cost[s] = 0;

heap.insert(s);


n = heap.deleteMax()For (each vertex a which is adjacent to n along edge e)

if (cost[n] + edge_cost[e] > cost[a]) then


previous_on_path_to[a] = n;if (a is in the heap) then heap.increaseKey(a)

else heap.insert(a)


88/123

Does it Work?

s

ta

b

3

1

5

6


89/123

Problem

No clear stopping condition!

How many times could a vertex be inserted in the

priority queue?

Exponential!

Not a good algorithm!

Is the better one?


90/123

Counting Connected Components


Num_cc = 0

While there are vertices of cost {

Pick an arbitrary such vertex S, set its cost to 0Find paths from S

Num_cc ++ }


91/123

Using DFS

Set each vertex to unvisitedNum_cc = 0

While there are unvisited vertices {

Pick an arbitrary such vertex S

Perform DFS from S, marking vertices as visited

Num_cc ++ }

Complexity = O(|V|+|E|)


92/123

Using Union / Find

Put each node in its own equivalence classNum_cc = 0

For each edge E =

Union(x,y)Return number of equivalence classes

Complexity =


93/123

Using Union / Find

Put each node in its own equivalence classNum_cc = 0

For each edge E =

Union(x,y)Return number of equivalence classes

Complexity = O(|V|+|E| ack(|E|,|V|))


94/123

Machine Vision: Blob Finding


95/123

Machine Vision: Blob Finding

1

2

34

5


96/123

Blob Finding

Matrix can be considered an efficient

representation of a graph with a very regular

structure

Cell = vertex

Adjacent cells of same color = edge between

vertices

Blob finding = finding connected components


97/123

Tradeoffs

Both DFS and Union/Find approaches are

(essentially) O(|E|+|V|) = O(|E|) for binary images

For each component, DFS (recursive labeling)

can move all over the imageentire image must

be in main memory

Better in practice: row-by-row processing

localizes accesses to memory typically 1-2 orders of magnitude faster!

High Level Blob Labeling


98/123

High-Level Blob-Labeling

Scan through image left/right and top/bottom

If a cell is same color as (connected to) cell to

right or below, then union them

Give the same blob number to cells in each

equivalence class

Blob Labeling Algorithm


99/123

Blob-Labeling Algorithm

Put each cell in its own equivalence classFor each cell

if color[x,y] == color[x+1,y] then

Union( , )

if color[x,y] == color[x,y+1] thenUnion( , )

label = 0

For each root

blobnum[x,y] = ++ label;For each cell

blobnum[x,y] = blobnum( Find() )


100/123

Spanning tree: a subset of the edges from a connected graphthat

touches all vertices in the graph (spans the graph)

forms a tree (is connected and contains no cycles)

Minimum spanning tree: the spanning tree with the least totaledge cost.

Spanning Tree

4 7

1 5

9

2

Applications of Minimal


101/123

Applications of Minimal

Spanning Trees Communication networks

VLSI design

Transportation systems

Kruskals Algorithm for
http://images.google.com/imgres?imgurl=www.metropla.net/am/nyrk/nyc-map-centre.gif&imgrefurl=http://www.metropla.net/am/nyrk/nyc.htm&h=716&w=748&prev=/images%3Fq%3Dnyc%2Bsubway%26start%3D40%26svnum%3D10%26hl%3Den%26sa%3DNhttp://www.gel.ulaval.ca/~vision/vlsi/gifs/vlsi.gif


102/123

Kruskal s Algorithm for

Minimum Spanning TreesA greedy algorithm:

Initialize all vertices to unconnectedWhile there are still unmarked edges

Pick a lowest cost edge e = (u, v) and mark it

Ifu and v are not already connected, add e to the

minimum spanning tree and connect u and v

Sound familiar?

(Think maze generation.)


103/123

Kruskals Algorithm in Action (1/5)

A

C

B

D

F H

G

E

2 2 3

2

1

4

10

8

1

94

2

7


104/123


A

C

B

D

F H

G

E

2 2 3

2

1

4

10

8

1

94

2

7


105/123


A

C

B

D

F H

G

E

2 2 3

2

1

4

10

8

1

94

2

7


106/123


A

C

B

D

F H

G

E

2 2 3

2

1

4

10

8

1

94

2

7


107/123

Kruskals Algorithm Completed (5/5)

A

C

B

D

F H

G

E

2 2 3

2

1

4

10

8

1

94

2

7

Why Greediness Works


108/123

yProof by contradictionthat Kruskals finds a minimum

spanning tree:

Assume another spanning tree has lower costthanKruskals.

Pick an edge e1 = (u, v)in that tree thats notinKruskals.

Consider the point in Kruskals algorithm where us setand vs set were about to be connected. Kruskal selectedsome edge to connect them: call it e2 .

But, e2 must have at most the same cost as e1 (otherwiseKruskal would have selected it instead).

So, swap e2 for e1 (at worst keeping the cost the same)

Repeat until the tree is identicalto Kruskals, where thecost is the same or lower than the original cost:contradiction!

Data Structures


109/123

Data Structures

for Kruskals AlgorithmPick the lowest cost edge

findMin/deleteMin

Ifu and v are not already connected

connect u and v.

union

|E| times:

|E| times:

runtime:

Once:

Initialize heap of edges

buildHeap

|E| + |E| log |E| + |E| ack(|E|,|V|)

Data Structures


110/123

Data Structures

for Kruskals AlgorithmPick the lowest cost edge

findMin/deleteMin

Ifu and v are not already connected

connect u and v.

union

|E| times:

|E| times:

runtime:

Once:

Initialize heap of edges

buildHeap

|E| + |E| log |E| + |E| ack(|E|,|V|) = O(|E|log|E|)


111/123

Prims Algorithm

Can also find Minimum Spanning Trees using avariation of Dijkstras algorithm:

Pick a initial node

Until graph is connected:

Choose edge (u,v) which is of minimum costamong edges where u is in tree but v is not

Add (u,v) to the tree Same greedy proof, same asymptotic

complexity


112/123

Coming Up

Application: Sentence Disambiguation

All-pairs Shortest Paths

NP-Complete Problems Advanced topics

Quad trees

Randomized algorithms

Sentence Disambiguation


113/123

Sentence Disambiguation

A person types a message on their cell phonekeypad. Each button can stand for three different

letter (e.g. 1 is a, b, or c), but the person does not

explicitly indicate which letter is meant. (Words are

separated by blanksthe 0 key.)

Problem: How can the system determine what

sentence was typed?

My Nokia cell phone does this! How can this problem be cast as a shortest-path

problem?


114/123

Sentence Disambiguation as


115/123

Sentence Disambiguation as

Shortest PathIdea: Possible words are vertices

Directed edge between adjacent possible words

Weight on edge from W1 to W2 is probability thatW2 appears adjacent to W1 Probabilities over what?! Some large archive (corpus)

of text

Word bi-gram model

Find the most probable path through the graph


116/123

W11

W11W3

1

W41

W21

W12

W22

W13

W23

W33

W43


117/123

Technical Concerns

Isnt most probable actually longest (most

heavily weighted) path?!

Shouldnt we be multiplying probabilities, not

adding them?!

1 2 3 1 2 1 3 2 3(# #) ( | #) ( | ) ( | ) (# | )P w w w P w P w w P w w P w


118/123

Logs to the Rescue

Make weight on edge fromW1 to W2 be

- log P(W2 | W1)

Logs of probabilities are always negativenumbers, so take negative logs

The lower the probability, the larger the negative

log! So this is shortest path

Adding logs is the same as multiplying theunderlying quantities

hi k b


119/123

To Think About

This really works in practice99% accuracy!

Cell phone memory is limitedhow can we use aslittle storage as possible?

How can the system customize itself to a user?

Q i


120/123

Question

Which graph algorithm is asymptotically

better:

(|V||E|log|V|)

(|V|3)

All P i Sh P h


121/123

All Pairs Shortest Path

Suppose you want to compute the length of the

shortest paths between all pairs of vertices in a

graph

Run Dijkstras algorithm (with priority queue)

repeatedly, starting with each node in the graph:

Complexity in terms of V when graph is dense:

Dynamic Programming


122/123

Dynamic Programming

Approach

,

, ,

1, 2

, , 1, , 1, ,

,

1, ,

Note that path for

distance from to that u

either does not use ,

or merges the paths

ses

only ,..., as intermediates

min{ , }

and

k i j

k i j i j

k

k i j

k

i

k i j k i k k k j

k k j

D v v

v v v

D D D D

D v

v v v v

Floyd-Warshall Algorithm


123/123

y g

// C adjacency matrix representation of graph

// C[i][j] = weighted edge i->j or if none// D computed distances

FW(int n, int C [][], int D [][]){

for (i = 0; i < N; i++){

for (j = 0; j < N; j++)

D[i][j] = C[i][j];

D[i][i] = 0.0;

}

for (k = 0; k < N; k++)

for (i = 0; i < N; i++)

for (j = 0; j < N; j++)

if (D[i][k] + D[k][j] < D[i][j])

Run time =

How could we

compute the paths?

Documents

Part8 Graphs