Minimum Spanning Trees Definition of MST Generic MST algorithm Kruskal's algorithm Prim's algorithm Binary Search Trees1

Minimum Spanning Trees

Definition of MSTGeneric MST algorithmKruskal's algorithmPrim's algorithm

Binary Search Trees 1

2

Problem: Laying Telephone Wire

Central office

3

Wiring: Naïve Approach

Central office

Expensive!

4

Wiring: Better Approach

Central office

Minimize the total length of wire connecting the customers

Definition of MST

• Let G=(V,E) be a connected, undirected graph. • For each edge (u,v) in E, we have a weight w(u,v) specifying

the cost (length of edge) to connect u and v.• We wish to find a (acyclic) subset T of E that connects all of

the vertices in V and whose total weight is minimized.• Since the total weight is minimized, the subset T must be

acyclic (no circuit). • Thus, T is a tree. We call it a spanning tree.• The problem of determining the tree T is called the

minimum-spanning-tree problem.


Application of MST: an example

• In the design of electronic circuitry, it is often necessary to make a set of pins electrically equivalent by wiring them together.

• To interconnect n pins, we can use n-1 wires, each connecting two pins.

• We want to minimize the total length of the wires.• Minimum Spanning Trees can be used to model this

problem.


Electronic Circuits:


Electronic Circuits:



Here is an example of a connected graph and its minimum spanning tree:

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

Notice that the tree is not unique: replacing (b,c) with (a,h) yields another spanning tree with the same minimum weight.

Growing a MST(Generic Algorithm)

• Set A is always a subset of some minimum spanning tree. This property is called the invariant Property.

• An edge (u,v) is a safe edge for A if adding the edge to A does not destroy the invariant.

• A safe edge is just the CORRECT edge to choose to add to T.


GENERIC_MST(G,w)1 A:={}2 while A does not form a spanning tree do3 find an edge (u,v) that is safe for A4 A:=A∪{(u,v)}5 return A

How to find a safe edge

We need some definitions and a theorem.• A cut (S,V-S) of an undirected graph G=(V,E) is a

partition of V.• An edge crosses the cut (S,V-S) if one of its endpoints

is in S and the other is in V-S.• An edge is a light edge crossing a cut if its weight is

the minimum of any edge crossing the cut.



V-S↓

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

S↑ ↑ S

↓ V-S

• This figure shows a cut (S,V-S) of the graph.

• The edge (d,c) is the unique light edge crossing the cut.

Theorem 1 (• Let G=(V,E) be a connected, undirected graph with a real-

valued weight function w defined on E.• Let A be a subset of E that is included in some minimum

spanning tree for G.• Let (S,V-S) be any cut of G such that for any edge (u, v) in

A, {u, v} S or {u, v} (V-S). • Let (u,v) be a light edge crossing (S,V-S).• Then, edge (u,v) is safe for A. (The proof is not required)• Proof: Let Topt be a minimum spanning tree.(Blue)• A --a subset of Topt and (u,v)-- a light edge crossing (S, V-S)• If (u,v) is NOT safe, then (u,v) is not in T. (See the red edge in Figure)• There MUST be another edge (u’, v’) in Topt crossing (S, V-S).(Green)• We can replace (u’,v’) in Topt with (u, v) and get another treeT’opt

• Since (u, v) is light , T’opt is also optimal.


1

2

11 1

1 1

1

Corollary .2• Let G=(V,E) be a connected, undirected graph with a real-

valued weight function w defined on E.• Let A be a subset of E that is included in some minimum

spanning tree for G.• Let C be a connected component (tree) in the forest

GA=(V,A).• Let (u,v) be a light edge (shortest among all edges

connecting C with others components) connecting C to some other component in GA.

• Then, edge (u,v) is safe for A. (For Kruskal’s algorithm)


Prim's algorithm(basic part)

MST_PRIM(G,w,r)1. A={}2. S:={r} (r is an arbitrary node in V)3. Q=V-{r}; 4. while Q is not empty do {5 take an edge (u, v) such that (1) u S and v Q (v S ) and (u, v)

is the shortest edge satisfying (1)

6 add (u, v) to A, add v to S and delete v from Q }


Prim's algorithmMST_PRIM(G,w,r)1 for each u in Q do2 key[u]:=∞3 parent[u]:=NIL4 key[r]:=0; parent[r]=NIL;5 QV[Q]6 while Q!={} do7 u:=EXTRACT_MIN(Q); if parent[u]Nil print (u, parent[u])8 for each v in Adj[u] do9 if v in Q and w(u,v)<key[v]10 then parent[v]:=u11 key[v]:=w(u,v)


• Grow the minimum spanning tree from the root vertex r.

• Q is a priority queue, holding all vertices that are not in the tree now.

• key[v] is the minimum weight of any edge connecting v to a vertex in the tree.

• parent[v] names the parent of v in the tree.• When the algorithm terminates, Q is empty; the

minimum spanning tree A for G is thus A={(v,parent[v]):v∈V-{r}}.

• Running time: O(||E||lg |V|). • When |E|=O(|V|), the algorithm is very fast.



a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

The execution of Prim's algorithm(moderate part)

the root vertex


a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8


a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8


a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

Adaptable Heap:

• An adaptable heap is a heap that allows to change the key value of an entry in heap.– How to find the entry?– After changing the key value, how to make sure that

the heap order holds.• We use array representation to store a complete

binary tree.


Adaptable Heap:

• Entry: (key, id, value), where key is a real number (double), value is a string (String) and id is an integer (int) indicating the entry.

• Here id uniquely determine the entry. The value of id is from 1 to n if there are n entries in the priority queue.

• For MST application, id is the name of a vertex in a graph. Value is the name of the other vertex of the edge that we will added into A.


Adaptable Heap:

class ArrayNode { double key; String value; // value stored at this position int id; // identificsation of the entry id=1,2, 3, …, n add necessary methods here}public class MyAdaptableHeap{ protected ArrayNode T[]; // array of elements stored in the tree protected int maxEntries; // maximum number of entries protected int numEntries; //num of entries in the heap protected int location[]; // an array with size=T.length. here location[id] stores the rank of entry id in T[]. Add necessary methods here. In particular, we need Replace(id, k) (The key of entry id is updated to be key=k and we assume that the

entry is already in the heap.)


Adaptable Heap:


11/6

13/412/7

15/314/1 16/2 17/5

11 12 13 14 15

1 2 3 4 50 6 7

16 17

4 6 5 3 7

1 2 3 4 50 6 7

1 2

Array-based rep of

the complete

binary tree T[]

location

Replace (id, k) =replace (4, 25):

go to location[4] to find the rank=3 of entry 4 and goto T[rank]=T[3] and update T[3]=25. After change T[3] from 13 to 25, the binary tree is not a

heap any more.

The algorithms of Kruskal and Prim

• The two algorithms are elaborations of the generic algorithm.

• They each use a specific rule to determine a safe edge in line 3 of GENERIC_MST.

• In Kruskal's algorithm, – The set A is a forest.– The safe edge added to A is always a least-weight edge

in the graph that connects two distinct components.• In Prim's algorithm,

– The set A forms a single tree.– The safe edge added to A is always a least-weight edge

connecting the tree to a vertex not in the tree.


Kruskal's algorithm(basic part)

1 (Sort the edges in an increasing order)2 A:={}3 while E is not empty do {3 take an edge (u, v) that is shortest in E and delete it from E4 if u and v are in different components then add (u, v) to A }Note: each time a shortest edge in E is considered.


Kruskal's algorithm

MST_KRUSKAL(G,w)1 A:={}2 for each vertex v in V[G]3 do MAKE_SET(v)4 sort the edges of E by nondecreasing weight w5 for each edge (u,v) in E, in order by nondecreasing

weight6 do if FIND_SET(u) != FIND_SET(v)7 then A:=A {(u,v)}∪8 UNION(u,v)9 return A(Disjoint set is discussed in Chapter 21, Page 498)


Disjoint-Set Keep a collection of sets S1, S2, .., Sk,

– Each Si is a set, e,g, S1={v1, v2, v8}.

• Three operations– Make-Set(x)-creates a new set whose only member is

x.– Union(x, y) –unites the sets that contain x and y, say, Sx

and Sy, into a new set that is the union of the two sets.

– Find-Set(x)-returns a pointer to the representative of the set containing x.

– Each operation takes O(log n) time.


• Our implementation uses a disjoint-set data structure to maintain several disjoint sets of elements.

• Each set contains the vertices in a tree of the current forest.

• The operation FIND_SET(u) returns a representative element from the set that contains u.

• Thus, we can determine whether two vertices u and v belong to the same tree by testing whether FIND_SET(u) equals FIND_SET(v).

• The combining of trees is accomplished by the UNION procedure.

• Running time O(|E| lg (|E|)). )



a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

The execution of Kruskal's algorithm (Moderate part)

•The edges are considered by the algorithm in sorted order by weight.

•The edge under consideration at each step is shown with a red weight number.


a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8


a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8


a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8


a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8


a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

a

b

h

c d

e

fg

i

4

8 79

10

1442

2

6

1

711

8

Disjoint Sets Data Structure

Disjoint Sets

• Some applications require maintaining a collection of disjoint sets.

• A Disjoint set S is a collection of sets

where

• Each set has a representative which is a member of the set (Usually the minimum if the elements are comparable)

1,...... nS S i j i jS S

Disjoint Set Operations

• Make-Set(x) – Creates a new set where x is it’s only element (and therefore it is the representative of the set).

• Union(x,y) – Replaces by one of the elements of becomes the representative of the new set.

• Find(x) – Returns the representative of the set containing x

,x yS S

x yS Sx yS S

Analyzing Operations

• We usually analyze a sequence of m operations, of which n of them are Make_Set operations, and m is the total of Make_Set, Find, and Union operations

• Each union operations decreases the number of sets in the data structure, so there can not be more than n-1 Union operations

Disjoint-Set Forests

• Maintain A collection of trees, each element points to it’s parent. The root of each tree is the representative of the set

• We use two strategies for improving running time– Union by Rank– Path Compression c

fba

d

Make Set

• MAKE_SET(x) p(x)=x

rank(x)=0

x

Find Set

• FIND_SET(d) if d != p[d]

p[d]= FIND_SET(p[d])

return p[d] (Compression)

c

fba

d

Union

• UNION(x,y) link(findSet(x),

findSet(y))

• link(x,y) if rank(x)>rank(y)

then p(y)=x

else

p(x)=y

if rank(x)=rank(y)

then rank(y)++

c

fba

d

w

x

c

fba

d

w

x

Analysis

• In Union we attach a smaller tree to the larger tree, results in logarithmic depth. O(log n).– Whenever height of a tree increases by 1, size doubles.

• Path compression can cause a very deep tree to become very shallow

The length of the Tree is O(log n)

If n=1, it is obvious.Assume that it is true for n<=k.Let us consider a tree T of size k+1. Assume that T is constructed from T1 and T2. Then rank(T1)<=log (n1) and rank(T2)<=log (n2). Let n*=mim{n1, n2}. If rank(T1)rank(T2), then rank(T)=max(rank(T1), rank(T2)}<=max{log (T1), log (T2)}<log (n1+n2)=log n . If rank(T1)=rank(T2), rank(T)=rank(T1)+1<=1+log n*=log (2n*)<=log(n) (note n=n1+n2).

Documents

Minimum Spanning Trees Definition of MST Generic MST algorithm Kruskal's algorithm Prim's algorithm Binary Search Trees1