Karger’s Min-Cut Approximate Max-Cut Monday, July 21st 1

1

Karger’s Min-Cut

Approximate Max-Cut

Monday, July 21st

2

Announcements

HW 4 is due this Friday (July 25th)

HW 5: (short hw, 2 or 3 questions)

Out on Wednesday (July 23rd)

Due next Friday (August 1st)

Programming Assignment

Out on Wednesday (July 23rd)

Due 16 days later on Friday (August 8th)

HW 6: (short hw, 2 or 3 questions)

Out on Wednesday (August 6th)

Due next Wednesday (August 13rd)

Fundamental Algorithms to Tractable Problems

Common Algorithm Design Paradigms

Mathematical Tools to Analyze Algorithms

Intractable Problems

• MergeSort• Strassen’s MM• BFS/DFS• Dijkstra’s SSSP• Kosaraju’s SCC• Kruskal’s MST• Floyd Marshall APSP• Karger’s Min-Cut• Quicksort• Bellman-Ford• …

• Big-oh notation• Recursion Tree• Master method• Substitution method• Exchange Arguments

•P vs NP•Approximation Algs.

• Divide-and-Conquer• Greedy• Dynamic Programming

Other• Data Structures

Hash Functions/Tables• Parallel Algorithms

Course RoadMap

4

Outline For Today

1. Karger’s MinCut

2. Approximate Max-Cut

5

Min Cut Problem

Input: Undirected G(V, E)

not necessarily simple, i.e., parallel (u, v) edges are

OK.

Output: cut (X, Y) with min # of crossing edges

Intuitively a measure of how-well connected the graph

is

i.e., how robust the graph is to edge removals

Applications:

transportation networks

clustering

graph partitioning

image segmentation

6

Example Cut

A

B C

D

E

7

Example Cut

A

B C

D

E

Cut Size: 4

8

Example Min Cut

A

B C

D

E

9

Example Min Cut

A

B C

D

E

Min Cut Size: 2

10

Observation 1

Claim: |min-cut| ≤ min-degree k in the graph

Proof: Let v be s.t deg(v) = k

then ({v}, V – {v}) has k edges crossing it

v

…

V-{v}

|({v}, V-{v})| = k

deg(v) = k

11

CorollaryIf min-cut has size k:Q1: what’s a bound on the degrees of vertices?A: ≥ kQ2: what’s a bound on m=|E|?

A:

v

…V-{v}

|({v}, V-{v})| = k

deg(v) = k

12

Karger’s Algorithm Pseudocode (1993) procedure Karger(G(V,E)):

while |V| > 2:pick an edge (u, v) uniformly at

randommerge u, v into a single node uvremove edges between u, v

return (X,Y) represented by last 2 vertices

13

Karger’s Algorithm Simulation 1

A

B C

D

E

G1=(m1=m, n)

14


A

B C

D

E

G1=(m1=m, n)

15


AB C

D

E

G2=(m2, n-1)

16


AB C

D

E

G2=(m2, n-1)

17


ABC

D

E

G3=(m3, n-2)

18


ABC

D

E

G3=(m3, n-2)

19


ABC

DE

Output Cut: ({A, B, C}, {D, E})

G4=(m4, n-3)

Q: What’s the size of the cut?A: m4=3

Observation: Edges crossing two last 2 (super)nodes are exactly the edges crossing the cut Karger outputs!

20


A

B C

D

E

Output Cut: ({A, B, C}, {D, E}), size: m4=3

21


A

B C

D

E

Output Cut: ({A, B, C}, {D, E}), size: m4=3Incorrect: Not Minimum-Cut

22


A

B C

D

E

G1=(m1=m, n)

23


A

B C

D

E

G1=(m1=m, n)

24


AC

B

D

E

G2=(m2`, n-1)

25


AC

B

D

E

G2=(m2`, n-1)

26


ACD

B

E

G3=(m3`, n-2)

27


ACD

B

E

G3=(m3`, n-2)

28


ACDE

B

Output Cut: ({A, C, D, E}, {B}), size: m4`=2

G4=(m4`, n-2)

29


A

B C

D

E

Output Cut: ({A, C, D, E}, {B}), size: m4`=2

30


A

B C

D

E

Output Cut: ({A, C, D, E}, {B}), size: m4`=2Correct: Minimum Cut

31

Fundamental Question

How often does Karger return a min cut?

OR: What’s the probability that Karger returns

a min-cut?

Weaker Question: Let (X, Y) be a min-cut?

What’s the probability Karger returns (X, Y).

32

Warm-up Question 1

Assume (X, Y) is a min cut and|(X, Y)| = k.

Assume first iteration Karger picks a (u, v) crossing (X, Y).

Q: Can we conclude whether Karger succeeds or fails at

the end?

A: Karger fails to return (X, Y).

Why?

=> (u, v) crossing implies u ∈ X, and v ∈ Y (or vice versa)

=> u and v will be merged into a supernode

u and v will be on the same side of the final cut output

the final cut will NOT be (X, Y)

33

Warm-up Question 2

Assume the min cut is (X, Y) and |(X, Y)| = k.

Assume in any iteration Karger picks (u, v) ∈ (X, Y)

crossing edges.

Q: Can we conclude whether Karger succeeds or fails at

the end?

A: Karger fails.

Why? => Same argument

34

Converse is Also True

Assume the min cut is (X, Y) and |(X, Y)| = k.

If Karger never picks a (u, v) crossing (X, Y)

All contractions were internal to X and internal to Y

At most |X|-1 possible contractions internal to X

At most (|Y|-1) possible contractions internal to Y

Since|X|-1+|Y|-1 is exactly n-2, all of X and Y must have

been contracted into supernodes

Karger outputs exactly (X, Y)

35

Question We Are After

What’s the probability Karger never picks in

any of the k edges crossing (X, Y)?

36

Key Events Si

Let Si be the event that one of the k (X, Y) edges is

contracted in iteration i.

**We’re after: Pr(¬S1∩¬S2∩…∩¬Sn-2)**

Q: Pr(S1)?

A:

Since we know that m ≥ kn/2

Pr(S1)

Pr(¬S1)Next Question: Pr(¬S1∩¬S2)

37

Probability Review: Conditional ProbabilityLet S, T be two events

S T

S∩T

Ex: Pr(roll > 4|roll>3):?

38

Pr(¬S1∩¬S2)=Pr(¬S1)Pr(¬S2|¬S1)

Recall after contracting one edge we have

G2=(m2, n-1).

Pr(S2|¬S1) =

Question: What do we know about the degrees of

nodes/super-nodes in the Gi?

Answer: ≥ k

Why?

Every cut in the contracted graph induces a cut in

the original graph.

39

Cuts in Contracted GraphA

B C

D

E

ACD

B

E

G1

G3

40

Cuts in Contracted GraphA

B C

D

E

ACD

B

E

G1

G3

41

Number of edges in G2

Degree of every vertex in contracted graph ≥ k

In general:

42

Pr(¬S1∩¬S2)=Pr(¬S1)Pr(¬S2|¬S1)

Pr(S2|¬S1) =

Pr(¬S2|¬S1)

Pr(¬S2∩¬S1) = Pr(¬S1)Pr(¬S2|¬S1)

Next Question: Pr(¬S1∩¬S2∩¬S3)

43

Pr(¬S1∩¬S2∩¬S3)=Pr(¬S1∩¬S2)Pr(¬S3|¬S1∩¬S2)Pr(S3|¬S1∩¬S2) =

Pr(¬S3|¬S1∩¬S2)

Pr(¬S1∩¬S2∩¬S3) = Pr(¬S1∩¬S2)Pr(¬S3|¬S1∩¬S2)

44

Pr(success)=Pr(¬S1∩¬S2….∩¬Sn-2)

**Karger outputs (X, Y) with

probability at least 1/n2**

45

Pr(success) Looks Very Small!

n=100

n=106

But we can easily boost this probability!

46

Probability Recap: Independence

Recall notion of independent events:

By definition: S and T are independent iff

Pr(S∩T) = Pr(S)Pr(T)

Pr(S|T)=Pr(S)

Pr(T|S)=Pr(T)

47

Trick: Running Karger Over and Over Again

KargerG(V,E)

O1

KargerG(V,E)

O2

KargerG(V,E)

O3

KargerG(V,E)

ON

Return best Oj

…

48

Trick: Running Karger Over and Over AgainHope: At least one of the N runs succeeds

Ti be the event Karger returns (X, Y) in ith

iteration.

By independenceof Ti

49

What’s the Rate of Decrease?

Calculus fact: ∀x real, 1+x ≤ ex

Let N=n2

=>

Let N=n2ln(n)

=>

quite small!

quite large!

50

Let’s Boost Success Probability Further…Let N=2n2ln(n)

=>

very small!

very large!

Let N=4n2ln(n)

=>

very very

small!

very very

large!

51

Definition: With High Probability (w.h.p)

We say an algorithm succeeds with high

probability if

Pr(success) ≥ (1-1/n) probability.

Summary

1. Described Karger’s Algorithm

2. Analyzed the probability of success in one

iteration

Pr(success in one iteration) ≥ 1/n2

(polynomially) small

3. Repeated the algorithm O(n2log(n)) (poly)

times

with high probability found the min-cut

Run-time Comments

O(n2log(n)) iterations

Each iteration at least O(m) times

at least O(mn2log(n))

A simple extension (Karger/Stein): O(n2log3(n)),

which is almost linear when m is O(n2)

Interesting Consequence of Our Analysis We showed that the prob. of Karger

outputting (X, Y) cut is ≥ 1/n2

Q: What can we conclude about the

number of distinct min-cuts?

A: ≤ n2

55

Given a fixed input, they may give:

1. Different (possibly wrong) outputs (Monte Carlo

Algs)

2. Different runtimes (possibly exponential) (Las

Vegas Algs)

depending on the outcomes of the coins

Final Comments (2)

Randomized AlgorithmInput Outpu

t

flip coins

Randomized Algorithms

56

Outline For Today

1. Karger’s MinCut

2. Approximate Max-Cut

57

Max Cut Problem

Input: Undirected G(V, E)

not necessarily simple, i.e., parallel (u, v) edges are

OK.

Output: cut (X, Y) with max # of crossing edges

Fact: Unlike Min-Cut, Max-Cut is Intractable (NP-

complete)

=> can’t hope to solve it exactly within reasonable

(poly-time) amount of time

58

Simple Randomized Algorithm

procedure Approximate-MaxCut(G(V,E)): let L, R be 2 empty sets

for each vertex v: toss a fair coin if heads: put v into L if tails: put v into R

59

Worst & Best Scenarios

Worst Scenario:

every vertex goes to L or R, so 0 edges cross (L, R)

Best Scenario:

let (X, Y) be a max-cut

algorithm returns (X, Y)

Let Z be the # edges crossing (L, R)

Q:What’s E[Z]?

60

E[Z]

Let Z be # edges crossing (L, R)

∀e ∈ E, Let Xe be an indicator random variable, s.t.

Xe = 1 if e crosses (L, R)

Xe = 0 o.w.

Consider e=(u,v). What’s

E[Xe]?

61

E[Xe]=Pr(Xe = 1)

vu

L R

v

u

L R

v

u

L R

uv

L R

E[Xe]=Pr(Xe = 1)=1/2

62

Back to E[Z]

All cuts have size ≤ m, so this is always within a factor of 2 of optimal!

=> we have randomized 1/2-approximation algorithm to max-cut.

Key Takeaway:Solving max-cut exactly is impossible but we have a

very simple linear time approximate randomized algorithm!

Interesting Consequence of Our AnalysisClaim: Every graph G(V, E) has a cut of size m/2!

Why?

Since the expected number of edges of our

algorithm is m/2, there has to be at least one cut

with ≥ m/2.

Problem With Expectations

Expectation of a R.V. does not give enough

information about the probability that the R.V.

takes a value around its expectation.

Different RV With Same Expectations (1)

100 150 200500

50%

50%

0.5*50 + 0.5*150 = 100

Probability Distribution of RV


100 150 200500

50%

50%

0.5* + 0.5*200 = 100



100 150 200500

100%

1*100 = 100


Expected Analysis of Previous Algorithms

Q2: We know **on expectation** Approximate-

Max-Cut outputs a large cut. But what’s the

probability that it actually outputs a large cut!

Q1: We know **on expectation** Quicksort takes

≤2nln(n) time. What’s the probability that it takes

~2nln(n) time.

Markov’s Inequality

For any random variable X, c > 1

Pr(X ≥ cE[X]) ≤ 1/c

Pr(QuickSort takes ≥ 2(2nlnn)) ≤ 1/2

Pr(QuickSort takes ≥ 8nlnn) ≤ 1/4

=>Pr(QuickSort takes ≥ 200nlnn)) ≤ 1/100

There are other techniques you can use to show

that Quicksort takes time around its expectation

with much higher probability

Analyzing Approximate Max-Cut

Q2: What’s the probability that Approximate Maxcut

outputs a large cut!

Recall Z = size of the cut output by Approximate

Maxcut

E[Z] = m/2

Let Y = m – Z

E[Y] = m – E[Z] = m/2

By Markov: Pr(Y ≥ 1.5(m/2))=Pr(Y > 3m/4) ≤

1/1.5=2/3

Pr(Z ≤ m/4) ≤ 2/3

=> with 33% probability Approx. Max-Cut outputs

0.25-approximation to the maximum cut

Boosting Success Probability With Repeats

Approx-MCG(V,E)

O1

Approx-MCG(V,E)

O2

Approx-MCG(V,E)

O3

Approx-MCG(V,E)

ON

Return best Oj

…

Boosting Success Probability With RepeatsLet Z1, Z2, …, ZN be the outputs of each independent

trial.

Pr(Zi ≤ m/4) ≤ 2/3

Pr(return a small cut) = Pr(Z1≤m/4∩…∩ZN≤m/4) ≤

(2/3)N

Pick N=log3/2(n)=O(log(n))

Pr(return a small cut) ≤ 1/n

Pr(retuning a large cut) Pr(max(Zi) ≥ m/4) ≥

(1-1/n)

Repeating Approx. MaxCut O(logn) times

returns w.h.p a 0.25-approximation to the

maximum cut!

Documents

Karger’s Min-Cut Approximate Max-Cut Monday, July 21st 1