Algebraic Structures and Algorithmsfor Matching and Matroid Problems

Nick Harvey

Perfect Matching

• Given graph G=(V,E)

• Perfect matching is M⊆E such that each vertex incident with exactlyone edge in M

Edmonds ’65 Blossom Alg. O(n2 m)

Micali-Vazirani ’80-’90 Blossoms + “Blocking Flow” O(√n m)

Mucha-Sankowski ’04 Fast Matrix Multiplication O(n)

Matching History

• All are non-trivial• Mucha-Sankowski uses sophisticated

dynamic connectivity data structures (Holm et al. ’01)

to maintain “canonical partition” (Cheriyan ’97)

O(n4)

O(n2.5)

(n = # vertices, m = # edges)• All are non-trivial!• Mucha ’05: “[Our] algorithm is quite complicated and

heavily relies on graph theoretic results and techniques. It would be nice to have a strictly algebraic, and possibly simpler, matching algorithm”.

O(n2.38)

(<2.38 is exponent for matrix multiplication)

Dense Graphs

m=n2

Edmonds ’65 Blossom Alg. O(n4)

Micali-Vazirani ’80-’90 Blossoms + “Blocking Flow” O(n2.5)

Mucha-Sankowski ’04 Fast Matrix Multiplication O(n2.38)

This Work Purely Algebraic,Clean Divide-and-Conquer,Fast Matrix Multiplication

O(n2.38)

Matching Algorithms

• Conceptually simple

• Implemented in ~200 lines of MATLAB

*

* Randomized, but Las Vegas via [Cheriyan ’97]

*

Dense Graphs

m=n2

(Linear) Matroid Intersection

1 1 1 0 1 0 1 01 0 0 1 1 1 0 10 0 1 1 0 1 1 00 1 0 1 1 0 1 1

7 6 6 3 7 1 9 09 2 9 2 4 9 6 13 3 4 7 0 4 3 31 4 8 4 2 2 5 3

• Find largest set of columns S such thatAS and BS are both linearly independent

A =

B =

AS

BS

Matroid Intersection• Find largest set of columns S such that

AS and BS are both linearly independent

• Why?

• Many problems are a special case:– Bipartite matching– Arboresence (directed spanning tree), …

• Powerful tool:– Bounded-Degree Sp. Tree [Goemans ’06]

Edmonds ’65-’70, Lawler ’75 Augmenting Paths O(mn2) oracle

Cunningham ’86 “Blocking Flows” O(mn2 log n)

Gabow-Xu ’89-’96 “Blocking Flows” & Fast Matrix Mult.

O(mn1.62)

Matroid Intersection History

(for matrices of size n x m)

• Essentially optimal:– Best known alg to compute rank takes O(mn-1)– Computing rank reduces to matroid intersection

Edmonds ’65-’70, Lawler ’75 Augmenting Paths O(mn2) oracle

Cunningham ’86 “Blocking Flows” O(mn2 log n)

Gabow-Xu ’89-’96 “Blocking Flows” & Fast Matrix Mult.

O(mn1.62)

This paper Purely Algebraic,Fast Matrix Mult.

O(mn-1)

Matroid Intersection History

*

*

*

* Assumes matroids are linear

†

† Randomized, and assumes matroids can be represented over same field

O(mn1.38)

Generic Matching Algorithm

For each e EIf e is contained in a perfect matching

Add e to solutionDelete endpoints of e

ElseDelete edge e

Generic Matching Algorithm

• How can we test this?• Randomization and linear algebra play key role

For each e EIf e is contained in a perfect matching

Add e to solutionDelete endpoints of e

ElseDelete edge e

Matching Outline

• Implementing Generic Algorithm– Tutte Matrix & Properties– Rabin-Vazirani Algorithm– Rank-1 Updates– Rabin-Vazirani with Rank-1 Updates– Our Recursive Algorithm (overview)– Our Recursive Algorithm (fast updates)

Matching & Tutte Matrix

• Let G=(V,E) be a graph

• Define variable x{u,v} {u,v}∈E

• Define a skew-symmetric matrix T s.t.

Tu,v =± x{u,v}

0

cb

a 0 -x{a,b} -x{a,c}

x{a,b} 0 -x{b,c}

x{a,c} x{b,c} 0

if {u,v}∈E

otherwise

Properties of Tutte Matrix

Lemma [Tutte’47]: G has a perfect matchingiff T is non-singular.

Lemma [RV’89]: G[ V\{u,v} ] has a perfect matching iff (T-1)u,v 0.

Properties of Tutte Matrix

Lemma [Tutte’47]: G has a perfect matchingiff T is non-singular.

Lemma [RV’89]: G[ V\{u,v} ] has a perfect matching iff (T-1)u,v 0.

v

u

(T-1)u,v 0

Properties of Tutte Matrix

Lemma [Tutte’47]: G has a perfect matchingiff T is non-singular.

Lemma [RV’89]: G[ V\{u,v} ] has a perfect matching iff (T-1)u,v 0.

(T-1)u,v 0G[ V\{u,v} ] has

perfectmatching

Properties of Tutte Matrix

Lemma [Tutte’47]: G has a perfect matchingiff T is non-singular.

Lemma [RV’89]: G[ V\{u,v} ] has a perfect matching iff (T-1)u,v 0.

(T-1)u,v 0G[ V\{u,v} ] has

perfectmatching

Properties of Tutte Matrix

Lemma [Tutte’47]: G has a perfect matchingiff T is non-singular.

Lemma [RV’89]: G[ V\{u,v} ] has a perfect matching iff (T-1)u,v 0.

(T-1)u,v 0G[ V\{u,v} ] has

perfectmatching

v

u

Properties of Tutte Matrix

Lemma [Tutte’47]: G has a perfect matchingiff T is non-singular.

Lemma [RV’89]: G[ V\{u,v} ] has a perfect matching iff (T-1)u,v 0.

Computing T-1 very slow: Contains variables!

Lemma [Lovász’79]: These results hold w.h.p. if we randomly choose values for x{u,v}’s.

Matching Algorithm

Compute T-1

For each {u,v} EIf T-1

u,v 0Add {u,v} to matchingRecurse on G[ V\{u,v} ]

(Takes O(n) time)

• Total time: O(n+1) time

• Natural question: Can we recomputeT-1 quickly in each iteration?

Rabin-Vazirani ’89

Matching Outline

– Tutte Matrix & Properties– Rabin-Vazirani Algorithm– Rank-1 Updates– Rabin-Vazirani with Rank-1 Updates– Our Recursive Algorithm (overview)– Our Recursive Algorithm (fast updates)

Rank-1 Updates

• Let T be a n x n matrix

• T + uvT is called a rank-1 update of T

1 1 1 1

1 1 1

1 1

1

1 2 3 n

1

2

3

n

∙ +

T u vT

1 2 3 n

2 4 6 2n

3 6 9 3n

n 2n3n n2

• Let T be a n x n matrix

• T + uvT is called a rank-1 update of T

• Computing rank-1 update takes O(n2) time

Rank-1 Updates

1 1 1 1

1 1 1

1 1

1

+

T uvT

Rank-1 Updates

• Let T be a n x n matrix

• T + uvT is called a rank-1 update of T

• Claim: If you modify one entry of T,then T-1 is modified by a rank-1 update.

T = T-1 =

Rank-1 Updates

• Let T be a n x n matrix

• T + uvT is called a rank-1 update of T

• Claim: If you modify one entry of T,then T-1 is modified by a rank-1 update.

• Claim: If you delete O(1) rows and columns of T then T-1 is affected by O(1) rank-1 updates.

T-1 can be updated in O(n2) time

Matching Outline

– Tutte Matrix & Properties– Rabin-Vazirani Algorithm– Rank-1 Updates– Rabin-Vazirani with Rank-1 Updates– Our Recursive Algorithm (overview)– Our Recursive Algorithm (fast updates)

RV Algorithm + Rank-1 Updates

Compute T-1

For each {u,v} EIf T-1

u,v 0

Add {u,v} to matchingUpdate T-1 (via rank-1 updates)Recurse on G[ V\{u,v} ]

• Total runtime: O(n3) time• Can updates use Fast Matrix Multiplication?

– Bipartite graphs: Yes– Non-bip. graphs: Yes, but non-trivial

Mucha-Sankowski ’04

[MS’04]

RV Algorithm + Rank-1 Updates

Compute T-1

For each {u,v} EIf T-1

u,v 0

Add {u,v} to matchingUpdate T-1 (via rank-1 updates)Recurse on G[ V\{u,v} ]

• Total runtime: O(n3) time• Can updates use Fast Matrix Multiplication?

Mucha-Sankowski ’04

New approach: An unusual recursion!

Matching Outline

– Tutte Matrix & Properties– Rabin-Vazirani Algorithm– Rank-1 Updates– Rabin-Vazirani with Rank-1 Updates– Our Recursive Algorithm (overview)– Our Recursive Algorithm (fast updates)

New Recursive Approach

(Here c=4 parts)

• Partition into c parts {V1,…,Vc} (arbitrarily)

• For each pair of parts {Va,Vb} (arbitrary

order)

– Recurse on G[Va ⋃ Vb]

New Recursive Approach

• Partition into c parts {V1,…,Vc} (arbitrarily)

• For each pair of parts {Va,Vb} (arbitrary

order)

– Recurse on G[Va ⋃ Vb]

New Recursive Approach

• Partition into c parts {V1,…,Vc} (arbitrarily)

• For each pair of parts {Va,Vb} (arbitrary

order)

– Recurse on G[Va ⋃ Vb]

New Recursive Approach

• Partition into c parts {V1,…,Vc} (arbitrarily)

• For each pair of parts {Va,Vb} (arbitrary order)

– Recurse on G[Va ⋃ Vb]

• Base case: 2 vertices

New Recursive Approach

• Partition into c parts {V1,…,Vc} (arbitrarily)

• For each pair of parts {Va,Vb} (arbitrary order)

– Recurse on G[Va ⋃ Vb]

• Base case: 2 vertices {u,v}

– If T-1u,v 0, add {u,v} to matching, update T-1

Recursion F.A.Q.• Why not just recurse on G[ Va ]?

– Edges between parts would be missed– Claim: Our recursion examines every

pair of vertices examines every edge• Why does algorithm work?

– It implements Rabin-Vazirani Algorithm!• Isn’t this horribly slow?

– No: we’ll see recurrence next

• Partition into c parts {V1,…,Vc} (arbitrarily)

• For each pair of parts {Va,Vb} (arbitrary order)

– Recurse on G[Va ⋃ Vb]

• Base case: 2 vertices {u,v}– If T-1

u,v 0, add {u,v} to matching, update T-1

Final Matching Algorithm

If base case with 2 vertices {u,v}

If T-1u,v 0, add {u,v} to matching

Else

Partition into c parts {V1,…,Vc}

For each pair {Va,Vb}

Recurse on G[Va ⋃ Vb]

Apply updates to current subproblem

s

cOs

cR

csR

2

2

2)(

s = size of subproblem

Final Matching Algorithm

If base case with 2 vertices {u,v}

If T-1u,v 0, add {u,v} to matching

Else

Partition into c parts {V1,…,Vc}

For each pair {Va,Vb}

Recurse on G[Va ⋃ Vb]

Apply updates to current subproblem

s

cOs

cR

csR

2

2

2)(

s = size of subproblem

Final Matching Algorithm

If base case with 2 vertices {u,v}

If T-1u,v 0, add {u,v} to matching

Else

Partition into c parts {V1,…,Vc}

For each pair {Va,Vb}

Recurse on G[Va ⋃ Vb]

Apply updates to current subproblem

s

cOs

cR

csR

2

2

2)(

Assume:O(s) time

s = size of subproblem

• Basic Divide-and-Conquer

– If then

• Since ,

just choose c large enough!

Time Analysis

22/

logc

c)()( nOnR

1log

12

22/log

c

cc

s

cOs

cR

csR

2

2

2)(

c = 13 is large enough if = 2.38

Handling Updates

T =

Handling Updates

T = T-1 =

• Delete vertices u and v

uv

u v

uv

u v

Handling Updates (Naively)

T = T-1 =

• Delete vertices u and v clear rows / columns

• Causes rank-1 updates to T-1

• Algorithm still takes (n3) time

uv

u v

uv

u v

Matching Outline

– Tutte Matrix & Properties– Rabin-Vazirani Algorithm– Rank-1 Updates– Rabin-Vazirani with Rank-1 Updates– Our Recursive Algorithm (overview)– Our Recursive Algorithm (fast updates)

Just-in-time Updates

T-1 =

• Don’t update entire matrix!• Just update parent in recursion tree• Updates outside of parent are postponed

uv

u v

Postponed Updates

T-1 =

• Accumulate batches of updates• Claim: New updates can be applied with matrix

multiplication and inversion

uv

u v

Final Matching Algorithm

If base case with 2 vertices {u,v}

If T-1u,v 0, add {u,v} to matching

Else

Partition into c parts {V1,…,Vc}

For each pair {Va,Vb}

Recurse on G[Va ⋃ Vb]

Apply updates to current subproblem

Invariant: Before / after child subproblem,parent’s submatrix is completely updated

in every base case, T-1u,v is up-to-date!

Graph G

children

2

c

All edges chosen in base cases

Only take edges that can be extended to a perfect matching in the whole graph.This decision is possible because invariant ensures that T-1

u,v is up-to-date.

Just apply postponed updates

Matching Summary

• Can compute a perfect matching inO(n) = O(n2.38) time

• Algorithm uses only simple randomization, linear algebra and divide-and-conquer

• Easy to implement• Extensions for: (by existing techniques)

– maximum matchings

– Las Vegas

More Extensions• Same philosophy applies to many

matching-like problems– Design matrix capturing problem– Design algorithm to test if element in OPT– Design method to efficiently do updates

x3

x4

x5x6

x7x8

A

BT

J

J

Matroid Intersection

O(mn-1) algorithm

– Design “algebraic structure” capturing problem

More Extensions• Same philosophy applies to many

matching-like problems– Design matrix capturing problem

Bipartite Matroid Matching& Basic Path-Matching

O(n) algorithm

– Design “algebraic structure” capturing problem

Open Problems

• Fast, deterministic methods to choosevalues for indeterminates?– Implications for Matching in NC

• Scaling algorithms for weighted problems?

• O(n2) algorithms for matching bycompletely different techniques?– e.g., via Karger-Levine randomized max flow