Zig-Zag Expanders Zig-Zag Expanders Seminar in Theory and Seminar in Theory and Algorithmic ResearchAlgorithmic Research

Sashka DavisSashka DavisUCSD, April 2005UCSD, April 2005

“ “ Entropy Waves, the Zig-Zag Graph Product, and New Entropy Waves, the Zig-Zag Graph Product, and New Constant-Degree Expanders”Constant-Degree Expanders”

O. Reingold, S. Vadhan, A. WigdersonO. Reingold, S. Vadhan, A. Wigderson

Talk OutlineTalk Outline

Introduction: notations, definitions, Introduction: notations, definitions, facts.facts.

Zig-Zag graph product: Zig-Zag graph product:

1.1. OverviewOverview

2.2. ConstructionConstruction

3.3. Analysis – IntuitionAnalysis – Intuition

4.4. AnalysisAnalysis

Expansion, ExpandersExpansion, Expanders

For undirected graph G=(V,E)For undirected graph G=(V,E)

Vertex expansion parameter Vertex expansion parameter is defined is defined as: as: εε = min = min ||ΓΓ(S)\S| / |S|.(S)\S| / |S|.

S | |S| S | |S| ≤|V|/2≤|V|/2

G is a good expander if for any S, s.t.G is a good expander if for any S, s.t.

|S| |S| ≤|V|/2, then |≤|V|/2, then |ΓΓ(S)|≥(1+(S)|≥(1+εε) |S|.) |S|.

Family of Expander GraphsFamily of Expander Graphs

A family of expander graphs {GA family of expander graphs {Gii} is a } is a collection of graphs such that collection of graphs such that for all for all i:i:

GGii is d-regular. is d-regular. |V(|V(GGii)| is strictly increasing.)| is strictly increasing. εε ≥c ≥c, for some constant c., for some constant c.

Undirected Undirected DD-regular -regular GraphsGraphs

Notation:Notation: Let G be undirected D-regular, then: Let G be undirected D-regular, then: the adjacency matrix is A(G).the adjacency matrix is A(G). the normalized adjacency matrix is M= 1/D A(G).the normalized adjacency matrix is M= 1/D A(G). Spectrum Spectrum σσ(A)={(A)={λλ00,,λλ11,…,,…,λλn-1n-1}.}. λλ(G)= (G)= λλ11..

1.1. Each row/column adds up to DEach row/column adds up to D2.2. A(G) is (real) symmetric, thereforeA(G) is (real) symmetric, therefore

A(G) is similar to a diagonal matrix.A(G) is similar to a diagonal matrix. σσ(A)={(A)={λλ00,,λλ11,…,,…,λλn-1n-1} are real.} are real. RRⁿ ⁿ has an orthonormal basis consisting of has an orthonormal basis consisting of

eigenvectors of A(G).eigenvectors of A(G). (D,1(D,1nn) is an eigenpair.) is an eigenpair.

Expansion, Convergence, and Expansion, Convergence, and λλ(G) (G)

G is a good expander then G is a good expander then λλ(G) is small(G) is smallCheeger & Buser: (d-Cheeger & Buser: (d-λλ11)/2D ≤ e ≤ 2√)/2D ≤ e ≤ 2√(d-(d-λλ11)/D )/D

Random walk on G converges to the uniform Random walk on G converges to the uniform distribution rapidly if distribution rapidly if λλ(G) is small.(G) is small. Proof: (on board)Proof: (on board)

We use Rayleligh-Ritz TheoremWe use Rayleligh-Ritz Theoremλλ(G) = max <Mx,x>/<x,x> = max ||Mx||/||x||(G) = max <Mx,x>/<x,x> = max ||Mx||/||x|| x perp. to uniformx perp. to uniform

Talk OutlineTalk Outline

1.1. Introduction: notations, Introduction: notations, definitions, facts.definitions, facts.

Zig-Zag Graph product: Zig-Zag Graph product:

1.1. OverviewOverview

2.2. ConstructionConstruction

3.3. Analysis – IntuitionAnalysis – Intuition

4.4. AnalysisAnalysis

Zig-Zag Graph ProductZig-Zag Graph Product

Delivers a Delivers a constant degree constant degree family of family of expanders.expanders.

Construction is Construction is iterativeiterative.. The analysis is The analysis is algebraicalgebraic..

Notation: G is (N,D,Notation: G is (N,D,μμ)-graph meaning )-graph meaning V(G)=N, G is D-regular and has V(G)=N, G is D-regular and has λλ(G) at (G) at most most μμ..

Standard OperationsStandard Operations

Squaring G:Squaring G: new edge are new edge are paths in paths in GG of length 2 of length 2

(N,D,(N,D,λλ))22 = (N,D = (N,D22,,λλ22))

Tensoring G (Kronecker product)Tensoring G (Kronecker product) (N,D,(N,D,λλ) ) (N,D,(N,D,λλ) = (N) = (N22,D,D22,,λλ))

Expander Construction Expander Construction Using the Zig-Zag Graph Using the Zig-Zag Graph

productproduct Start with a Start with a constant-sizeconstant-size expander expander H.H.

Apply simple operations to Apply simple operations to HH to construct to construct arbitrarily large expanders.arbitrarily large expanders.

Main Challenge: Main Challenge: prevent the degree from growing. prevent the degree from growing.

New Graph Product: New Graph Product: compose large graph w/ small compose large graph w/ small graph to obtain a new graph which (roughly) graph to obtain a new graph which (roughly) inheritsinherits Size of largeSize of large graph graph Degree of smallDegree of small graph graph Expansion from bothExpansion from both

The Zig-Zag Graph Product:The Zig-Zag Graph Product:

Theorem 1Theorem 1 Let Let GG1 1 be (N,D,be (N,D,λλ11)-)-graphgraph and and

GG2 2 bebe ((D,D,d,d,λλ22)-graph, then)-graph, then

((GG1 1 GG22) = (ND, ) = (ND, dd22, , λλ11 + + λλ22+ + λλ2222))

ProofProof: Later. (Big portion of remaining : Later. (Big portion of remaining 23 slides...)23 slides...)

z

Talk OutlineTalk Outline

1.1. Introduction: notations, Introduction: notations, definitions, facts.definitions, facts.

Zig-Zag graph product: Zig-Zag graph product:

1.1. OverviewOverview

2.2. ConstructionConstruction

3.3. Analysis – IntuitionAnalysis – Intuition

4.4. Analysis (all the gory details..)Analysis (all the gory details..)

The ConstructionThe Construction Building block: Building block: Let Let H H be be (D(D44,D,1/5,D,1/5))--

graphgraph

ConstructConstruct a family a family {{GGii}} ofof DD22-regular-regular graphs such thatgraphs such that GG11=H=H22

GGi+1i+1= (G= (Gii))2 2 HH

Theorem 2Theorem 2 For every i, For every i, GGii is (D is (D4i4i, D, D22, 2/5)-, 2/5)-graph.graph.

ProofProof: By induction (on the board).: By induction (on the board).

z

Zig-Zag Graph Product – Zig-Zag Graph Product – Construction (by example)Construction (by example)

Vertices in V(Vertices in V(GG1 1 G G22) = V() = V(GG11)) V(GV(G22))

uG1

z

G21 2

3

1 2

3

1 2

3

1 2

3

1 2

3

v

1 2

3

1 2

3

Zig-Zag Graph Product – Zig-Zag Graph Product – Construction (by example)Construction (by example)

Vertices in Vertices in GG1 1 G G22 = = GG11GG22z

(v,1)

(u,1)

(u,3)

(u,2)

(v,2)

(v,3)

Edge of Edge of GG1 1 G G22 = V = VEE22EE22

(v,1)

(u,1)

(u,3)

(u,2)

(v,2)

(v,3)1 3

2

1 2

30

10

0

0

0

0

11

1

11

1

2

33

Consider ((u,1),0,0) - edge(0,0) incident to vertex (u,1).

z

2

Edges of Edges of GG1 1 G G22z

(v,1)

(u,1)

(u,3)

(u,2)

(v,2)

(v,3)3

0

10

0

0

0

0

11

1

11

1

2

33

Vertex (u,1) and all its neighbors.

10

1

1 2

3

0

0

0

1

1

1

(w,1)

1

Edges of Edges of GG1 1 G G22 = V = VEE22EE22z

(v,1)

(u,1)

(u,3)

(u,2)

(v,2)

(v,3)3

1 2

30

10

0

00

01

1

11

1

1

23

3

Connect (u,i) and (v, j) iff i, j such that

1. i and i connected in G2

2. (u, i ) and (v, j ) correspond to same edge of G1

3. j and j connected in G2

G = GG = G1 1 GG22

GG11 is (N,D, is (N,D,λλ11)-graph and G)-graph and G22 is (D,d, is (D,d,λλ2 2 )-graph)-graph

|V(G)| = |V(|V(G)| = |V(GG11)||V()||V(GG22)| = ND)| = ND

Degree of G = deg(Degree of G = deg(GG22))22=d=d22

Edge set of G: Edge set of G: a step in a step in GG22 a step in a step in GG11 a step in a step in GG22

λλ(G) ≤ (G) ≤ λλ1 1 ++ λλ2 2 ++λλ2222

z

ANALYSIS of the ZIG-ZAG ANALYSIS of the ZIG-ZAG Graph ProductGraph Product

IntuitionIntuition

The Eigenvalue BoundThe Eigenvalue Bound

Need to show: Need to show: Random step on Random step on GG1 1

GG22 makes non-uniform probability makes non-uniform probability distributions distributions closer to uniform. closer to uniform.

Random step on Random step on GG11 GG22 1. random step within “cloud”.1. random step within “cloud”. 2. jump between clouds. 2. jump between clouds. 3. random step within new cloud.3. random step within new cloud.

z

z

Analysis, Intuition (cont.)Analysis, Intuition (cont.)A,C – normalized adjacency matrices of GA,C – normalized adjacency matrices of G11,G,G22

M – normalized adjacency matrix of GM – normalized adjacency matrix of G

Must show: Must show: GG1 1 GG2-matrix 2-matrix MM shrinks every shrinks every vector vector ααNDND that is perp. to uniform that is perp. to uniform (Rayeigh-Ritz Thm, for 2-nd eigenvalue).(Rayeigh-Ritz Thm, for 2-nd eigenvalue).

Decompose Decompose αα==αα||||+ + αα, where , where αα|||| is probability is probability distribution, where distribution within clouds distribution, where distribution within clouds is uniform, and is uniform, and αα is a distribution, where is a distribution, where probabilities within cloud are far from probabilities within cloud are far from uniform.uniform.

z

Case I: Non-uniform Case I: Non-uniform DistributionDistribution

–Case I: α very non-uniform (far from (far from uniformuniform) within “clouds”

•Step 1 makes α more uniform (by expansion of G2 ).

•Steps 2 & 3 cannot make α less uniform.

Case II: Uniform DistributionCase II: Uniform DistributionCase II:Case II: αα uniform within clouds. uniform within clouds. Step 1: does not change Step 1: does not change αα.. Step 2: Jump between clouds Step 2: Jump between clouds

random step on random step on GG11

Distribution on clouds themselves Distribution on clouds themselves becomes more uniform (by expansion becomes more uniform (by expansion of of GG11))

Analysis of Analysis of λλ(G)(G)

To show that To show that λλ(G) ≤ ((G) ≤ (λλ(G(G11) + ) + λλ(G(G22)+)+λλ(G(G22))22) )

suffices to prove that to show that for any suffices to prove that to show that for any

ααNDND, perpendicular to , perpendicular to 11NDND

<<MMαα,,αα> > ≤ (≤ (λλ(G(G11) + ) + λλ(G(G22)+)+λλ(G(G22))22) ) <<αα,,αα > >

<<MMαα,,αα> > ≤ (≤ (λλ11++λλ22++λλ2222))<<αα,,αα > >

Normalized Adj. Matrix of the Normalized Adj. Matrix of the ProductProduct

A,C – normalized adjacency matrices of GA,C – normalized adjacency matrices of G11,G,G22

M – normalized adjacency matrix of M – normalized adjacency matrix of GG1 1 GG22

M=M=ĈÂĈ, where ĈÂĈ, where Ĉ = IĈ = INN C C  is a permutation matrix (length preserving), where element  is a permutation matrix (length preserving), where element

(u,v) goes to the v-th neighbor of v in G(u,v) goes to the v-th neighbor of v in G11.. We relate  to A next:We relate  to A next:

z

 is.. is..

Given any Given any ααNDND,, αα==αα1111, …,, …,αα1D1D,…, ,…, ααN1N1, …,, …,ααNDND

For iFor i[N], define:[N], define: ((αα))i i DD, , ((αα))ii== αα1111, …,, …,αα1D1D distribution within the cloud. distribution within the cloud.

ββii==∑∑j=1,Dj=1,D ααij ij “distribution” on clouds themselves.“distribution” on clouds themselves.

((αα))ii||||= (= (ββii/D) 1/D) 1DD

((αα))ii

= = ((αα))ii-(-(αα))ii||||

L: L: NDND → →NN, , L( L(αα) = () = (ββ11,…, ,…, ββNN)= )= β β NN

LLÂ(Â(ββ 1 1DD))= A))= Aββ

Â(Â(ββ 1 1DD) = A) = Aββ 1 1DD

Proof (cont.)Proof (cont.)

1.1. <<MMαα,,αα> = <> = <ĈÂĈĈÂĈαα,,αα> = > = ααT T ĈÂĈĈÂĈ αα = =

(Ĉ(Ĉαα))TT Â(Ĉ Â(Ĉ αα) = ) = <<ÂĈÂĈαα, , ĈĈαα>>

2.2. αα = = αα|||| ++ αα

3.3. ĈĈαα|| || = = αα||||

4.4. ĈĈαα = = Ĉ( Ĉ(αα++αα||||) = ) = αα|| || ++ ĈĈαα

Proof (cont.)Proof (cont.)ĈĈαα = = Ĉ( Ĉ(αα + + αα||||) = ) = αα|| || ++ ĈĈαα

<<MMαα,,αα>=|<>=|<ÂĈÂĈαα,,ĈĈαα>|=<>|=<Â (Â (αα||||++ĈĈαα)),(,(αα||||++ĈĈαα))>>

= = <<ÂÂαα||||,,αα||||>+<>+<ÂÂαα||||,Ĉ,Ĉαα>+<>+<ÂĈÂĈαα,,αα||||>+<>+<ÂĈÂĈαα,, ĈĈαα>>

≤≤<<ÂÂαα||||, , αα||||>+>+||Â||Âαα||||||||Ĉ||||Ĉαα||||++||ÂĈ||ÂĈαα|||| |||| αα||||||||+<+<ÂĈÂĈαα,, ĈĈαα>>

== <<ÂÂαα|| || ,,αα||||>>+ 2||+ 2||αα||||||.||.||||ĈĈαα|| || + + ||||ĈĈαα||||22

Proof (cont.)Proof (cont.)<<MMαα,,αα>>≤≤ ||<<ÂÂαα|| || ,,αα||||>>|| + 2||+ 2||αα||||||.||.||||ĈĈαα|| || + + ||||ĈĈαα||||22

Claim1: Claim1: ||||ĈĈαα|| || ≤ ≤ λλ(G(G22)||)||αα|| = || = λλ22||||αα||||

Claim2: Claim2: <<ÂÂαα|| || ,,αα|||| >> ≤≤ λλ11<<αα|| || ,,αα||||>=>=λλ11||||αα||||22

αα|| || = = β β U UDD

ÂÂαα|| || = = Â(Â(ββ U UDD) = A) = Aββ U UDD

By By expansion of Gexpansion of G2 2 - A - Aββ ≤≤ λλ1 1 ββ

<<ÂÂαα|| || ,,αα|||| > = <> = <AAββUUD D ,,β β U UD D > = < > = < λλ1 1 ββUUDD , ,ββ U UDD > >

<<ÂÂαα|| || ,,αα|||| > > ≤≤ λλ11<<αα|| || ,,αα||||>= >= λλ11||||αα||||22

Proof.Proof.

<<MMαα,,αα>=<>=<ÂĈÂĈαα,,ĈĈαα> > ≤ ≤ λλ11||||αα||||||||22+2+2λλ22 ||||αα||||||.||.||||αα||+ ||+ λλ2222.||.||αα||||22

||||αα||||22=||=||αα + + αα|||||| || 22 =|| =||αα||||22 +|| +||αα||||||||22

<<MMαα,,αα> /<> /<αα,,αα> = <> = <MMαα,,αα> / ||> / ||αα||||22

==λλ11||||αα||||||||22/||/||αα||||22 + 2 + 2λλ22||||αα||||||.||.||||αα|| || /||/||αα||||22 + + λλ2222||||αα||||2 2 /||/||αα||||22

<<MMαα,,αα>/<>/<αα,,αα> > ≤ ≤ λλ11++λλ22+ + λλ2222

Q.E.D.Q.E.D.

The Zig-Zag Graph Product:The Zig-Zag Graph Product:

Theorem 1Theorem 1 Let Let GG1 1 be (N,D,be (N,D,λλ11)-)-graphgraph and and

GG2 2 bebe ((D,D,d,d,λλ22)-graph, then)-graph, then

((GG1 1 GG22) = (ND, ) = (ND, dd22, , λλ11 + + λλ22+ + λλ2222).).

Theorem 2Theorem 2 For every i, For every i, GGii is (D is (D4i4i, D, D22, ,

2/5)-graph.2/5)-graph.

z