HDRF: Stream-Based Partitioning for Power-Law Graphs.€¦ · Experiments - Settings I standalone partitioner. B VGP, a software package for one-pass vertex-cut balanced graph partitioning

HDRF: Stream-Based Partitioning for Power-Law Graphs.

travel grant from

Fabio Petroni, L. Querzoni, K. Daudjee, S. Kamali, G. Iacoboni

Graphs

I large amount of real data can be represented as a graph.

VerticesEdges

G(V,E)I the problem of optimally partitioning a graph whilemaintaining load balance is important in several contexts.

CIKM 2015. October 19-23, 2015. Melbourne, Australia. 2 of 22

Distributed Graph Computing Frameworks

I support e�cient computation on really large graphs.B graph analytics; data mining and machine learning tasks.

I input data partitioning can have a significant impact on theperformance of the graph computation.B a↵ects network usage, memory occupation, synchronization.


Balanced Graph Partitioning

vertex-cutedge-cutI partition G into smaller components of (ideally) equal size

edge-cutvertex disjoint partitions

vertex-cutedge disjoint partitions

I a vertex can be cut in multiple ways and span severalpartitions while a cut edge connects only two partitions.


Balanced Graph Partitioning

vertex-cutedge-cutI partition G into smaller components of (ideally) equal size

edge-cutvertex disjoint partitions

vertex-cutedge disjoint partitions

I a vertex can be cut in multiple ways and span severalpartitions while a cut edge connects only two partitions.


Vertex-Cut

computation steps are associated with edges

v-cut perform better on power law graphs

create less storage and network overhead

(Gonzalez et al., 2012)

I modern distributed graph computingframeworks use vertex-cut.


Power-Law GraphsI characteristic of real graphs: power-law degree distribution.

B most vertices have few connections while a few have many.

100

101

102

103

104

105

106

100 101 102 103 104 105

nu

mb

er

of

vert

ice

s

degree

crawlTwitter

connection 2010

log-log scale

I the probability that a vertex has degree d is P(d) / d

�↵ .

I ↵ controls the “skewness” of the degree distribution.


Balanced Vertex-Cut Graph PartitioningIv 2 V vertex; e 2 E edge; p 2 P partition.

IA(v) set of partitions where vertex v is replicated.

I � � 1 tolerance to load imbalance.I the size |p| of partition p is its edge cardinality.

minimize replicasreduce (1) bandwidth, (2) memory

usage and (3) synchronization

balance the loadefficient usage of available

computing resources

min1

|V |X

v2V

|A(v)| s.t. maxp2P

|p| < �|E ||P |

I the objective function is the replication factor (RF).B average number of replicas per vertex.


Streaming SettingI input data is a list of edges, consumed in streaming fashion,requiring only a single pass.

partitioningalgorithm

stream of edges

graph

3 handle graphs that don’t fit in the main memory.

3 impose minimum overhead in time.

3 scalable, easy parallel implementations.

7 assignment decision taken cannot be later changed.


Algorithms Taxonomy

Grid PDS HDRFDBH Greedy

hashing

stream-based vertex-cut graph

partitioning

Hashing

constrained partitioning

(Gonzalez et al., 2012)(Jain et al., 2013) (Petroni et al., 2015)(Xie et al., 2014)

history aware

(Jain et al., 2013)(Gonzalez et al., 2012)

history agnostic


Algorithms Taxonomy

Grid PDS HDRFDBH Greedy

hashing

stream-based vertex-cut graph

partitioning

Hashing

constrained partitioning

(Gonzalez et al., 2012)(Jain et al., 2013) (Petroni et al., 2015)(Xie et al., 2014)

history aware

(Jain et al., 2013)(Gonzalez et al., 2012)

history agnostic


HDRF: High Degree are Replicated First

I favor the replication of high-degree vertices.

I the number of high-degree vertices in power-law graphs isvery low.

I overall reduction of the replication factor.


HDRF: High Degree are Replicated First

I in the context of robustness to network failure.

I if few high-degree vertices are removed from a power-lawgraph then it is turned into a set of isolated clusters.

I focus on the locality of low-degree vertices.


The HDRF Algorithm

vertex without replicas

vertex with replicas

case 1vertices not assigned to partitions

incoming edge

e


The HDRF Algorithm



case 1place e in the least loaded partition

incoming edge

e


The HDRF Algorithm

case 2only one vertex has been assigned




incoming edge

e

e


The HDRF Algorithm

case 2place e in the partition




incoming edge

e

e


The HDRF Algorithm





incoming edge

e

e

case 3vertices assigned, common partition

e


The HDRF Algorithm





incoming edge

e

e

case 3place e in the intersection

e


Create Replicas

case 4empty intersection

e


Create Replicas

case 4least loaded partition in the union


e

estandard Greedy solution


Create Replicas

case 4least loaded partition in the union


e

case 4replicate vertex with highest degree

eδ(v1) > δ(v2)

estandard Greedy solution

HDRF


Equivalent Formulation: Maximize The Score

I computing a score C (e, p) for all partitions p 2 P .

I assigns e to the partition that maximizes C (e, p).

C (e, p) = CREP(e, p) + CBAL(p)

I the balance term breaks ties in replication term .

I this may not be enough to ensure load balance.

� control the importance of the balance term.

balanced partitions even when classical greedy fails.


Equivalent Formulation: Maximize The Score

I computing a score C (e, p) for all partitions p 2 P .

I assigns e to the partition that maximizes C (e, p).

C (e, p) = CREP(e, p) + � · CBAL(p)

the balance term breaks ties in replication term.

this may not be enough to ensure load balance.

I � controls the importance of the balance term .

I balanced partitions even when classical greedy fails.


Ordered Stream Of Edges

I without �



I without �



I without �



I without �



I without �

single partitionall the graph in a



I with �


Experiments - Settings

I standalone partitioner.B VGP, a software package for one-pass vertex-cut balanced

graph partitioning.

B measure the performance: replication and balancing.

I GraphLab.B HDRF has been integrated in GraphLab PowerGraph 2.2.

B measure the impact on the execution time of graphcomputation in a distributed graph computing frameworks.

I stream of edges in random order.


Experiments - Datasets

I real-wordgraphs.

I syntheticgraphs.

1M vertices

60M to 3M edges

Dataset |V | |E |MovieLens 10M 80.6K 10M

Netflix 497.9K 100.4MTencent Weibo 1.4M 140M

twitter-2010 41.7M 1.47B

100

101

102

103

104

105

101 102 103 104 105

nu

mb

er

of

vert

ice

s

degree

α = 1.8α = 2.2α = 2.6α = 3.0α = 3.4α = 4.0


Results - Synthetic Graphs Replication FactorI 128 partitions

4

8

1.8 2.0 2.4 2.8 3.2 3.6 4.0

rep

lica

tion

fa

cto

r

alpha

HDRFPDS |P|=133

grid |P|=121

greedyDBH

skewed homogeneousCIKM 2015. October 19-23, 2015. Melbourne, Australia. 17 of 22


4

8

1.8 2.0 2.4 2.8 3.2 3.6 4.0

rep

lica

tion

fa

cto

r

alpha

HDRFPDS |P|=133

grid |P|=121

greedyDBH

skewed homogeneous

less denseless edges

easyer to partition



4

8

1.8 2.0 2.4 2.8 3.2 3.6 4.0

rep

lica

tion

fa

cto

r

alpha

HDRFPDS |P|=133

grid |P|=121

greedyDBH

skewed homogeneous


easier to partition

more densemore edges

difficult to partition



4

8

1.8 2.0 2.4 2.8 3.2 3.6 4.0

rep

lica

tion

fa

cto

r

alpha

HDRFPDS |P|=133

grid |P|=121

greedyDBH

skewed homogeneous


easier to partition

more densemore edges

difficult to partition

exploitpower-law



4

8

1.8 2.0 2.4 2.8 3.2 3.6 4.0

rep

lica

tion

fa

cto

r

alpha

HDRFPDS |P|=133

grid |P|=121

greedyDBH

skewed homogeneousCIKM 2015. October 19-23, 2015. Melbourne, Australia. 17 of 22

Results - Real-Word Graphs Replication FactorI 133 partitions

1

2

4

8

16

Tencent Weibo Netflix MovieLens 10M twitter-2010

replic

atio

n fact

or

PDS

7.9

11.3 11.5

8.0

greedy

2.8

8.1

10.7

5.9

DBH

1.5

7.1

12.9

6.8

HDRF

1.3

4.9

9.1

4.8

1

2

4

8

16


replic

atio

n fact

or

PDSgreedy

DBHHDRF


Results - Real-Word Graphs Replication FactorI 133 partitions

1

2

4

8

16


replic

atio

n fact

or

PDS

7.9

11.3 11.5

8.0

greedy

2.8

8.1

10.7

5.9

DBH

1.5

7.1

12.9

6.8

HDRF

1.3

4.9

9.1

4.8

1

2

4

8

16


replic

atio

n fact

or

PDSgreedy

DBHHDRF

HDRF achieves a replication factor about 40% smaller than DBH,

more than 50% smaller than Greedy, almost 3× smaller than PDS,

more than 4× smaller than Grid and almost 14× smaller than Hashing.


Results - Load Relative Standard DeviationI MovieLens 10M

0

1

2

3

4

5

6

7

8 16 32 64 128 256

loa

d r

ela

tive

sta

nd

ard

de

via

tion

(%

)

partitions

PDSgridgreedyDBHHDRFhashing


Results - Graph Algorithm Runtime SpeedupISGD algorithm for collaborative filtering on Tencent Weibo.

1

1.5

2

2.5

3

32 64 128

spe

ed

up

(×)

partitions

greedyPDS

1

1.5

2

2.5

3

32 64 128

spe

ed

up

(×)

partitions

greedyPDS |P|=31,57,133

I the speedup is proportional to both:B the advantage in replication factor.B the actual network usage of the algorithm.


ConclusionI HDRF is a one-pass vertex-cut graph partitioning algorithm.

I based on a greedy vertex-cut approach that leveragesinformation on vertex degrees.

I we provide a theoretical analysis of HDRF with anaverage-case upper bound for the vertex replication factor.

I experimental study shows that:

B HDRF provides the smallest replication factor with close tooptimal load balance.

B HDRF significantly reduces the time needed to performcomputation on graphs.

I the stand-alone software package for one-pass v-cut balancedpartitioning at https://github.com/fabiopetroni/VGP.


https://github.com/fabiopetroni/VGP

Thank you!

Questions?

Fabio PetroniSapienza University of Rome, Italy

Current position:

PhD Student in Engineering in Computer Science

Research Interests:

data mining, machine learning, big data

[email protected]


[email protected]

Documents

HDRF: Stream-Based Partitioning for Power-Law Graphs.€¦ · Experiments - Settings I standalone partitioner. B VGP, a software package for one-pass vertex-cut balanced graph partitioning