Community detection in complex...

Community detection incomplex networks

Vinh Loc DAO

Summary

1 Introduction

2 Datasets and Benchmarks

3 Community detection in network

4 Evaluating partition quality

5 Objectives and Perspectives

2/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Summary

1 Introduction

What is a network or a graph ?

Some notions

Structure properties of real networks

2 Datasets and Benchmarks

3 Community detection in network

4 Evaluating partition quality

5 Objectives and Perspectives

3/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

What is a network or a graph ?

Example : Internet, transport network, power grid, food web, social network

Figure – A very simple network G(V ,E) with |V | Vertices and |E | Edges

Node : Entity in real life

Edge : Relation between two entities to which it connects

A natural language to describe complex systems.

4/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

A local mesh network - very small network

Node : Computer

Edge : Connection between two computer

5/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Student network in a university - relatively small network

Node : Student

Edge : Relation (Could be any kind of relation)

6/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

The French highway network - large network

Node : City

Edge : Highway

7/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

The Internet - international network

8/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Some notions

Complex network : Graph with non-trivial topology features

Network analysis : Studies of graph to extract non-trivial features

Community detection algorithm : Divide nodes into groups calledcommunities whose members are connected densely.

Figure – Uncover graph modules without specifying clusters’ size

9/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Structure properties of real networks

- Graph representing real systems are normaly neither regular nor random.- Degree (nb of connections of a node) distribution often follows a power law,as connections often follow preferential patterns.- Nodes are often found to cluster into high density groups.

Figure – Regular lattice graph Figure – Graph with modular structure

10/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Why community detection ?

- Comprehend network global organization- Reveal modular structures- Reveal hidden properties between nodes- Understand information diffusion process throughout network

APPLICATIONS :- Detect web clients with similar interests- Prevent epidemic transmission- Managing collaboration network- etc,- You name it

11/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Summary

1 Introduction

2 Datasets and Benchmarks

Datasets - Real networks

Datasets - Artificial networks

3 Community detection in network

4 Evaluating partition quality

5 Objectives and Perspectives

12/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Datasets - Real networks

- Social network of friendships between 34 members of a karate club at a USuniversity in the 1970s. (left)- An identity graph with 25 vertices and 31 edges. An identity graph has asingle graph auto-morphism, the trivial one. (right)

Figure – Zachary karate network Figure – Walther network

13/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Datasets - Artificial networks

Parameters : Graph size, node distribution, link distribution, densitydistribution, etc.

Figure – GN benchmark network Figure – LFR benchmark network

14/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Summary

1 Introduction

2 Datasets and Benchmarks

3 Community detection in network

Dense structure in modular network

From dense structure to community

4 Evaluating partition quality

5 Objectives and Perspectives

15/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

An example of graph partitioning by K-means

Apply K-means graph partitioning on karate club network (K = 2)

Figure – The Zachary karate networkFigure – K-means partition on theZachary karate club

- Good solution ?- How do we know the club has 2 communities ?- Wait, tell me again what is a community ?

16/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Did we say a ”community” ? Isn’t it a ”cluster” ?

- What is a community ? - Answer : ”I know it when I see it”- No universal accepted definition.- More edges inside the community than edges linking its vertices with the restof the network.- Many detection methods : overlapping/non-overlapping, fast/slow,single-scale/multi-scale.

Figure – A graph division

Figure – The karate club is separateddue to a conflict coach/president

17/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Summary

1 Introduction

2 Datasets and Benchmarks

3 Community detection in network

4 Evaluating partition quality

5 Objectives and Perspectives

18/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Partition quality

- What is a good partition of a network into modules ?

- Quality function : assigns score to each partition of a graph- The most popular quality function is modularity

Q = 12|V |Σij(Aij − Pij)δ(Ci ,Cj)

Aij : Adjacency matrix, Pij : Expected connection adjacency matrixQ = Fraction of edges within communities - expected fraction of such edgesin a randomModularity favors inter community links.

19/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Community detection on the karate club

Edge betweenness O(|V ||E |2)

Fast greedy O(|V ||E |log(|V |))

Label propagation O(|V |+ |E |)Louvain method O(|V |log(|V |)

Leading eigenvector O(|V |2 + |E |)Modular optimization (NP-complete)

Infomap O (|V |(|V |+ |E |))

Walktrap O(|E ||V |2)

20/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Community detection on the Walther network

Many possible meaning divisions on a less modular network.

21/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Quality is relative ? Goodness is subjective

- Community characteristics :

Community density

Community connectiveness

Robustness to perturbation, etc

Question : How to choose appropriate method to satisfy certaincharacteristics and utilizing as much as possible available information ?

Ex : Which method to chose to :

- Divide students into the most cohesive groups.

- Establish geographic sites to minimize remote works of collaborators.

- Compromise between community density and calculation time.

- Maximize range of ages in a dancing community.

22/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Summary

1 Introduction

2 Datasets and Benchmarks

3 Community detection in network

4 Evaluating partition quality

5 Objectives and Perspectives

23/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Objectives and Perspectives

Guide to choose best methods base on expected quality indicators, graphcharacteristics and available resources

Create generative model to summarize community characteristics

Create a benchmark base on generative model

Construct criteria for evaluating partition quality base on end user pointof view

Propose methods to improve detection quality

Tools :- R for network analyzing, data analyzing, visualization- Gephi : Visualization

24/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks