Overlapping community Detection Using Bayesian NMF

Overlapping Community detection using Bayesian NonNegative Matrix Factorization

Rajkumar SinghRishi Barua

Guide: Dr. Ashish Anand

Dept. of Computer ScienceIndian Institute of Technology, Guwahati

April 18, 2013

R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 1 / 15

Network Paradigm

V ∈ RN×N is the adjacency matrix

vij can be boolean or denote connection weight

ki =∑

j vij degree of node i

data in relational form


Network Paradigm



ki =∑




Network Paradigm



ki =∑




Network Paradigm



ki =∑




Network Paradigm



ki =∑




Commnity Detection

Q. What is Community ?As such no Specific Definition. It is Context Dependent.Defn. : Here we define communities as Modules which are subgraphs withmore links connecting the nodes inside than outside them.

A given real world network is assumed to be clustered into a numberof latent classes of nodes.

These nodes form regions of increased connectivity in the network.

These communities usually reflect functional modules that affect theoverall behavior of the system.

Examples: Friend cliques in social networks, Similar proteins in aprotein interaction network, Research groups in a scientificcollaboration network


Commnity Detection

Q. What is Community ?

As such no Specific Definition. It is Context Dependent.Defn. : Here we define communities as Modules which are subgraphs withmore links connecting the nodes inside than outside them.






Commnity Detection

Q. What is Community ?As such no Specific Definition. It is Context Dependent.

Defn. : Here we define communities as Modules which are subgraphs withmore links connecting the nodes inside than outside them.






Commnity Detection







Commnity Detection







Commnity Detection







Commnity Detection







Commnity Detection







Community Detection


Non-negative Matrix Factorzation

Decompose data matrix V to a product of two other matrices W , H undernon-negativity constraints.V∼V̂ = WH V , V̂ ∈ RF×N and W ∈ RF×K , H ∈ RK×N so that,FK + KN < FN

Non-negativity constraints avoid the problem of an ill-posed solution

They also reflect the idea of parts-based representation. V can beexpressed as additive combination of certain basis structures definedby wk , given an encoding hk

V̂ =∑k

w:khk: (1)


Poisson Model

Given an Adjacency matrix V ∈ RN×N

Assume, pairwise interactions vij is generated by a Poissondistribution with rate v̂ij .

Hence V̂∼ V

Expectation network V̂ is composed of W and H, so that V̂ = WH

Inner rank K : Unknown number of communities

wik , hkj ∈ 1, . . . ,K : The contribution of each latent community tov̂ij =

∑k wikhkj


Cost Function

posterior:

p(V |W ,H, β) = p(V |W ,H)p(W |β)p(H|β)p(β) (2)

by taking the − log of: p(V ,W ,H)p(W |β)p(H|β)p(β),we define cost as U.

U = − log p(V |W ,H)︸︷︷︸− log p(W |β)− log p(W |β)− log p(β)︸︷︷︸ (3)

U is our minimization objective under non-negativity constraints.


Cost Function

posterior:






Cost Function

posterior:






Cost Function

posterior:






Cost Function

posterior:






Cost Function

posterior:






Parameter inference

U =∑

i

∑j

[vij log

(vijv̂ij

)+ v̂ij

]+

1

2

∑k

[(∑i βkw

2ik

)+(∑

j βkh2kj

)− 2N log βk

]+∑

k

(βkbk − (ak − 1) log βk

)+ k

We find w∗, h∗,β∗ that minimize U using the gradient descentalgorithm.

The effective number of communities K is the number of non-zerocolumns of W∗ and H∗


Parameter inference

U =∑

i

∑j

[vij log

(vijv̂ij

)+ v̂ij

]+

1

2

∑k

[(∑i βkw

2ik

)+(∑

j βkh2kj

)− 2N log βk

]+∑

k


)+ k




Parameter inference

U =∑

i

∑j

[vij log

(vijv̂ij

)+ v̂ij

]+

1

2

∑k

[(∑i βkw

2ik

)+(∑

j βkh2kj

)− 2N log βk

]+∑

k


)+ k




Results

W∗,H∗ describe a bipartite network of node allocations tocommunities.

If our original adjacency matrix V is symmetric, then W∗ = HT∗ .

Each wik or hki denotes the participation strength of node i tocommunity k.

The i th row of W or column of H describes a soft-membershipdistribution of node i across communities.

Varying node participation scores allow us to describe overlapsbetween communities in a disciplined manner.


Example: 1


Example: 1


Example: 1


Example: 2

We now take a 1000X1000 Adjacency matrix V , which is very sparse.

The graph is a non-community graph, i.e. having no communitystructure.

Several of the nodes are isolated.

Solutions from EO, Louvian (as described in the paper) offer highermodularity than NMF.

NMF clearly shows that the modularity is low and the nodes have nopreference to lie in any particular community, or are non-communal.

NMF does not suffer from resolution limit of modularity (whichgroups smaller communities), as can be clearly seen.

The Output shows 1 community, with 428 isolated nodes i.e.Non-communal


Example: 2


References

1 Overlapping community detection using Bayesian non-negative matrixfactorization - I. Psorakis, S. Roberts and M.Ebden

2 Signal Processing with Adaptive Sparse Structured Representations -V.Tan and C.Fevotte

3 D.D. Lee and H.S. Seung, Nature ,401


Self Improvement

Overlapping community Detection Using Bayesian NMF