© 2013 Columbia University
E6885 Network Science Lecture 2: Network Representations and Characteristics
E 6885 Topics in Signal Processing -- Network Science
Ching-Yung Lin, Dept. of Electrical Engineering, Columbia University
September 16th, 2013
© 2013 Columbia University2 E6885 Network Science – Lecture 2: Network Characteristics
Course Structure
Class Date Lecture Topics Covered
09/09/13 1 Overview of Network Science
09/16/13 2 Network Representation and Feature Extraction
09/23/13 3 Network Paritioning, Clustering and Visualization
09/30/13 4 Graph Database
10/07/13 5 Network Sampling, Estimation, and Modeling
10/14/13 6 Network Analysis Use Case
10/21/13 7 Network Tolopogy Inference and Prediction
10/28/13 8 Graphical Model and Bayesian Networks
11/11/13 9 Final Project Proposal Presentation
11/18/13 10 Dynamic and Probabilistic Networks
11/25/13 11 Information Diffusion in Networks
12/02/13 12 Impact of Network Analysis
12/09/13 13 Large-Scale Network Processing System
12/16/13 14 Final Project Presentation
© 2013 Columbia University3 E6885 Network Science – Lecture 2: Network Characteristics
TA of the course
Xiao-Ming Wu <xw2223>; Office Hours: Friday 2-4pm (7LE3 Schapiro Building)
© 2013 Columbia University4 E6885 Network Science – Lecture 2: Network Characteristics
Graphs and Matrix Algebra
The fundamental connectivity of a graph G may be captured in an binary symmetric matrix A with entries:
v vN N´
1, { , }
0,ij
if i j EA
otherwise
Îì= íî
A is called the Adjacency Matrix of G
© 2013 Columbia University5 E6885 Network Science – Lecture 2: Network Characteristics
Some properties of adjacency matrix
The row sum is equal to the degree of vertex.
i i ijj
d A A+= =å Symmetry:
i iA A+ +=
Number of walks of length r from the r-th power of A : Ar
rijA
© 2013 Columbia University6 E6885 Network Science – Lecture 2: Network Characteristics
For directional graph
{i,j} represents an directed edge from i to j.
In and Out degrees:
outi iA d+ =
inj jA d+ =
© 2013 Columbia University7 E6885 Network Science – Lecture 2: Network Characteristics
Algorithms
Some questions:
–Are vertex i and j linked by an edge?
–What is the degree of vertex i?
–What is the shortest path(s) between vertex i and j?
–How many connected component does the graph have?
–(for a directed graph,) does it have cycles or is it acyclic?
–What is the maximal clique in a graph?
© 2013 Columbia University8 E6885 Network Science – Lecture 2: Network Characteristics
Algorithmic Complexity
‘Tractable’: Polynomial Time:
( )pO n 0p >
‘Intractable:’ Super-Polynomial Time, e.g.:
( )nO a 1a >
The design of efficient algorithms is usually nontrivial.
Usually improved by indexing, storage, removing redundant computations, etc.
© 2013 Columbia University9 E6885 Network Science – Lecture 2: Network Characteristics
Example – Finding all vertices that are reachable from a vertex
Breadth-First Search (BFS)
Depth-First Search (DFS)
© 2013 Columbia University10 E6885 Network Science – Lecture 2: Network Characteristics
Characteristics and Structure Properties of Network
Some questions to ask:
–Triplets of vertices (triads) in social dynamics
–Paths and flows in graph
–Importance of individual element => how ‘central’ the corresponding vertex is in the network
–Finding communities
Characteristics of individual vertices
Characteristics of network cohesion
© 2013 Columbia University11 E6885 Network Science – Lecture 2: Network Characteristics
Centrality
“There is certainly no unanimity on exactly what centrality is or its conceptual foundations, and there is little agreement on the procedure of its measurement.” – Freeman 1979.
Degree (centrality)
Closeness (centrality)
Betweeness (centrality)
Eigenvector (centrality)
© 2013 Columbia University12 E6885 Network Science – Lecture 2: Network Characteristics
Degree Distribution Example: Power-Law Network
A. Barbasi and E. Bonabeau, “Scale-free Networks”, Scientific American 288: p.50-59, 2003.
/kkp C k et k- -= ×
Newman, Strogatz and Watts, 2001!
km
km
p ek
-= ×
© 2013 Columbia University13 E6885 Network Science – Lecture 2: Network Characteristics
Another example of complex network: Small-World Network
Six Degree Separation: – adding long range link, a regular graph can be transformed into a small-
world network, in which the average number of degrees between two nodes become small.
from Watts and Strogatz, 1998
C: Clustering Coefficient, L: path length, (C(0), L(0) ): (C, L) as in a regular graph; (C(p), L(p)): (C,L) in a Small-world graph with randomness p.
© 2013 Columbia University14 E6885 Network Science – Lecture 2: Network Characteristics
Indication of ‘Small’
A graph is ‘small’ which usually indicates the average distance between distinct vertices is ‘small’
1( , )
( 1) / 2 u v Vv v
l dist u vN N ¹ Î
=+ å
For instance, a protein interaction network would be considered to have the small-world property, as there is an average distance of 3.68 among the 5,128 vertices in its giant component.
© 2013 Columbia University15 E6885 Network Science – Lecture 2: Network Characteristics
Some examples of Degree Distribution
(a) scientist collaboration: biologists (circle) physicists (square), (b) collaboration of move actors, (d) network of directors of Fortune 1000 companies
© 2013 Columbia University16 E6885 Network Science – Lecture 2: Network Characteristics
Degree Distribution
Kolaczyk, “Statistical Analysis of Network Data: Methods and Models”, Springer 2009.
© 2013 Columbia University17 E6885 Network Science – Lecture 2: Network Characteristics
Degree Correlations
Kolaczyk, “Statistical Analysis of Network Data: Methods and Models”, Springer 2009.
© 2013 Columbia University18 E6885 Network Science – Lecture 2: Network Characteristics
Conceptual Descriptions of Three Centrality Measurements
Kolaczyk, “Statistical Analysis of Network Data: Methods and Models”, Springer 2009.
© 2013 Columbia University19 E6885 Network Science – Lecture 2: Network Characteristics
Closeness
Closeness: A vertex is ‘close’ to the other vertices
1( )
( , )CI
u V
c vdist v u
Î
=å
where dist(v,u) is the geodesic distance between vertices v and u.
© 2013 Columbia University20 E6885 Network Science – Lecture 2: Network Characteristics
Betweenness
Betweenness measures are aimed at summarizing the extent to which a vertex is located ‘between’ other pairs of vertices.
Freeman’s definition:
( , | )( )
( , )Bs t v V
s t vc v
s t
ss¹ ¹ Î
= å
Calculation of all betweenness centralities requires– calculating the lengths of shortest paths among all pairs of vertices
–Computing the summation in the above definition for each vertex
© 2013 Columbia University21 E6885 Network Science – Lecture 2: Network Characteristics
Eigenvector Centrality
Try to capture the ‘status’, ‘prestige’, or ‘rank’.
More central the neighbors of a vertex are, the more central the vertex itself is.
{ , }
( ) ( )Ei Eiu v E
c v c uaÎ
= å
The vector ( (1),..., ( ))TEi Ei Ei vc c N=c is the solution of the
eigenvalue problem: 1Ei Eia -× =A c c
© 2013 Columbia University22 E6885 Network Science – Lecture 2: Network Characteristics
PageRank Algorithm (Simplified)
© 2013 Columbia University23 E6885 Network Science – Lecture 2: Network Characteristics
PageRank Steps
Example: Simplified Initial State:
R(A) = R(B) = R(C) = R(D) = 0.25
Iterative Procedure:
R(A) = R(B) / 2 + R(C) / 1 + R(D) / 3
A B
C D
( )( )
vv B v
R uR u d e
NÎ
= +å
u uN F=
uF
uB
where
The set of pages u points to
The set of pages point to u
Number of links from u
Normalization / damping factord
1 de
N
-= In general, d=0.85
© 2013 Columbia University24 E6885 Network Science – Lecture 2: Network Characteristics
Solution of PageRank
The PageRank values are the entries of the dominant eigenvector of the modified adjacency matrix.
1
2
( )
( )
:
( )N
R p
R p
R p
é ùê úê ú=ê úê úë û
R
where R is the solution of the equation
1 1 1 1 1
2 1
1
( , ) ( , ) ( , )(1 ) /
( , )(1 ) /
( , )
( , ) ( , )(1 ) /
N
i j
N N N
l p p l p p l p pd N
l p pd Nd
l p p
l p p l p pd N
- é ùé ùê úê ú- ê úê ú= +ê úê úê úê ú-ë û ë û
R R
⋱ ⋮
⋮ ⋮⋮
where R is the adjacency function if page pj does not link to pi, and normalized such that for each j,
( , ) 0i jl p p =
1
( , ) 1N
i ji
l p p=
=å
© 2013 Columbia University25 E6885 Network Science – Lecture 2: Network Characteristics
Example: Brain of an epliepsy patient
© 2013 Columbia University26 E6885 Network Science – Lecture 2: Network Characteristics
Network Representation of Cortical-level Coupling
© 2013 Columbia University27 E6885 Network Science – Lecture 2: Network Characteristics
Visual Summaries of Degree and Closeness Centrality
preictal ictaldiff
© 2013 Columbia University28 E6885 Network Science – Lecture 2: Network Characteristics
Visual Summaries of Betweenness Centrality and Clustering Coefficient
preictal ictaldiff
© 2013 Columbia University29 E6885 Network Science – Lecture 2: Network Characteristics
Connectivity of Graph
A measure related to the flow of information in the graph
Connected every vertex is reachable from every other
A connected component of a graph is a maximally connected subgraph.
A graph usually has one dominating the others in magnitude giant component.
© 2013 Columbia University30 E6885 Network Science – Lecture 2: Network Characteristics
Network Cohesion
Questions to answer:– Do friends of a given actor in a SN tend to be friends of one another?– What collections of proteins in a cell appear to work closely together?– Does the structure of the pages in the WWW tend to separate with respect to
distinct types of content?– What portion of a measured Internet topology would seem to constitute the
backbone?
Definitions differ in– Scale– Local to Global– Explicity (.e.g., cliques) vs implicity (e.g. clusters)
© 2013 Columbia University31 E6885 Network Science – Lecture 2: Network Characteristics
Local Density
A coherent subset of nodes should be locally dense.
Cliques:
3-cliques
A sufficient condition for a clique of size n to exist in G is:
2 2
2 1v
e
N nN
n
æ ö -æ ö> ç ÷ç ÷-è øè ø
© 2013 Columbia University32 E6885 Network Science – Lecture 2: Network Characteristics
Weakened Versions of Cliques -- Plexes
A subgraph H consisting of m vertices is called n-plex, for m > n, if no vertex has degree less than m – n.
1-plex
1-plex No vertex is missing more than 1 of its possible m-1 edges.
© 2013 Columbia University33 E6885 Network Science – Lecture 2: Network Characteristics
Another Weakened Versions of Cliques -- Cores
A k-core of a graph G is a subgraph H in which all vertices have degree at least k.
3-core
Batagelj et. al., 1999. A maximal k-core subgraph may be computed in as little as O( Nv + Ne) time.
Computes the shell indices for every vertex in the graph
Shell index of v = the largest value, say c, such that v belongs to the c-core of G but not its (c+1)-core.
For a given vertex, those neighbors with lesser degree lead to a decrease in the potential shell index of that vertex.
© 2013 Columbia University34 E6885 Network Science – Lecture 2: Network Characteristics
Density measurement
The density of a subgraph H = ( VH , EH ) is:
( )( 1) / 2
H
H H
Eden H
V V=
-
Range of density
and
0 ( ) 1den H£ £
( ) ( 1) ( )Hden H V d H= -
average degree of H
© 2013 Columbia University35 E6885 Network Science – Lecture 2: Network Characteristics
Use of the density measure
Density of a graph: let H=G
‘Clustering’ of edges local to v: let H=Hv, which is the set of neighbors of a vertex v, and the edges between them
Clustering Coefficient of a graph: The average of den(Hv) over all vertices
© 2013 Columbia University36 E6885 Network Science – Lecture 2: Network Characteristics
An insight of clustering coefficient
A triangle is a complete subgraph of order three.
A connected triple is a subgraph of three vertices connected by two edges (regardless how the other two nodes connect).
The local clustering coefficient can be expressed as:
The clustering coefficient of G is then:
3
( )( ) ( )
( )v
vden H cl v
v
ttD= =
1( ) ( )
v V
cl G cl vV ¢Î
=¢å
Where V’ V is the set of vertices v with dv ≥ 2.
# of triangles
# of connected triples for which 2 edges are both incident to v.
© 2013 Columbia University37 E6885 Network Science – Lecture 2: Network Characteristics
An example
© 2013 Columbia University38 E6885 Network Science – Lecture 2: Network Characteristics
Transitivity of a graph
A variation of the clustering coefficient takes weighted average
where
3
3 3
( ) ( )3 ( )
( )( ) ( )
v VT
v V
v cl vG
cl Gv G
tt
t t¢Î D
¢Î
= =åå
1( ) ( )
3 v V
G vt tD DÎ
= å
3 3( ) ( )v V
G vt tÎ
=å
is the number of triangles in the graph
is the number of connected triples
The friend of your friend is also a friend of yours
Clustering coefficients have become a standard quantity for network structure analysis. But, it is important on reporting which clustering coefficients are used.
© 2013 Columbia University39 E6885 Network Science – Lecture 2: Network Characteristics
Vertex / Edge Connectivity
If an arbitrary subset of k vertices or edges is removed from a graph, is the remaining subgraph connected?
A graph G is called k-vertex-connected, if (1) Nv>k, and (2) the removal of any subset of vertices X in V of cardinality |X| smaller than k leaves a subgraph G – X that is connected.
The vertex connectivity of G is the largest integer such that G is k-vertex-connected.
• Similar measurement for edge connectivity
© 2013 Columbia University40 E6885 Network Science – Lecture 2: Network Characteristics
Vertex / Edge Cut
If the removal of a particular set of vertices in G disconnects the graph, that set is called a vertex cut.
For a given pair of vertices (u,v), a u-v-cut is a partition of V into two disjoint non-empty subsets, S and S’, where u is in S and v is in S’.
Minimum u-v-cut: the sum of the weights on edges connecting vertices in S to vertices in S’ is a minimum.
© 2013 Columbia University41 E6885 Network Science – Lecture 2: Network Characteristics
Minimum cut and flow
Find a minimum u-v-cut is an equivalent problem of maximizing a measure of flow on the edges of a derived directed graph.
Ford and Fulkerson, 1962. Max-Flow Min-Cut theorem.
© 2013 Columbia University42 E6885 Network Science – Lecture 2: Network Characteristics
Graph Partitioning
Many uses of graph partitioning:– E.g., community structure in social networks
A cohesive subset of vertices generally is taken to refer to a subset of vertices that– (1) are well connected among themselves, and– (2) are relatively well separated from the remaining vertices
Graph partitioning algorithms typically seek a partition of the vertex set of a graph in such a manner that the sets E( Ck , Ck’ ) of edges connecting vertices in Ck to vertices in Ck’ are relatively small in size compared to the sets E(Ck) = E( Ck , Ck’ ) of edges connecting vertices within Ck’ .
© 2013 Columbia University43 E6885 Network Science – Lecture 2: Network Characteristics
Classify the nodes
© 2013 Columbia University44 E6885 Network Science – Lecture 2: Network Characteristics
Example: AIDS blog network
© 2013 Columbia University45 E6885 Network Science – Lecture 2: Network Characteristics
Example: Karate Club Network
© 2013 Columbia University46 E6885 Network Science – Lecture 2: Network Characteristics
Hierarchical Clustering
Agglomerative
Divisive
In agglomerative algorithms, given two sets of vertices C1 and C2, two standard approaches to assigning a similarity value to this pair of sets is to use the maximum (called single-linkage) or the minimum (called complete linkage) of the similarity xij over all pairs.
( ) ( 1)i jv v
ijv v
N Nx
d N d N
D=
+ -
The “normalized” number of neighbors of vi and vj that are not shared.
© 2013 Columbia University47 E6885 Network Science – Lecture 2: Network Characteristics
Hierarchical Clustering Algorithms Types
Primarily differ in [Jain et. al. 1999]:– (1) how they evaluate the quality of proposed clusters, and– (2) the algorithms by which they seek to optimze that quality.
Agglomerative: successive coarsening of parittions through the process of merging.
Divisive: successive refinement of partitions through the process of splitting.
At each stage, the current candidate partition is modified in a way that minizes a specific measure of cost.
In agglomerative methods, the least costly merge of two previously existing partition elements is executed
In divisive methods, it is the least costly split of a single existing partition element into two that is executed.
© 2013 Columbia University48 E6885 Network Science – Lecture 2: Network Characteristics
Hierarchical Clustering The resulting hierarchy typically is represented in the form of a tree, called a
dendrogram.
The measure of cost incorporated into a hierarchical clustering method used in graph partitioning should reflect our sense of what defines a ‘cohesive’ subset of vertices.
In agglomerative algorithms, given two sets of vertices C1 and C2, two standard approaches to assigning a similarity value to this pair of sets is to use the maximum (called single-linkage) or the minimum (called complete linkage) of the dissimilarity xij over all pairs.
Dissimlarities for subsets of vertices were calculated from the xij using the extension of Ward (1963)’s method and the lengths of the branches in the dendrogram are in relative proportion to the changes in dissimilarity.
( ) ( 1)i jv v
ijv v
N Nx
d N d N
D=
+ -
xij is the “normalized” number of neighbors of vi and vj that are not shared.
Nv is the set of neighbors of a vertex.
Δ is the symmetric difference of two sets which is the set of elements that are in one or the other but not both.
© 2013 Columbia University49 E6885 Network Science – Lecture 2: Network Characteristics
Other dissimilarity measures
There are various other common choices of dissimilarity measures, such as:
2
,
( )ij ik jkk i j
x A A¹
= -å
Hierarchical clustering algorithms based on dissimilarities of this sort are reasonably efficient, running in time.2( log )v vO N N
© 2013 Columbia University50 E6885 Network Science – Lecture 2: Network Characteristics
Hierarchical Clustering Example
© 2013 Columbia University51 E6885 Network Science – Lecture 2: Network Characteristics
Questions?