Network Growth and the Spectral Evolution Model

Web Science & Technologies

University of Koblenz ▪ Landau, Germany

Network Growth and the Spectral Evolution Model

Jérôme Kunegis¹, Damien Fay², Christian Bauckhage³¹ University of Koblenz-Landau

² University of Cambridge³ Fraunhofer IAIS

CIKM 2010, Toronto, Canada

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 20102 / 24

Recommender Systems

Example: Find friends of Facebook

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 20103 / 24

Graph Theory

Known links (A)Unknown links (B)

The users of Facebook are connected by friendship links, forming a graph. This graph is undirected.

Let A be the set of links in the network. Let B be the set of links that will appear in the future.

Task: Find a suitable function f(A) = B.

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 20104 / 24

Algebraic Graph Theory

Known links (A)Unknown links (B)

Use adjacency matrices A, B ∈ {0, 1}n×n :

A = B = [0 1 0 0 01 0 1 0 00 1 0 1 00 0 1 0 10 0 0 1 0

] [0 0 1 1 00 0 0 0 01 0 0 0 01 0 0 0 00 0 0 0 0

]

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 20105 / 24

Spectral Graph Theory

Use th eigenvalue decomposition :

A = UΛUT

B = VΣVT

U and V are orthogonal and contain eigenvectors Λ and Σ are diagonal and contain eigenvalues

Task : find an f of the following form :f(UΛUT) = VΣVT

In this talk : The observation that U ≈ V Extrapolation of Λ to Σ

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 20106 / 24

Outline

Eigenvalue evolution

Eigenvector evolution

Diagonality test

The spectral evolution model

Explanations

Control tests

Spectral extrapolation

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 20107 / 24

Eigenvalue Evolution

Wikipedia Facebook

Eigenvectors grow Not all eigenvectors grow at the same speed, even in a single network

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 20108 / 24

Eigenvector Evolution

Wikipedia Facebook

Compute the cosine over time between eigenvectors and their initial value.

Some eigenvectors stay constant Some eigenvectors change suddenly

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 20109 / 24

Eigenvector Permutation

Wikipedia Facebook

Compute the cosine between all eigenvector pairs at two times.

Eigenvalues get permuted :A = UΛUT

B = VΣVT

Ui = V

j

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201010 / 24

Diagonality Test

A = UΛUT

B = VΣVT

Diagonalize B using U.

B = UDUT

D = U−1B(UT)−1

D = UTBU

UTBU should be diagonal!

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201011 / 24

Diagonality Test

Wikipedia Facebook

D is nearly diagonal The diagonal of D is irregular

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201012 / 24

The Spectral Evolution Model

Networks grow spectrally

Eigenvectors stay constantEigenvalues change

Why?

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201013 / 24

Explanation : Matrix Powers

Known links (A)Unknown links (B)

The square of A constains the number of paths of length two between any node pair:

A² =

Generally, Ak contains the number of k-paths between any node pair.

[1 0 1 0 00 2 0 1 01 0 2 0 10 1 0 2 00 0 1 0 1

]

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201014 / 24

Explanation: Polynomials

p(A) = αA² + βA³ + γA + …⁴

Polynomials are good link prediction functions :

Count parallel paths Weight paths by length (α > β > γ > …)

The matrix power is a spectral transformation, e.g.:A² = (UΛUT)(UΛUT) = UΛ²UT

Polynomials are spectral transformations:p(A) = p(UΛUT) = Up(Λ)UT

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201015 / 24

Explanation: Graph Kernels

Matrix exponential

exp(A) = I + A + ½ A² + … = Uexp(Λ)UT

Von Neumann kernel

(I − αA)−1 = I + αA + α²A² + … = U(I − αΛ)−1UT

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201016 / 24

Explanation: Preferential Attachment

Write A as a sum of rank-1 matrices:

A = A1 + A

2 + …

Ai = λ

iu

iu

iT

Interpret each Ai as the adjacency matrix of one

weighted graph In A

i, vertex j has degree Σ

k λ

iu

iju

ik ~ u

ij

Consider the process of preferential attachment in each latent dimension separately:

Σiu

iu

iT = λ

iu

iu

iT + ε

iu

iu

iT

pa(A) = U(Λ + Ε)UT

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201017 / 24

Control: Matrix Perturbation

Add edges at random to A = UΛUT.

The evolution of A should then be :

A + E = Ũ Λ ŨT

|| Λ − Λ ||F = O(ε²)

| U.k

T Ũ.k | = O(ε)

Using ||E||2 = ε.

Random growth is not spectral.

~

~

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201018 / 24

Control: Random Sampling

Split an Erdős–Rényi random graph into A + B. Apply the diagonality test for transforming A into B.

Only one latent dimension is preserved.

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201019 / 24

Spectral Extrapolation

To predict links, extrapolate the evolution of eigenvalues.

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201020 / 24

Experiments

Methodology :

Retain newest edges as training set Compute link prediction scores Evaluate using the mean average precision (MAP) User over a hundred datasets

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201021 / 24

Experiments: Symmetric Networks

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201022 / 24

Experiments: Weighted Networks

Use the weighted adjacency matrix A.

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201023 / 24

Experiments: Bipartite and Directed Networks

Use the singular value decomposition A = UΛVT.

Jérôme Kunegiskunegis@uni-koblenz.de

CIKM 201024 / 24

Summary & Discussion

Eigenvectors remain constant Eigenvalues grow irregularly Extrapolate the eigenvalues to predict links

Experimental results:

Extrapolation works best for bipartite and directed networks