Upload
tiffany-clarke
View
218
Download
0
Tags:
Embed Size (px)
Citation preview
Random Dot Product GraphsRandom Dot Product Graphs
Ed ScheinermanApplied Mathematics & Statistics
Johns Hopkins University
IPAMIntelligent Extraction of Information from
Graphs & High Dimensional DataJuly 26, 2005
CoconspiratorsCoconspirators
• Libby Beer• John Conroy (IDA)• Paul Hand (Columbia)• Miro Kraetzl (DSTO)• Christine Nickel• Carey Priebe• Kim Tucker• Stephen Young (Georgia Tech)
OverviewOverview
• Mathematical context
• Modeling networks
• Random dot product model
• The inverse problem
Mathematical ContextMathematical Context
Graphs I Have LovedGraphs I Have Loved
• Interval graphs & intersection graphs
• Random graphs
• Random intersection graphs
• Threshold graphs & dot product graphs
Interval GraphsInterval Graphs
€
v a Ivv ~ w ⇔ Iv ∩ Iw ≠∅
Intersection GraphsIntersection Graphs
€
v a Sv
v ~ w ⇔ Sv ∩ Sw ≠∅
{1}
{1}
{1,2}
{2}
Random GraphsRandom Graphs
Erdös-Rényi style…
p 1 – p
Randomness is “in” the edges. Vertices are “dumb” placeholders.
Random Intersection GraphsRandom Intersection Graphs
• Assign random sets to vertices.
• Two vertices are adjacent iff their sets intersect.
• Randomness is “in” the vertices.
• Edges reflect relationships between vertices.
Threshold GraphsThreshold Graphs
€
v a xv ∈ R
v ~ w ⇔ xv + xw ≥1
0.5
0.6
0.8
0.3
Dot Product GraphsDot Product Graphs
€
v a xv ∈ Rd
v ~ w ⇔ xv ⋅xw ≥1
[1 0]
[2 0]
[1 1]
[0 1]
Fractional intersection graphs
Communication NetworksCommunication Networks
Physical NetworksPhysical Networks
Telephone
Local area network
Power grid
Internet
Social NetworksSocial Networks
Alice
Bob
A B
2003-4-10
Social Network GraphsSocial Network Graphs
Vertices (Actors) Edges (Dyads)
Telephones Calls
Email addresses Messages
Computers IP Packets
Human beings Acquaintance
Academicians Coauthorship
Example: Email at HPExample: Email at HP
• 485 employees
• 185,000 emails
• Social network (who emails whom) identified 7 “communities”, validated by interviews with employees.
Properties of Social NetworksProperties of Social Networks
• Clustering
• Low diameter
• Power law
Properties of Social NetworksProperties of Social Networks
• Clustering
• Low diameter
• Power law
€
P a ~ c | a ~ b ~ c[ ] > P a ~ c[ ]
a
b
c
Properties of Social NetworksProperties of Social Networks
• Clustering
• Low diameter
• Power law
“Six degrees of separation”
Properties of Social NetworksProperties of Social Networks
• Clustering
• Low diameter
• Power law
log d
log
N(d
)
Degree Histogram
Degree Histogram Example 1Degree Histogram Example 1
2838 vertices
degree
Num
ber
of v
erti
ces
Degree Histogram Example 2Degree Histogram Example 2
16142 vertices
degree
Num
ber
of v
erti
ces
Random Graph ModelsRandom Graph Models
Goal: Simple and realistic random graph models of social networks.
Erdös-Rényi?Erdös-Rényi?
• Low diameter!
• No clustering: P[a~c]=P[a~c|a~b~c].
• No power-law degree distribution.
Not a good model.
Model by Fan Chung et alModel by Fan Chung et al
€
N(d) = α d−β⎣ ⎦
Consider only those graphs with
with all such graphs equally likely.
People as VectorsPeople as Vectors
€
a1
a2
a3
a4
⎡
⎣
⎢ ⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥ ⎥
Sports
Politics
Movies
Graph theory
€
b1
b2
b3
b4
⎡
⎣
⎢ ⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥ ⎥
Shared InterestsShared Interests
€
P a ~ b[ ] = f a ⋅b( )
Alice and Bob are more likely to communicate when they have more shared interests.
Selecting the FunctionSelecting the Function
€
P a ~ b[ ] = f a ⋅b( )
€
f(t)=1
πtan
−1(t)+
1
2 f:[−∞,+∞]→[0,1]
Selecting the FunctionSelecting the Function
€
P a ~ b[ ] = f a ⋅b( ) €
f(t)=t
1+t a⋅b≥0 f:[0,∞]→[0,1]
Selecting the FunctionSelecting the Function
€
f(t)=tr
a⋅b∈[0,1]
f:[0,1]→[0,1]
€
P a ~ b[ ] = f a ⋅b( )
Random Dot Product Graphs, IRandom Dot Product Graphs, I
€
Given x1,x2 ,K ,xn ∈ Rd
P[i ~ j] = xi ⋅x j or = f (xi ⋅x j )( )
Write X = [x1,x2,K ,xn ]
PX (G) = (xi ⋅x j )ij∈E
∏ ⎛
⎝ ⎜ ⎜
⎞
⎠ ⎟ ⎟× (1−xi ⋅x j )
ij∉E i≠ j
∏ ⎛
⎝ ⎜ ⎜
⎞
⎠ ⎟ ⎟
Generalize Erdös-RényiGeneralize Erdös-Rényi
€
Take x1 = x2 =L = xn = x
with x ⋅x = p.
Generalize Intersection GraphsGeneralize Intersection Graphs
€
If i a Ai ⊆{1,2,K ,k}
take xi = χ (Ai )∈ {0,1}k
and f (t) =0 t = 0
1 t > 0
⎧ ⎨ ⎩
Whence the Vectors?Whence the Vectors?
• Vectors are given in advance.
• Vectors chosen (iid) from some distribution.
€
P(G) = PX (G) dX∫
Random Dot Product Graphs, IIRandom Dot Product Graphs, II
• Step 1: Pick the vectors Given by fiat. Chosen from iid a distribution.
• Step 2: For all i<j Let p=f(xi•xj).
Insert an edge from i to j with probability p.
MegageneralizationMegageneralization
• Generalization of: Intersection graphs (ordinary & random) Threshold graphs Dot product graphs Erdös-Rényi random graphs
• Randomness is “in” both the vertices and the edges.
• P[a~b] independent of P[c~d] when a,b,c,d are distinct.
Results in Dimension 1Results in Dimension 1
€
Choose xi iid uniform in [0,1].
Use f (t) = t r .
€
Choose xi independently from U r[0,1]
P(i ~ j) = xix j f (t) = t
Probability/Number of EdgesProbability/Number of Edges
€
P[i ~ j] = (xix j )rdxidx j
0
1
∫0
1
∫ =1
(1+ r)2
€
Expected number of edges =n
2
⎛
⎝ ⎜
⎞
⎠ ⎟1+ r( )
−2.
ClusteringClustering
€
P[a ~ c | a ~ b ~ c] =P[a ~ c & a ~ b ~ c]
P[a ~ b ~ c]
=(xy)r (xz)r (yz)rdxdydz∫∫∫
(xy)r (yz)rdxdydz∫∫∫
=(1+ r)2(1+ 2r)
(1+ 2r)3>
1
(1+ r)2= P[a ~ c]
Power LawPower Law
€
Believe :
N(d)∝ d−c, c =1−1/r
Can show :
N (1−ε)d,(1+ ε)d( )∝ 2εd−c
Power Law ExamplePower Law Example
€
n = 30000
P[i ~ j] = (xix j )3
Isolated VerticesIsolated Vertices
€
E[N(0)] ~ Crn(r−1)/r = o(n)
where Cr =(1+ r)1/r Γ(1/ r)
r.
Thus, the graph is not connected, but…
“Mostly” Connected“Mostly” Connected
“Giant” connected component
A “few” isolated vertices
Six Degrees of SeparationSix Degrees of Separation
Diameter ≤ 6
Attached
Attachedpair
Diameter ≤ 6 Proof OutlineDiameter ≤ 6 Proof Outline
€
{i : xi ≥ τ }
Diameter = 2
Isolated
€
{i : xi < τ }
Diameter ≤ 6 Proof OutlineDiameter ≤ 6 Proof Outline
Graphs to VectorsGraphs to Vectors
The Inverse Problem
Given Graphs, Find VectorsGiven Graphs, Find Vectors
• Given: A graph, or a series of graphs, on a common vertex set.
• Problem: Find vectors to assign to vertices that “best” model the graph(s).
Maximum Likelihood MethodMaximum Likelihood Method
• Feasible in dimension 1. Awful d>1.
• Nice results for f(t) = t / (1+t).€
arg maxX
{PX (G)}
Gram Matrix ApproachGram Matrix Approach
€
Given G1,G2 ,K ,Gm .
Let A =1
mA(G j )
j=1
m
∑ .
∴ aij ≈ P[i ~ j] = xi ⋅x j (i ≠ j)
X =[x1,x2,K ,xn ] (d ×n)
A ≈ XTX
Wrong Best SolutionWrong Best Solution
€
Minimize f (X) = A− XTXF
2
€
A =UTΛU; λ 1 ≥ λ 2 ≥L ≥ λ n
X = gd (A) :=
λ 1+ 0 L 0
0 λ 2+ L 0
M M O M
0 0 L λ d+
⎡
⎣
⎢ ⎢ ⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥ ⎥ ⎥
u1T
u2T
M
udT
⎡
⎣
⎢ ⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥ ⎥
Real ProblemReal Problem
€
Minimize f (X) = A− XTX + I o(XTX)F
2
€
We don’ t want xi ⋅xi ≈ 0 = aii .
Idea : aii ← xi ⋅xi
Iterative AlgorithmIterative Algorithm
€
1. D = 0n×n
2. X = gd (A+ D)
3. D = I o(XTX)
4. Go to 2
€
Minimize f (X) = A− XTX + I o(XTX)F
2
ConvergenceConvergence
€
If (when) the algorithm converges,
then the rows of X are eigenvectors
of A+ I o(XTX) and X is a local min
of f (X) = A− XTX + I o(XTX)F
2.
ConvergenceConvergence
iteration
diag
onal
ent
ries
€
G(n = 40,m =115) d = 2
ConvergenceConvergence
iteration
diag
onal
ent
ries
€
G(n = 40,m =115) d = 5
€
max{xi ⋅x j : i ≠ j} =1.05
ConvergenceConvergence
iteration
diag
onal
ent
ries
€
max{xi ⋅x j : i ≠ j} =1.152
€
G(n = 40,m =115) d =12
ConvergenceConvergence
iteration
diag
onal
ent
ries
€
G = C12 d =1
30 iterations
ConvergenceConvergence
iteration
diag
onal
ent
ries
€
G = C12 d =1
150 iterations
ConvergenceConvergence
iteration
diag
onal
ent
ries
€
G = C12 d =1
500 iterations
Enron exampleEnron example
ApplicationsApplications
Network Change/Anomaly Detection
Clustering
Change/Anomaly DetectionChange/Anomaly Detection
€
G1,G2 ,K ,Gr
X1 2 4 4 3 4 4 H1,H2,K ,H s
Y1 2 4 4 3 4 4
Align X,Y
Find xi − yi large.
Change/Anomaly DetectionChange/Anomaly Detection
Graph ClusteringGraph Clustering
Graph ClusteringGraph Clustering
Synthetic Lethality GraphsSynthetic Lethality Graphs
• Vertices are genes in yeast
• Edge between u and v iff Deleting one of u or v does not kill, but Deleting both is lethal.
SL Graph StatusSL Graph Status
• Yeast has about 6000 genes.
• Full graph known on 126 “query” genes (about 1300 edges).
• Partial graph known on 1000 “library” genes.
What Next?What Next?
Random Dot Product GraphsRandom Dot Product Graphs
• Extension to higher dimension Cube Unit ball intersect positive orthant
• Small world measures: clustering coefficient
• Other random graph properties
€
γ(v) = E(N(v)) ÷N (v)
2
⎛
⎝ ⎜
⎞
⎠ ⎟
γ(G) = average γ (v) :d(v) ≥ 2.
Vector EstimationVector Estimation
• MLE method Computationally efficient? More useful?
• Eigenvalue method Understand convergence Prove that it globally minimizes Extension to missing data
• Validate against real data
Network EvolutionNetwork Evolution
• Communication influences interests:
€
X =[x1,x2 ,…,xn ]
X(k +1) = F[G, X(k)]
Rapid GenerationRapid Generation
• Can we generate a sparse random dot product graph with n vertices and m edges in time O(n+m)?
• Partial answer: Yes, but.
The EndThe End