Embed Size (px)
Citation preview
ECS 253 / MAE 253, Lecture 9May 1, 2018
“Web search and decentralized search onsmall-world networks”
Announcements
• Last week:
– Modularity & community structure– Signed, weighted networks– Software tools!∗ Graphviz∗ NetworkX & python∗ iGraph
• HW2, HW2a, HW2b all due on Thurs(Turn in via Canvas by midnight, or hardcopy in Class)
• D’Souza office hours this week, Thurs 3:30-4:25pm(30 min shift forward)
Search for information
Assume some resource of interest is stored at the vertices of anetwork:
• Web pages
• Files in a file-sharing network
Would like to determine rapidly where in the network aparticular item of interest can be found.
To warehouse data or search on demand?
• Centralized : Catalogue data in one central place.
– Makes most sense when high cost to search networkin real time.
– Requires resources for learning the data and storing it.
• Decentralized : Data is spread out in a distributed data base.
– Can be a very slow process to search.
– But dependent on network topology may be able to devise“quick” algorithms.
Web search
• Centralized warehousing of information (need results asquickly as possible)
• Key: Use information contained in the edges as well asthe vertices! (Assumes edges contain information aboutrelevance).
• Process: Query arrives, select subset of pages which match,order that subset by ranking based on link structure.
Typical web domain
M. E. J. Newman
Ranking pages in a connected component
• Each site starts with unit “rank” (i.e., weight).
• Each site transfers a fraction of this rank equally to eachneighbor, in a discrete time process.
• So the rank of vertex i, ri:
ri ∝∑j
Aij (rj/dj) ,
where A is the adjacency matrix, and dj degree of node j.This is rj times the random walk dynamics on the network!
Ranking pages, cont.
• The vector of ranks: ~r = [r1, r2, ....ri, ...rN ]T
• The dynamics: ~r =M~r , where Mij ∝ Aij (rj/dj).
• Looking for a steady-state solution to the equation: ~r =M~r ,means finding eigenvector ~r1 with corresponding eigenvalueλ1 = 1.
Thus ~r1 =M~r1 = λ1~r1 = ~r1
Aside on Random Walks:Sample graph structure
Random walk: State Transition Matrix(Column-normalize the adjacency matrix)
M =
1/4 1/3 1/2 1/4 0
1/4 1/3 0 1/4 0
1/4 0 1/2 0 0
1/4 1/3 0 1/4 1/2
0 0 0 1/4 1/2
M will have a basis set of eigenvectors {~ui} and correspondingeigenvalues λi.
Perron-Frobenius Theorem
• Applies to irreducible, positive, stochastic matrices.
• “Irreducible” means cannot be block-diagonalized into disjointpieces. (i.e., network is connected — only one component).
• “Positive” means each entry Mij > 0.
• “Stochastic” means column normalized (or row normalized).
Perron-Frobenius TheoremLeading eigenvalue
• One leading eigenvalue with λ1 = 1.
• The corresponding eigenvector, v1, has strictly positive entriesand the sum over all the entries,
∑i v1[i] = 1.
• This is the stationary distribution of the random walk dynamics.
• For non-negative matrices (Mij ≥ 0), similar results, but can’tguarantee eigenvectors are positive (in practice, normally stillworks ... see R-code if interested : “transMatrix.R”, “samp-matrices.R” and the “Session log”).
Perron-Frobenius Theorem: Remaining eigenvalues(“Spectral partitioning” of graphs)
• All other eigenvalues are less than λ1.(i.e., all λi < 1 for i > 1).
• These represent decaying modes that will eventually convergeto the steady-state solution described by v1.
• How long do they take to decay?
●●
●
●
● ●
●●
●●
● ●
●●
●●
●●
●
●
●●
●
●
● ●●
●●
●
● ●
●
●● ●
●
●
●
●
●
●● ●●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●●
● ●● ●
●●●
●●
●
●●
● ●
●
0 5 10 20
05
1020
X
Y
Timescale, τ1 = 5314 to
++++
+
+++++
+++
++
++
++
++
+++
+
+
+
+ +++
++
++ ++
+++
++
oo oo
o o oo oo
o
o ooo
o
ooo
o
o
o
o
o
o
o
o o
o
●●
●
●
● ●
●●
●●
● ●
●●
●●
●●
●
●
●●
●
●
● ●●
●●
●
● ●
●
●● ●
●
●
●
●
●
●● ●●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●●
● ●● ●
●●●
●●
●
●●
● ●
●
0 5 10 20
05
1020
X
Y
Timescale, τ2 = 1092 to
+
+ ++ +++
+
++
+
+++
++
++
o
o
o oo o
o
o
o
oo
o
o
o
o
oo
o
o
o
o
o
o
o
o
o o
o
o
oo
ooo
o
o
o
o
o
o
o
oo
o
o
oo o
o
o
o
o
o
o
oo
o●
●
●
●
● ●
●●
●●
● ●
●●
●●
●●
●
●
●●
●
●
● ●●
●●
●
● ●
●
●● ●
●
●
●
●
●
●● ●●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●●
● ●● ●
●●●
●●
●
●●
● ●
●
0 5 10 20
05
1020
X
Y
Timescale, τ3 = 157 to
+++
++
++
++
+++
+++
++
++
+
+++
++
++
++
+
o
o
o
o
o
oo
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
oo
oooo
o
o
o
oo
o
o o
o
o
o
o
o
o
Decay / mixing times
• For each eigenmode, with eigenvector ~vi and eigenvalue λi,M~vi = (λi)~vi thus M t~vi = (λi)
t~vi.
• The “relaxation time” for that mode τi: time for the original
amplitude to decay to 1/e. Mτi~vi =1e~vi =⇒ τi = −1/ ln(λi) .
• Recall λi < 1 for all i ≥ 2.
• So the biggest relaxation time is: τmax = τ2 = −1/ ln(λ2).
• The difference λ1 − λ2 is called the spectral gap.
(λ2 < 1. Bigger λ2 means smaller | ln(λ2)|, so larger τ2.In other words, smaller spectral gap means worse mixingproperties.)
Back to: Ranking pages in a single component
• Each site starts with unit “rank” (i.e., weight).
• Transfers fraction of this rank equally to each connected site.
• So at each iteration the rank of vertex i, ri:
ri =∑j
Aij(rj/dj) =∑j
Mijrj
More compactly : ~r =M~r
• Using a random walk formulation, the occupancy probabilitiesin steady-state (i.e., the vector corresponding the λ = 1):
~r =M~r , where M is state transition matrix.
The Web as a whole
(This is an old picture, circa 2000)
Size of the indexed Web2012-2013, see http://www.worldwidewebsize.com/
The size of the World Wide Web (The Internet)
The Indexed Web contains at least 1.78 billion pages (Thursday, 24 April, 2014).
The Dutch Indexed Web contains at least 261.12 million pages (Thursday, 24 April, 2014).
288
TweetTweet
424
The Indexed Web | The Dutch Indexed Web
Last Month Last Three Months Last Year
GB = Sorted on Google and Bing
BG = Sorted on Bing and Google
The size of the World Wide Web:Estimated size of Google's index
Last Month Last Three Months Last Year
893
LikeLike
ShareShare
Last Two Years
Last Two Years
• Neglects the “Dark Web” (e.g. subscriptions required)
• How much valid content is really out there?
What about disconnected components?
We understand how to deal with each components. How do wedeal with getting a consistent rank across the whole web?
The “Random Surfer” model
[Brin and Page, “The anatomy of a large-scale hypertextual Websearch engine”, Computer Networks, 30 (1998)]
[L. Page, S. Brin, R. Motwani, and T. Winograd, “The PagerankCitation Ranking: Bringing Order to the web”, technical report,
Stanford University, Stanford, CA, 1998. ]
• With probability ε follow an out-link of current page.
• With probability [1 − ε] jump at random to some other webpage. (Usually assume jump is random, so land at any sitewith prob 1/N ).
The “Random Surfer” model
The weight of a page j is the sum over all the in-links pointing to
it, including those gained by the random jump:
rj =∑i→j{riε(i) + [1− ε(i)]Jij} .
(Note, this also makes the matrix positive, recall Perron-Frobenius).
Rules of thumb (empirical):
ε(i) = ε
Jij = 1/N
ε = 0.8.
Real-world complications: Page Rank
• Newer pages have less rank, even though they may beextremely relevant.
• Calculate eigenvalues of 109 by 109 matrix!!!(Many advances here due to Page Rank introduction)
• spam, spam, spam
• “Search engine optimizers” (reverse engineer search engines)
• Link farms (both invisible and visible)
• Selling highly ranked domain names
• delisting web sites
Real-world complications in general
• Stop words typically removed: and, of, the to, be, or.So how to handle query “to be or not to be”?
• Dealing with complications of multiple languages and cultures.
– Chinese: no use of pronouns (he/she) instead repeat propernames (which is often a sign of low-quality page/spam inEnglish).
– Japanese: want breadth as well as depth (search 5 pages).US want a quick, definitive answer.
Alternate approaches – topology based
[Kleinberg and Lawrence, “The structure of the Web”, Science,294 (2001)]
[Kleinberg, “Authoritative sources in a hyperlinked environment”,J. ACM, 46 (1999)]
• Slightly more sophisticated. Kleinberg proposes to use in-linksand out-Links.
• Google assumes a page is important if other important pagespoint to it.
• Kleinberg identifies two kinds of importance: “hubs” and“authorities”
Hubs and authorities
• A page pointed to by highly ranked pages in an authority
• A page that points to highly ranked pages is a hub . (May notcontain the information, but will tell you where to find it).
In use:
• Teoma search engine (http://search.ask.com/)
• Citeseer literature search engine.
Alternate approaches
• Usage/click based.
• Negative edge weights: (Penalize spam linking to you.)
• Clustering results by subject, e.g., http://clusty.com/ (nowyippy.com)
• . . .
Distributed search
Some resource of interest is stored at the vertices of a network:i.e., Files in a file-sharing network
Search on arbitrary networks: O(N) (search every node)
• Depth-first
• Breadth-first
Search on power-law random graphs
• Breadth-first passing always to highest-degree node possible.[Adamic, R. M. Lukose, A. R. Puniyani, and B. A. Huberman,“Search in power-law networks”, Phys. Rev. E, 64 2001], findbetween O(N2/3) and O(N1/2).
Complex Systems 535/Physics 508: Homework 2
1. One can calculate the diameter of certain types of networks exactly:
(i) What is the diameter of a clique?
(ii) What is the diameter of a square portion of square lattice, with L edges (or equiva-lently L + 1 vertices) along each side, like this:
L
L
What is the diameter of the corresponding hypercubic lattice in d dimensions withL edges along each side? Hence what is the diameter of such a lattice as a functionof the number n of vertices?
(iii) A Cayley tree is a symmetric regular tree in which each vertex is connected to thesame number k of others. Here, for example, is a Cayley tree for k = 3:
Show that the number of vertices reachable in d steps from the central vertex is k(k!1)d!1 for d " 1. Hence find an expression for the diameter of the network in termsof k and the number of vertices n.
(iv) Which of the networks in parts (i), (ii), and (iii) displays the small-world effect, de-fined as having a diameter that increases as log n or slower?
2. A particular network is believed to have a degree distribution that follows a power lawfor degree greater than or equal to 10. A random sample of vertices is taken and theirdegrees measured. The degrees of the first twenty vertices with degrees 10 or greater are:
16 17 10 26 1314 28 45 10 1212 10 136 16 2536 12 14 22 10
1
Decentralized search on small-world networks
Consider a network with small diameter.
Will this help with decentralized search? (i.e. Find a localalgorithm with performance less than O(N)?)
What is a small-world
[Watts and Strogatz, “Collective dynamics of ‘small-world’networks”, Nature, 393 (1998)]
• Start with regular 1D lattice, with each node connected to it’s knearest neighbors.
• Randomly re-wire each link independently with probability p.
Ave shortest path L(p) andclustering coefficient C(p)
Nature © Macmillan Publishers Ltd 1998
8
letters to nature
NATURE | VOL 393 | 4 JUNE 1998 441
removed from a clustered neighbourhood to make a short cut has, atmost, a linear effect on C; hence C(p) remains practically unchangedfor small p even though L(p) drops rapidly. The important implica-tion here is that at the local level (as reflected by C(p)), the transitionto a small world is almost undetectable. To check the robustness ofthese results, we have tested many different types of initial regulargraphs, as well as different algorithms for random rewiring, and allgive qualitatively similar results. The only requirement is that therewired edges must typically connect vertices that would otherwisebe much farther apart than Lrandom.
The idealized construction above reveals the key role of shortcuts. It suggests that the small-world phenomenon might becommon in sparse networks with many vertices, as even a tinyfraction of short cuts would suffice. To test this idea, we havecomputed L and C for the collaboration graph of actors in featurefilms (generated from data available at http://us.imdb.com), theelectrical power grid of the western United States, and the neuralnetwork of the nematode worm C. elegans17. All three graphs are ofscientific interest. The graph of film actors is a surrogate for a socialnetwork18, with the advantage of being much more easily specified.It is also akin to the graph of mathematical collaborations centred,traditionally, on P. Erdos (partial data available at http://www.acs.oakland.edu/!grossman/erdoshp.html). The graph ofthe power grid is relevant to the efficiency and robustness ofpower networks19. And C. elegans is the sole example of a completelymapped neural network.
Table 1 shows that all three graphs are small-world networks.These examples were not hand-picked; they were chosen because oftheir inherent interest and because complete wiring diagrams wereavailable. Thus the small-world phenomenon is not merely acuriosity of social networks13,14 nor an artefact of an idealized
model—it is probably generic for many large, sparse networksfound in nature.
We now investigate the functional significance of small-worldconnectivity for dynamical systems. Our test case is a deliberatelysimplified model for the spread of an infectious disease. Thepopulation structure is modelled by the family of graphs describedin Fig. 1. At time t ! 0, a single infective individual is introducedinto an otherwise healthy population. Infective individuals areremoved permanently (by immunity or death) after a period ofsickness that lasts one unit of dimensionless time. During this time,each infective individual can infect each of its healthy neighbourswith probability r. On subsequent time steps, the disease spreadsalong the edges of the graph until it either infects the entirepopulation, or it dies out, having infected some fraction of thepopulation in the process.
p = 0 p = 1 Increasing randomness
Regular Small-world Random
Figure 1 Random rewiring procedure for interpolating between a regular ring
lattice and a random network, without altering the number of vertices or edges in
the graph. We start with a ring of n vertices, each connected to its k nearest
neighbours by undirected edges. (For clarity, n ! 20 and k ! 4 in the schematic
examples shown here, but much larger n and k are used in the rest of this Letter.)
We choose a vertex and the edge that connects it to its nearest neighbour in a
clockwise sense. With probability p, we reconnect this edge to a vertex chosen
uniformly at random over the entire ring, with duplicate edges forbidden; other-
wise we leave the edge in place. We repeat this process by moving clockwise
around the ring, considering each vertex in turn until one lap is completed. Next,
we consider the edges that connect vertices to their second-nearest neighbours
clockwise. As before, we randomly rewire each of these edges with probability p,
and continue this process, circulating around the ring and proceeding outward to
more distant neighbours after each lap, until each edge in the original lattice has
been considered once. (As there are nk/2 edges in the entire graph, the rewiring
process stops after k/2 laps.) Three realizations of this process are shown, for
different values of p. For p ! 0, the original ring is unchanged; as p increases, the
graph becomes increasingly disordered until for p ! 1, all edges are rewired
randomly. One of our main results is that for intermediate values of p, the graph is
a small-world network: highly clustered like a regular graph, yet with small
characteristic path length, like a random graph. (See Fig. 2.)
Table 1 Empirical examples of small-world networks
Lactual Lrandom Cactual Crandom.............................................................................................................................................................................Film actors 3.65 2.99 0.79 0.00027Power grid 18.7 12.4 0.080 0.005C. elegans 2.65 2.25 0.28 0.05.............................................................................................................................................................................Characteristic path length L and clustering coefficient C for three real networks, comparedto random graphs with the same number of vertices (n) and average number of edges pervertex (k). (Actors: n ! 225;226, k ! 61. Power grid: n ! 4;941, k ! 2:67. C. elegans: n ! 282,k ! 14.) The graphs are defined as follows. Two actors are joined by an edge if they haveacted in a film together. We restrict attention to the giant connected component16 of thisgraph, which includes !90% of all actors listed in the Internet Movie Database (available athttp://us.imdb.com), as of April 1997. For the power grid, vertices represent generators,transformers and substations, and edges represent high-voltage transmission linesbetween them. For C. elegans, an edge joins two neurons if they are connected by eithera synapse or a gap junction. We treat all edges as undirected and unweighted, and allvertices as identical, recognizing that these are crude approximations. All three networksshow the small-world phenomenon: L ! Lrandom but C q Crandom.
0
0.2
0.4
0.6
0.8
1
0.0001 0.001 0.01 0.1 1
p
L(p) / L(0)
C(p) / C(0)
Figure 2 Characteristic path length L(p) and clustering coefficient C(p) for the
family of randomly rewired graphs described in Fig. 1. Here L is defined as the
number of edges in the shortest path between two vertices, averaged over all
pairs of vertices. The clustering coefficient C(p) is defined as follows. Suppose
that a vertex v has kv neighbours; then at most kv"kv " 1#=2 edges can exist
between them (this occurs when every neighbour of v is connected to everyother
neighbour of v). Let Cv denote the fraction of these allowable edges that actually
exist. Define C as the average of Cv over all v. For friendship networks, these
statistics have intuitive meanings: L is the average number of friendships in the
shortest chain connecting two people; Cv reflects the extent to which friends of v
are also friends of each other; and thus C measures the cliquishness of a typical
friendship circle. The data shown in the figure are averages over 20 random
realizations of the rewiring process described in Fig.1, and have been normalized
by the values L(0), C(0) for a regular lattice. All the graphs have n ! 1;000 vertices
and an average degree of k ! 10 edges per vertex. We note that a logarithmic
horizontal scale has been used to resolve the rapid drop in L(p), corresponding to
the onset of the small-world phenomenon. During this drop, C(p) remains almost
constant at its value for the regular lattice, indicating that the transition to a small
world is almost undetectable at the local level.
• Small-worlds have small diameter and large clusteringcoefficient.
• They are remarkably easy to generate (just a tiny p required).
Nature © Macmillan Publishers Ltd 1998
8
letters to nature
NATURE | VOL 393 | 4 JUNE 1998 441
removed from a clustered neighbourhood to make a short cut has, atmost, a linear effect on C; hence C(p) remains practically unchangedfor small p even though L(p) drops rapidly. The important implica-tion here is that at the local level (as reflected by C(p)), the transitionto a small world is almost undetectable. To check the robustness ofthese results, we have tested many different types of initial regulargraphs, as well as different algorithms for random rewiring, and allgive qualitatively similar results. The only requirement is that therewired edges must typically connect vertices that would otherwisebe much farther apart than Lrandom.
The idealized construction above reveals the key role of shortcuts. It suggests that the small-world phenomenon might becommon in sparse networks with many vertices, as even a tinyfraction of short cuts would suffice. To test this idea, we havecomputed L and C for the collaboration graph of actors in featurefilms (generated from data available at http://us.imdb.com), theelectrical power grid of the western United States, and the neuralnetwork of the nematode worm C. elegans17. All three graphs are ofscientific interest. The graph of film actors is a surrogate for a socialnetwork18, with the advantage of being much more easily specified.It is also akin to the graph of mathematical collaborations centred,traditionally, on P. Erdos (partial data available at http://www.acs.oakland.edu/!grossman/erdoshp.html). The graph ofthe power grid is relevant to the efficiency and robustness ofpower networks19. And C. elegans is the sole example of a completelymapped neural network.
Table 1 shows that all three graphs are small-world networks.These examples were not hand-picked; they were chosen because oftheir inherent interest and because complete wiring diagrams wereavailable. Thus the small-world phenomenon is not merely acuriosity of social networks13,14 nor an artefact of an idealized
model—it is probably generic for many large, sparse networksfound in nature.
We now investigate the functional significance of small-worldconnectivity for dynamical systems. Our test case is a deliberatelysimplified model for the spread of an infectious disease. Thepopulation structure is modelled by the family of graphs describedin Fig. 1. At time t ! 0, a single infective individual is introducedinto an otherwise healthy population. Infective individuals areremoved permanently (by immunity or death) after a period ofsickness that lasts one unit of dimensionless time. During this time,each infective individual can infect each of its healthy neighbourswith probability r. On subsequent time steps, the disease spreadsalong the edges of the graph until it either infects the entirepopulation, or it dies out, having infected some fraction of thepopulation in the process.
p = 0 p = 1 Increasing randomness
Regular Small-world Random
Figure 1 Random rewiring procedure for interpolating between a regular ring
lattice and a random network, without altering the number of vertices or edges in
the graph. We start with a ring of n vertices, each connected to its k nearest
neighbours by undirected edges. (For clarity, n ! 20 and k ! 4 in the schematic
examples shown here, but much larger n and k are used in the rest of this Letter.)
We choose a vertex and the edge that connects it to its nearest neighbour in a
clockwise sense. With probability p, we reconnect this edge to a vertex chosen
uniformly at random over the entire ring, with duplicate edges forbidden; other-
wise we leave the edge in place. We repeat this process by moving clockwise
around the ring, considering each vertex in turn until one lap is completed. Next,
we consider the edges that connect vertices to their second-nearest neighbours
clockwise. As before, we randomly rewire each of these edges with probability p,
and continue this process, circulating around the ring and proceeding outward to
more distant neighbours after each lap, until each edge in the original lattice has
been considered once. (As there are nk/2 edges in the entire graph, the rewiring
process stops after k/2 laps.) Three realizations of this process are shown, for
different values of p. For p ! 0, the original ring is unchanged; as p increases, the
graph becomes increasingly disordered until for p ! 1, all edges are rewired
randomly. One of our main results is that for intermediate values of p, the graph is
a small-world network: highly clustered like a regular graph, yet with small
characteristic path length, like a random graph. (See Fig. 2.)
Table 1 Empirical examples of small-world networks
Lactual Lrandom Cactual Crandom.............................................................................................................................................................................Film actors 3.65 2.99 0.79 0.00027Power grid 18.7 12.4 0.080 0.005C. elegans 2.65 2.25 0.28 0.05.............................................................................................................................................................................Characteristic path length L and clustering coefficient C for three real networks, comparedto random graphs with the same number of vertices (n) and average number of edges pervertex (k). (Actors: n ! 225;226, k ! 61. Power grid: n ! 4;941, k ! 2:67. C. elegans: n ! 282,k ! 14.) The graphs are defined as follows. Two actors are joined by an edge if they haveacted in a film together. We restrict attention to the giant connected component16 of thisgraph, which includes !90% of all actors listed in the Internet Movie Database (available athttp://us.imdb.com), as of April 1997. For the power grid, vertices represent generators,transformers and substations, and edges represent high-voltage transmission linesbetween them. For C. elegans, an edge joins two neurons if they are connected by eithera synapse or a gap junction. We treat all edges as undirected and unweighted, and allvertices as identical, recognizing that these are crude approximations. All three networksshow the small-world phenomenon: L ! Lrandom but C q Crandom.
0
0.2
0.4
0.6
0.8
1
0.0001 0.001 0.01 0.1 1
p
L(p) / L(0)
C(p) / C(0)
Figure 2 Characteristic path length L(p) and clustering coefficient C(p) for the
family of randomly rewired graphs described in Fig. 1. Here L is defined as the
number of edges in the shortest path between two vertices, averaged over all
pairs of vertices. The clustering coefficient C(p) is defined as follows. Suppose
that a vertex v has kv neighbours; then at most kv"kv " 1#=2 edges can exist
between them (this occurs when every neighbour of v is connected to everyother
neighbour of v). Let Cv denote the fraction of these allowable edges that actually
exist. Define C as the average of Cv over all v. For friendship networks, these
statistics have intuitive meanings: L is the average number of friendships in the
shortest chain connecting two people; Cv reflects the extent to which friends of v
are also friends of each other; and thus C measures the cliquishness of a typical
friendship circle. The data shown in the figure are averages over 20 random
realizations of the rewiring process described in Fig.1, and have been normalized
by the values L(0), C(0) for a regular lattice. All the graphs have n ! 1;000 vertices
and an average degree of k ! 10 edges per vertex. We note that a logarithmic
horizontal scale has been used to resolve the rapid drop in L(p), corresponding to
the onset of the small-world phenomenon. During this drop, C(p) remains almost
constant at its value for the regular lattice, indicating that the transition to a small
world is almost undetectable at the local level.
Infection rates on Small-worlds
Nature © Macmillan Publishers Ltd 1998
8
letters to nature
442 NATURE | VOL 393 | 4 JUNE 1998
Two results emerge. First, the critical infectiousness rhalf, at whichthe disease infects half the population, decreases rapidly for small p(Fig. 3a). Second, for a disease that is sufficiently infectious to infectthe entire population regardless of its structure, the time T(p)required for global infection resembles the L(p) curve (Fig. 3b).Thus, infectious diseases are predicted to spread much more easilyand quickly in a small world; the alarming and less obvious point ishow few short cuts are needed to make the world small.
Our model differs in some significant ways from other networkmodels of disease spreading20–24. All the models indicate that net-work structure influences the speed and extent of disease transmis-sion, but our model illuminates the dynamics as an explicit functionof structure (Fig. 3), rather than for a few particular topologies, suchas random graphs, stars and chains20–23. In the work closest to ours,Kretschmar and Morris24 have shown that increases in the numberof concurrent partnerships can significantly accelerate the propaga-tion of a sexually-transmitted disease that spreads along the edges ofa graph. All their graphs are disconnected because they fix theaverage number of partners per person at k ! 1. An increase in thenumber of concurrent partnerships causes faster spreading byincreasing the number of vertices in the graph’s largest connectedcomponent. In contrast, all our graphs are connected; hence thepredicted changes in the spreading dynamics are due to more subtlestructural features than changes in connectedness. Moreover,
changes in the number of concurrent partners are obvious to anindividual, whereas transitions leading to a smaller world are not.
We have also examined the effect of small-world connectivity onthree other dynamical systems. In each case, the elements werecoupled according to the family of graphs described in Fig. 1. (1) Forcellular automata charged with the computational task of densityclassification25, we find that a simple ‘majority-rule’ running on asmall-world graph can outperform all known human and geneticalgorithm-generated rules running on a ring lattice. (2) For theiterated, multi-player ‘Prisoner’s dilemma’11 played on a graph, wefind that as the fraction of short cuts increases, cooperation is lesslikely to emerge in a population of players using a generalized ‘tit-for-tat’26 strategy. The likelihood of cooperative strategies evolvingout of an initial cooperative/non-cooperative mix also decreaseswith increasing p. (3) Small-world networks of coupled phaseoscillators synchronize almost as readily as in the mean-fieldmodel2, despite having orders of magnitude fewer edges. Thisresult may be relevant to the observed synchronization of widelyseparated neurons in the visual cortex27 if, as seems plausible, thebrain has a small-world architecture.
We hope that our work will stimulate further studies of small-world networks. Their distinctive combination of high clusteringwith short characteristic path length cannot be captured bytraditional approximations such as those based on regular latticesor random graphs. Although small-world architecture has notreceived much attention, we suggest that it will probably turn outto be widespread in biological, social and man-made systems, oftenwith important dynamical consequences. !
Received 27 November 1997; accepted 6 April 1998.
1. Winfree, A. T. The Geometry of Biological Time (Springer, New York, 1980).2. Kuramoto, Y. Chemical Oscillations, Waves, and Turbulence (Springer, Berlin, 1984).3. Strogatz, S. H. & Stewart, I. Coupled oscillators and biological synchronization. Sci. Am. 269(6), 102–
109 (1993).4. Bressloff, P. C., Coombes, S. & De Souza, B. Dynamics of a ring of pulse-coupled oscillators: a group
theoretic approach. Phys. Rev. Lett. 79, 2791–2794 (1997).5. Braiman, Y., Lindner, J. F. & Ditto, W. L. Taming spatiotemporal chaos with disorder. Nature 378,
465–467 (1995).6. Wiesenfeld, K. New results on frequency-locking dynamics of disordered Josephson arrays. Physica B
222, 315–319 (1996).7. Gerhardt, M., Schuster, H. & Tyson, J. J. A cellular automaton model of excitable media including
curvature and dispersion. Science 247, 1563–1566 (1990).8. Collins, J. J., Chow, C. C. & Imhoff, T. T. Stochastic resonance without tuning. Nature 376, 236–238
(1995).9. Hopfield, J. J. & Herz, A. V. M. Rapid local synchronization of action potentials: Toward computation
with coupled integrate-and-fire neurons. Proc. Natl Acad. Sci. USA 92, 6655–6662 (1995).10. Abbott, L. F. & van Vreeswijk, C. Asynchronous states in neural networks of pulse-coupled oscillators.
Phys. Rev. E 48(2), 1483–1490 (1993).11. Nowak, M. A. & May, R. M. Evolutionary games and spatial chaos. Nature 359, 826–829 (1992).12. Kauffman, S. A. Metabolic stability and epigenesis in randomly constructed genetic nets. J. Theor. Biol.
22, 437–467 (1969).13. Milgram, S. The small world problem. Psychol. Today 2, 60–67 (1967).14. Kochen, M. (ed.) The Small World (Ablex, Norwood, NJ, 1989).15. Guare, J. Six Degrees of Separation: A Play (Vintage Books, New York, 1990).16. Bollabas, B. Random Graphs (Academic, London, 1985).17. Achacoso, T. B. & Yamamoto, W. S. AY’s Neuroanatomy of C. elegans for Computation (CRC Press, Boca
Raton, FL, 1992).18. Wasserman, S. & Faust, K. Social Network Analysis: Methods and Applications (Cambridge Univ. Press,
1994).19. Phadke, A. G. & Thorp, J. S. Computer Relaying for Power Systems (Wiley, New York, 1988).20. Sattenspiel, L. & Simon, C. P. The spread and persistence of infectious diseases in structured
populations. Math. Biosci. 90, 341–366 (1988).21. Longini, I. M. Jr A mathematical model for predicting the geographic spread of new infectious agents.
Math. Biosci. 90, 367–383 (1988).22. Hess, G. Disease in metapopulation models: implications for conservation. Ecology 77, 1617–1632
(1996).23. Blythe, S. P., Castillo-Chavez, C. & Palmer, J. S. Toward a unified theory of sexual mixing and pair
formation. Math. Biosci. 107, 379–405 (1991).24. Kretschmar, M. & Morris, M. Measures of concurrency in networks and the spread of infectious
disease. Math. Biosci. 133, 165–195 (1996).25. Das, R., Mitchell, M. & Crutchfield, J. P. in Parallel Problem Solving from Nature (eds Davido, Y.,
Schwefel, H.-P. & Manner, R.) 344–353 (Lecture Notes in Computer Science 866, Springer, Berlin,1994).
26. Axelrod, R. The Evolution of Cooperation (Basic Books, New York, 1984).27. Gray, C. M., Konig, P., Engel, A. K. & Singer, W. Oscillatory responses in cat visual cortex exhibit inter-
columnar synchronization which reflects global stimulus properties. Nature 338, 334–337 (1989).
Acknowledgements. We thank B. Tjaden for providing the film actor data, and J. Thorp and K. Bae for theWestern States Power Grid data. This work was supported by the US National Science Foundation(Division of Mathematical Sciences).
Correspondence and requests for materials should be addressed to D.J.W. (e-mail: [email protected]).
0.15
0.2
0.25
0.3
0.35
0.0001 0.001 0.01 0.1 1
rhalf
p
a
Figure 3 Simulation results for a simple model of disease spreading. The
community structure is given by one realization of the family of randomly rewired
graphs used in Fig. 1. a, Critical infectiousness rhalf, at which the disease infects
half the population, decreases with p. b, The time T(p) required for a maximally
infectious disease (r ! 1) to spread throughout the entire population has essen-
tially the same functional form as the characteristic path length L(p). Even if only a
few per cent of the edges in the original lattice are randomly rewired, the time to
global infection is nearly as short as for a random graph.
0
0.2
0.4
0.6
0.8
1
0.0001 0.001 0.01 0.1 1
T(p) /T(0)
L(p) /L(0)
p
b• Start with one infected individual.
• r is transmission rate of disease (infect neighbors at rate r).
• rhalf is the the value of r required for half the population to getthe disease.
Their concluding words
“We hope that our work will stimulate further studies of small-world networks. Their distinctive combination of high clustering
with short characteristic path length cannot be captured bytraditional approximations such as those based on regular
lattices or random graphs. Although small-world architecturehas not received much attention, we suggest that it will probably
turn out to be widespread in biological, social and man-madesystems,often with important dynamical consequences.”
Watts-Strogatz small-world model
• Together with Barabasi-Albert launched the flurry of activity onnetworks.
• Watts and Strogatz showed that networks from both the naturaland manmade world, such as the neural network of C. elegansand power grids, exhibit the small-world property.
• Originally they wanted to understand the synchronization ofcricket chirps.
• Introduced the mathematical formalism, which interpolatesbetween lattices and networks .
A new paradigm
”I think I’ve been contacted by someone from just about everyfield outside of English literature. I’ve had letters from
mathematicians, physicists, biochemists, neurophysiologists,epidemiologists, economists, sociologists; from people in
marketing, information systems, civil engineering, and from abusiness enterprise that uses the concept of the small world for
networking purposes on the Internet.” – Duncan Watts
Navigation
Clearly if central coordination, can use short paths to deliver infoquickly.
But, can someone living in a small world actually make use ofthis info and do efficient decentralized routing?
• Instead of designing search algorithms, given a local greedyalgorithm, are there any topologies that enable O(logN)
delivery times?
Precise topologies required
[J. M. Kleinberg, “Navigation in a small world”, Nature, 406(2000)]
• Start with a regular 2D square lattice (consider vertices andedges).
• Add random long links, with bias proportional to distancebetween two nodes:
p(eij) ∝ 1/dαij
• Call α the “clustering exponent”
The small-world phenomenon — theprinciple that most of us are linked byshort chains of acquaintances — was
first investigated as a question in sociolo-gy1,2 and is a feature of a range of networksarising in nature and technology3–5. Experi-mental study of the phenomenon1 revealedthat it has two fundamental components:first, such short chains are ubiquitous, andsecond, individuals operating with purelylocal information are very adept at findingthese chains. The first issue has beenanalysed2–4, and here I investigate the sec-ond by modelling how individuals can findshort chains in a large social network.
I have found that the cues needed fordiscovering short chains emerge in a verysimple network model. This model is basedon early experiments1, in which source indi-viduals in Nebraska attempted to transmit aletter to a target in Massachusetts, with theletter being forwarded at each step to some-one the holder knew on a first-name basis.The networks underlying the model followthe ‘small-world’ paradigm3: they are richin structured short-range connections andhave a few random long-range connections.
Long-range connections are added to atwo-dimensional lattice controlled by aclustering exponent, !, that determines theprobability of a connection between twonodes as a function of their lattice distance(Fig. 1a). Decentralized algorithms arestudied for transmitting a message: at eachstep, the holder of the message must pass itacross one of its short- or long-range con-nections; crucially, this current holder doesnot know the long-range connections ofnodes that have not touched the message.The primary figure of merit for such analgorithm is its expected delivery time T,which represents the expected number ofsteps needed to forward a message betweena random source and target in a networkgenerated according to the model. It is cru-cial to constrain the algorithm to use onlylocal information — with global knowledgeof all connections in the network, the short-est chain can be found very simply6.
A characteristic feature of small-worldnetworks is that their diameter is exponen-tially smaller than their size, being boundedby a polynomial in logN, where N is thenumber of nodes. In other words, there isalways a very short path between any twonodes. This does not imply, however, that adecentralized algorithm will be able to dis-cover such short paths. My central findingis that there is in fact a unique value of theexponent ! at which this is possible.
When !"2, so that long-range connec-
tions follow an inverse-square distribution,there is a decentralized algorithm thatachieves a very rapid delivery time; T isbounded by a function proportional to(logN)2. The algorithm achieving thisbound is a ‘greedy’ heuristic: each messageholder forwards the message across a con-
nection that brings it as close as possible tothe target in lattice distance. Moreover,!"2 is the only exponent at which anydecentralized algorithm can achieve a deliv-ery time bounded by any polynomial inlogN: for every other exponent, an asymp-totically much larger delivery time isrequired, regardless of the algorithmemployed (Fig. 1b).
These results indicate that efficient navi-gability is a fundamental property of onlysome small-world structures. The resultsalso generalize to d-dimensional lattices forany value of d#1, with the critical value ofthe clustering exponent becoming !"d.Simulations of the greedy algorithm yieldresults that are qualitatively consistent withthe asymptotic analytical bounds (Fig. 1c).
In the areas of communication net-works7 and neuroanatomy8, the issue ofrouting without a global network organiza-tion has been considered; also in social psy-chology and information foraging some ofthe cues that individuals use to constructpaths through a social network or hyper-linked environment have been discovered9,10.Although I have focused on a very cleanmodel, I believe that a more general conclu-sion can be drawn for small-world networks— namely that the correlation between localstructure and long-range connections pro-vides critical cues for finding paths throughthe network.
When this correlation is near a criticalthreshold, the structure of the long-rangeconnections forms a type of gradient thatallows individuals to guide a message effi-ciently towards a target. As the correlationdrops below this critical value and the socialnetwork becomes more homogeneous,these cues begin to disappear; in the limit,when long-range connections are generateduniformly at random, the result is a worldin which short chains exist but individuals,faced with a disorienting array of socialcontacts, are unable to find them.Jon M. KleinbergDepartment of Computer Science, CornellUniversity, Ithaca, New York 14853, USA1. Milgram, S. Psychol. Today 1, 61–67 (1967).2. Kochen, M. (ed.) The Small World (Ablex, Norwood, NJ, 1989).3. Watts, D. & Strogatz, S. Nature 393, 440–442 (1998).4. Albert, R. et al. Nature 401, 130–131 (1999).5. Adamic, L. in Proc. 3rd European Conference on Digital Libraries
(eds Abiteboul, S. & Vercoustre, A.-M.) 443–452 (SpringerLecture Notes in Computer Science, Vol. 1696, Berlin, 1999).
6. Cormen, T., Leiserson, C. & Rivest, R. Introduction toAlgorithms (McGraw-Hill, Boston, 1990).
7. Peleg, D. & Upfal, E. J. Assoc. Comput. Machinery 36, 510–530(1989).
8. Braitenberg, V. & Schüz, A. Anatomy of the Cortex (Springer,Berlin, 1991).
9. Killworth, P. & Bernard, H. Social Networks 1, 159–192 (1978).10.Pirolli, P. & Card, S. Psychol. Rev. 106, 643–675 (1999).
brief communications
NATURE | VOL 406 | 24 AUGUST 2000 | www.nature.com 845
Navigation in a small worldIt is easier to find short chains between points in some networks than others.
00 1 2 3 4
0.2
Clustering exponent (!)
Clustering exponent (!)
0.8
0.6
0.4.
(!–2)/(!–1)(2–!)/3
7.0
5.0
6.0
1 0 2
ua c
d
bv
a
b
c
Expo
nent
" in
low
er b
ound
on
TIn
T fo
r gr
eedy
alg
orith
m
Figure 1 The navigability of small-world networks. a, The network
model is derived from an n$n lattice. Each node, u, has a short-
range connection to its nearest neighbours (a, b, c and d ) and a
long-range connection to a randomly chosen node, where node vis selected with probability proportional to r%!, where r is the lat-
tice (‘Manhattan’) distance between u and v, and !#0 is a fixed
clustering exponent. More generally, for p,q#1, each node u has
a short-range connection to all nodes within p lattice steps, and qlong-range connections generated independently from a distribu-
tion with clustering exponent !. b, Lower bound from my charac-
terization theorem: when ! ! 2, the expected delivery time Tof any decentralized algorithm satisfies T#cn&, where
&"(2%!)/3 for 0'!(2 and &"(!%2)/(!%1) for !)2,
and where c depends on !, p and q, but not n. c, Simulation of
the greedy algorithm on a 20,000$20,000 toroidal lattice, with
random long-range connections as in a. Each data point is the
average of 1,000 runs.
© 2000 Macmillan Magazines Ltd
• Find mean delivery time t ∼ Nβ, unless α = 2.
• Only for α = 2 will decentralized routing work, and packet cango from source to destination in O(logN) steps.
The small-world phenomenon — theprinciple that most of us are linked byshort chains of acquaintances — was
first investigated as a question in sociolo-gy1,2 and is a feature of a range of networksarising in nature and technology3–5. Experi-mental study of the phenomenon1 revealedthat it has two fundamental components:first, such short chains are ubiquitous, andsecond, individuals operating with purelylocal information are very adept at findingthese chains. The first issue has beenanalysed2–4, and here I investigate the sec-ond by modelling how individuals can findshort chains in a large social network.
I have found that the cues needed fordiscovering short chains emerge in a verysimple network model. This model is basedon early experiments1, in which source indi-viduals in Nebraska attempted to transmit aletter to a target in Massachusetts, with theletter being forwarded at each step to some-one the holder knew on a first-name basis.The networks underlying the model followthe ‘small-world’ paradigm3: they are richin structured short-range connections andhave a few random long-range connections.
Long-range connections are added to atwo-dimensional lattice controlled by aclustering exponent, !, that determines theprobability of a connection between twonodes as a function of their lattice distance(Fig. 1a). Decentralized algorithms arestudied for transmitting a message: at eachstep, the holder of the message must pass itacross one of its short- or long-range con-nections; crucially, this current holder doesnot know the long-range connections ofnodes that have not touched the message.The primary figure of merit for such analgorithm is its expected delivery time T,which represents the expected number ofsteps needed to forward a message betweena random source and target in a networkgenerated according to the model. It is cru-cial to constrain the algorithm to use onlylocal information — with global knowledgeof all connections in the network, the short-est chain can be found very simply6.
A characteristic feature of small-worldnetworks is that their diameter is exponen-tially smaller than their size, being boundedby a polynomial in logN, where N is thenumber of nodes. In other words, there isalways a very short path between any twonodes. This does not imply, however, that adecentralized algorithm will be able to dis-cover such short paths. My central findingis that there is in fact a unique value of theexponent ! at which this is possible.
When !"2, so that long-range connec-
tions follow an inverse-square distribution,there is a decentralized algorithm thatachieves a very rapid delivery time; T isbounded by a function proportional to(logN)2. The algorithm achieving thisbound is a ‘greedy’ heuristic: each messageholder forwards the message across a con-
nection that brings it as close as possible tothe target in lattice distance. Moreover,!"2 is the only exponent at which anydecentralized algorithm can achieve a deliv-ery time bounded by any polynomial inlogN: for every other exponent, an asymp-totically much larger delivery time isrequired, regardless of the algorithmemployed (Fig. 1b).
These results indicate that efficient navi-gability is a fundamental property of onlysome small-world structures. The resultsalso generalize to d-dimensional lattices forany value of d#1, with the critical value ofthe clustering exponent becoming !"d.Simulations of the greedy algorithm yieldresults that are qualitatively consistent withthe asymptotic analytical bounds (Fig. 1c).
In the areas of communication net-works7 and neuroanatomy8, the issue ofrouting without a global network organiza-tion has been considered; also in social psy-chology and information foraging some ofthe cues that individuals use to constructpaths through a social network or hyper-linked environment have been discovered9,10.Although I have focused on a very cleanmodel, I believe that a more general conclu-sion can be drawn for small-world networks— namely that the correlation between localstructure and long-range connections pro-vides critical cues for finding paths throughthe network.
When this correlation is near a criticalthreshold, the structure of the long-rangeconnections forms a type of gradient thatallows individuals to guide a message effi-ciently towards a target. As the correlationdrops below this critical value and the socialnetwork becomes more homogeneous,these cues begin to disappear; in the limit,when long-range connections are generateduniformly at random, the result is a worldin which short chains exist but individuals,faced with a disorienting array of socialcontacts, are unable to find them.Jon M. KleinbergDepartment of Computer Science, CornellUniversity, Ithaca, New York 14853, USA1. Milgram, S. Psychol. Today 1, 61–67 (1967).2. Kochen, M. (ed.) The Small World (Ablex, Norwood, NJ, 1989).3. Watts, D. & Strogatz, S. Nature 393, 440–442 (1998).4. Albert, R. et al. Nature 401, 130–131 (1999).5. Adamic, L. in Proc. 3rd European Conference on Digital Libraries
(eds Abiteboul, S. & Vercoustre, A.-M.) 443–452 (SpringerLecture Notes in Computer Science, Vol. 1696, Berlin, 1999).
6. Cormen, T., Leiserson, C. & Rivest, R. Introduction toAlgorithms (McGraw-Hill, Boston, 1990).
7. Peleg, D. & Upfal, E. J. Assoc. Comput. Machinery 36, 510–530(1989).
8. Braitenberg, V. & Schüz, A. Anatomy of the Cortex (Springer,Berlin, 1991).
9. Killworth, P. & Bernard, H. Social Networks 1, 159–192 (1978).10.Pirolli, P. & Card, S. Psychol. Rev. 106, 643–675 (1999).
brief communications
NATURE | VOL 406 | 24 AUGUST 2000 | www.nature.com 845
Navigation in a small worldIt is easier to find short chains between points in some networks than others.
00 1 2 3 4
0.2
Clustering exponent (!)
Clustering exponent (!)
0.8
0.6
0.4.
(!–2)/(!–1)(2–!)/3
7.0
5.0
6.0
1 0 2
ua c
d
bv
a
b
c
Expo
nent
" in
low
er b
ound
on
TIn
T fo
r gr
eedy
alg
orith
m
Figure 1 The navigability of small-world networks. a, The network
model is derived from an n$n lattice. Each node, u, has a short-
range connection to its nearest neighbours (a, b, c and d ) and a
long-range connection to a randomly chosen node, where node vis selected with probability proportional to r%!, where r is the lat-
tice (‘Manhattan’) distance between u and v, and !#0 is a fixed
clustering exponent. More generally, for p,q#1, each node u has
a short-range connection to all nodes within p lattice steps, and qlong-range connections generated independently from a distribu-
tion with clustering exponent !. b, Lower bound from my charac-
terization theorem: when ! ! 2, the expected delivery time Tof any decentralized algorithm satisfies T#cn&, where
&"(2%!)/3 for 0'!(2 and &"(!%2)/(!%1) for !)2,
and where c depends on !, p and q, but not n. c, Simulation of
the greedy algorithm on a 20,000$20,000 toroidal lattice, with
random long-range connections as in a. Each data point is the
average of 1,000 runs.
© 2000 Macmillan Magazines Ltd
• For d-dimensional lattice need α = d.
But we know greedy decentralized routing works for humannetworks (c.f. Milgram’s experiments “six-degrees of separation”[ S. Milgram, “The small world problem”, Psych. Today, 2, 1967.]
So how do we get beyond a lattice model?
Navigating social networks
[Watts, P. S. Dodds, and M. E. J. Newman, “Identity and searchin social networks”, Science, 296 (2002)]
[Kleinberg, “Small world phenomena and the dynamics ofinformation”, in Proceedings of NIPS 2001].
• Premise: people navigate social networks by looking forcommon features between there acquaintances and thetargets (occupation, city inhabited, age, ....)
• Brings in DATA!
Hierarchical “social distance” tree
• Individuals are grouped into categories along many attributes.
• One tree for each attribute.
• Trees are not the network, but complementary mentalconstructs believed to be at work.
• Assume likelyhood of acquaintance falls of exponentially with“social distance”.
Building P2P architectures
• “Chord A Scalable Peer-to- peer Lookup Service for InternetApplications
I. Stoica, R. Morris, D. Karger, F. Kaashoek, H. Balakrishnan,ACM SIGCOMM, 2001.(Cited 4825 times)
• Gnutella
“Peer-to-Peer Architecture Case Study: Gnutella Network”, MRipeanu, Proceedings of International Conference on Peer-to-peer Computing, 2001.
Summary
Web search
• Centralized
• Make use of link structure (topology)
Decentralized search
• Efficiency/Speed depends on underlying topology
• Gossip algorithms (D. Kempe and J. Kleinberg): spreadingshared information quickly through local exchanges (e.g.,sums, local averages/consensus).
• Applications to sensor networks .... communications ...satellites
