Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
Linear Operators on CorrelatedLandscapesPeter F. Stadler
SFI WORKING PAPER: 1993-06-042
SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent theviews of the Santa Fe Institute. We accept papers intended for publication in peer-reviewed journals or proceedings volumes, but not papers that have already appeared in print. Except for papers by our externalfaculty, papers must be based on work done at SFI, inspired by an invited visit to or collaboration at SFI, orfunded by an SFI grant.©NOTICE: This working paper is included by permission of the contributing author(s) as a means to ensuretimely distribution of the scholarly and technical work on a non-commercial basis. Copyright and all rightstherein are maintained by the author(s). It is understood that all persons copying this information willadhere to the terms and constraints invoked by each author's copyright. These works may be reposted onlywith the explicit permission of the copyright holder.www.santafe.edu
SANTA FE INSTITUTE
Linear Operators on Correlated Landscapes
By
PETER F. STADLER
Santa Fe Institute*
1660 Old Pecos Trail, Santa Fe, NM 87501, USA
Institute for Theoretical Chemistry, University of Vienna
WiihringerstraBe 17, A-lOgO Vienna, Austria
Email: [email protected] or [email protected]
Address for Correspondence. Phone *(505)984-8800, Fax *(505)982-0565
PACS numbers: 0250,0550
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
Abstract
In this contribution we consider the effect of a class of "averaging op
erators" on isotropic fitness landscapes. Explicit expressions for the cor
relation function of the averaged landscapes are derived. A new class of
tuneably rugged landscapes, obtained by iterated smoothing of the random
energy model, is established. The correlation structure of certain landscapes,
among them the Sherrington-Kirkpatrick model, remains unchanged under
the action of all averaging operators considered here.
1. Introduction
Evolutionary adaptation as well as combinatorial optimization take place
on "landscapes" that result from mapping (micro)configurations to scalar
quantities like fitness values or energies, or, more generally, to nonscalar
objects like structures. The assumption of a particular mechanism for muta-
tion, or the choice of a specific move-set for interconverting configurations,
introduces the notions of neighborhood and distance between configurations.
Viewing the set of all configurations, i.e., the configuration space, as a non
directed graph r has turned out to be particularly useful. Each configura
tion is a vertex of r, and neighboring configurations are connected by edges.
When configurations are sequences of equal length and the neighborhood
relations are defined by the number of differing positions, one obtains the
sequence space with the Hamming metric.
Among the most important properties of a landscape is its "rugged
ness", which influences the speed of evolutionary adaptation as well as the
quality of solutions of heuristic optimization strategies. While ruggedness
has never been precisely defined, a number of empirical measures have been
proposed, such as the number of local optima or the average length of up
or downhill walks (Kauffman and Levin 1987, Weinberger 1991a), and the
-1-
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
pair-correlation as a function of distance in configuration space (Eigen et al.
1989, Weinberger 1990).
The very notion of "uphill" or "optimum" implies a mapping to a scalar
quantity. The definition of a pair correlation, however, can be extended to
mappings to arbitrary objects, as long as there is a metric distance mea
sure defined for them. Extensive studies on folding of RNA sequences into
secondary structures have been based on this fact (Fontana et al. 1993ab).
The landscapes of a number of well known combinatorial optimization
problems, such as the Travelling Salesman Problem (Stadler and Schnabl
1992), the Graph Bipartitioning Problem (Stadler and Happel 1992), and the
Graph Matching Problem (Stadler 1992) have been investigated. Detailed
information on the distribution of local optima and the statistical character
istics of downhill walks have been obtained for the uncorrelated landscape
of the random energy model (Macken et al. 1991, Flyvbjerg and Lautrup
1993, Bak et al. 1993). Furthermore, two one-parameter families of tuneably
rugged landscapes have been studied in detail: the Nk model and its variants
(Kauffman and Levin 1987, Kauffman et al. 1988, Weinberger 1991a, Fontana
et al. 1993a), and the p-spin models (Amitrano et al. 1989, Weinberger and
Stadler 1993).
Here we will be concerned with the effect of averaging procedures related
to running averages on landscapes. After some formal definitions (sect. 2)
we discuss the Fourier decomposition of landscapes (sect. 3) which is used in
sect. 4 to derive the basic properties of linear operators applied to landscapes.
In sect. 5 we briefly address the relation of averages over ensembles of land
scapes and averages over configurations in single instances of landscapes. In
sect. 6 particular operators, namely weighted averages over one-step neigh
borhoods, are used to construct a novel tuneable family of landscapes. In
sect. 7 the behaviour of p-spin and Nk landscapes under the iterated action
of these operators is investigated.
-2-
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
2. Some Definitions
Let r be a graph with N ::; 00 vertices. A graph automorphism a is a
one-to-one mapping of r onto itself such that the vertices a(x) and a(y) are
connected by an edge if and only if x and yare connected by an edge. Two
vertices of r are said to be equivalent if there is an automorphism a such that
y = a(x). A graph is vertex tran8itive if all vertices are equivalent. A vertex
transitive graph is always regular, i.e., each vertex has the same number of
neighbors. The distance d(x, y) of two vertices x, y E r is defined as the
minimum number of edges that separate them. This distance is a metric. A
graph is di8tance tran8itive if for any two pairs of vertices (x,y) and (u,v)
with d(x,y) = d(u,v) there is an automorphism a such that a(x) = u and
a(y) = v. (See e.g. Buckley and Harrary 1990). A graph r is defined by its
adjacency matrix A, with A xy = 1 if the vertices x and yare connected by
an edge, and Axy = 0 otherwise.
In the following we will only consider graphs that are at least vertex
transitive. Following Bollobas (1979) we may then partition the vertex set
by the following procedure: Choose an arbitrary vertex as reference point,
which in the following will be labelled O. Then construct pairwise disjoint
sets of vertices V(O) = {O}, V(I), V(2), ... , V(M) such that
aJl":= L Axy
xEV(Jl)
is independent of y E V(v). (1)
This partitioning is induced by the symmetry of r: the vertices in each of
the sets V(t<) are permutated by automorphisms that leave 0 invariant. In
the worst case each set contains a single vertex. In general we will use greek
letters to index the sets of such a partition, while roman indices are reserved
for the vertices of r.
-3-
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
For highly symmetric graphs the collapsed adjacency matrix A = (&I'v)
can be much smaller than the adjacency matrix A. In case of distance tran
sitive graphs, for instance, the distance classes (i.e., the sets of all vertices
with common distance from the reference vertex 0) form a partition fulfilling
equ.(I), and the collapsed adjacency matrix A coincides with the intersection
matrix as defined by Biggs (1974) (see also Cvetkovic et al. 1980). We will
use d instead of greek letters for indexing the distance classes.
The following property of collapsed adjacency matrices will be used in
the subsequent sections.
Lemma 1. L (Am)xy = (Am)I'K for all x E V(p), and all non-negativeyEV(K)
intergers m, and hence for each polynomial p holds 2:YEV(K)[P(A)]xy =
[P(A)JI'K' i.e., peA) = peA).
Proof. We proceed inductively: The assertion is true by definition for m = 1.
For m = 0 it holds trivially. Now suppose it is true for m - 1; then
L (Am-IA)xy = L L L (Am-l)xzAzy =yEV(K) v zEV(v) yEV(K)
= L L (Am-I)xz LAzy = L(Am-I)I'VAVK = (Am)I'Kv zEV(v) yEV(K) v
The generalization to polynomials is straight forward. •Definition 2. A landscape! : r --4 IR. is an instance of a random field :F on
the set of vertices of r defined by the distribution function
P(yl, Yz,···, Yn) = Prob {!(Xi) ::; Vi, 1::; i ::; N}
where N is the number of vertices of r.
(2)
Let £[.] denote the mathematical expectation. For example, the ex
pected value of the product of the values !(Xk) and !(Xl) at the vertices Xk
and Xl is
£[!(Xk)!(Xl)] = JYkyldP(YI,yZ, ... ,YN).
-4-
(3)
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
Definition 3. A landscape is isotropic if
(i) E[f(x)] =! for all configurations x E rj(ii) for any two pairs of configurations (x, y) and (u, v) such that there
is a graph automorphism a with a(x) = u and a(y) = v holds
E[f(x)f(y)] = E[f(u)f(v)].
The second condition means that any two pairs of configurations that are
equivalent in configuration space have the same pair-correlation. The covari-
ance matrix of :F is defined by
Cxy = E[f(x)f(y)] - E[f(x)]£[f(y)]· (4)
By vertex transitivity there is an automorphism a such that a(x) = O.
Isotropy hence implies Cxy = CO,a(y) = c(p,), where yep,) is the partition
to which a(y) belongs. The variance is given by
Var[f] = Cxx = Coo = c(O)
The autocorrelation function is defined as pcp,) = c(p,)jc(O).
3. Fourier Series on Landscapes
(5)
Let C be the covariance matrix of a random field :F on some graph r.Denote by {<p s} a complete set of orthonormal eigenvectors (which exists by
symmetry of C). Although {<Ps} can be chosen real, we will admit complex
vectors as well. Then the landscape f on r can be represented in mean
square sense as
f(x) = La(y)<py(x) (6)yEr
This series representation is known as the Karhunen-Loeve expansion.
Remark. The coefficients a(y) are uncorrelated. For finite sets the Karhunen
Loeve expansion coincides with the well known principal component analysis
introduced by Hotelling (1933) (see, e.g., Rao 1973).
-5-
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
-- -------------------::;;-" -----.... -
..,/
0.8
0.6
0.4
0.2
,.// >:./ .
..._...... n=25--_. n=50--- n=100-- n=200
1.0 ~~~....,,-~~--.-,--~----=:::::.. _;::=_..;:;;.. :;:::~...~.....3............................
0.8
0.6
0.4
0.2
........... n=25n_. n=50--- n=100-- n=200
0.0 '::--~~~--..,~~~~~":"":-~~~~~,.--~~~~0.0 0.5 1.0 1.5 2.0
r/n
Figure 3: Nearest neighbor correlation for ISLs on generalized hypercubes as a function
of the number of iterated smoothings. a) K=2, b) K=4.
-16 -
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
7. Averaging Operators on Spin Glass and Nk Landscapes
We first consider the p-spin models, i.e., the long range spin glasses with
Hamiltonian
7-f.p (<T) = L Aj.j, ... jp <Tj, <Tj, .•. <Tjp
j. <j,<...<jp
(27)
with <Ti E {-I, +1} and coupling constants Aij ... k drawn from some common
distribution. The ruggedness of the landscapes increases as the order p of
the spin-interactions increases. We have then
Theorem 14. Let Q be any polynomial of the adjacency matrix of the
Boolean Hypercube. Then the landscapes 7-f.p and Q7-f.p have the same auto
correlation functions.
Proof. As shown in Weinberger and Stadler (1993) the autocorrelation
functions of the p-spin models are
(28)
These are suitably normalized Krawtchouk polynomials, which are known
to be the eigenvectors of the collapsed adjacency matrix of the Boolean Hy
percube (see, e.g., Dunkl 1979, Stadler 1993). Application of Corollary 11
completes the proof. •In Kauffman's Nk-model the fitness of a binary string x is defined as the
average over "site fitnesses" Ii, which depend on Xi and k additional positions
i 1 through ik. The site fitnesses Ii = Ii(Xi,Xi" ... ,Xi.) are assumed to be
uncorrelated random variables drawn from a common distribution (Kauffman
and Levin 1987). The ruggedness of the landscape increases as the number k
of positions contributing to a single site fitness increases. Different variants
of these landscapes haven been investigated, depending on which positions
contribute to a given site fitness.
-17 -
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
In contrast to the p-spin models, the landscapes of Nk type are affected
by the smoothing operator iii.
Theorem 15. Let f be an arbitrary Nk landscape. Then the autocorrelation
function ofthe landscapes ili r f converges to p(d) = ~(1-2d/n)+(1-0(_1)d
with a < ~ :::; 1 as r tends towards infinity.
Proof. It is necessary and sufficient to show that the autocorrelation func
tion p of f and PI are not orthogonal, i.e., that
(29)
Note that PI(d) = -PI(n - d). Equ.(29) can hence be rewritten as
(30)
If p(d) ~ p(n - d) for all d < n/2, and if the inequality holds strictly for
at least one d < n/2, we have ~ > O. It follows immediately from the very
definition of the Nk model (irrespective of how the neighboring sites are
choosen) that p is monotonically decreasing and non-constant. •Remark. The variants of the Nk model do not behave identically under
the action of iii. Whenever the autocorrelation function is a polynomial of
degree less than n, then we have ~ = 1 in theorem 15. For exact expressions of
the autocorrelation functions of various Nk models see (Weinberger 1991a)
and (Fontana et al. 1993). Recently Nk-models on sequences of arbitrary
alphabets have been considered (Weinberger 1991a). Theorem 15 holds with
~ = 1 for all of them.
As a final example consider an isotropic landscape on the circle of length
n > 4 (Millonas, pers. communication). The configuration space hence is the
Cayley graph of the cyclic group en = {,k 10 :::; k < n} with set of generators
<P = b, ,-I}. Its vertex degree is D = 2. The eigenvectors of the adjacency
-18 -
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
matrix are ek(X) = expe",::X), as an immediate consequence of lemma 5.
The eigenvectors of the collapsed adjacency matrix are Pk(d) = cos( 2",::d) for
o::; k ::; n/2, associated with the eigenvalues Ak = 2 cos( 2~k). The eigenvalue
of 4r2 corresponding to An /2 is much smaller that the one corresponding to
Al for n > 4, hence P converges to PI under the iterated action of q;r whenever
P is not orhogonal to Pl.
Suppose f has a monotone non-increasing autocorrelation function p.
As a particular example consider the random energy model on the circle.
By the same reasoning as in the proof of theorem 15, P is not orthogonal to
PI, and hence iterated smoothing eventually leads to a landscape with auto
correlation function Pl. Note that the smoothed landscapes on the Boolean
Hypercube (and on related graphs) eventually have autocorrelation func
tions of the form p(d) = 1 - ~d + O(d2 ) while for the circle graphs we find
p(d) = 1 - ad2 + O(d3). The landscape on the circle hence becomes ex
ceptionally smooth (Weinberger and Stadler 1993) as a consequence of the
special geometry of the configuration space.
This example also shows that the degeneracy of q;r on the Boolean Hy
percube is not a necessary consequence of the fact that the configuration
space is bipartite, i.e., that there is an eigenvalue -D of the adjacency ma
trix, since the circle graphs are bipartite for even n, and q;r is not degenerate
there.
Discussion
A general theory of the action of linear (averaging) operators on isotropic
landscapes is developed that allows to predict the changes in the correlation
structure of the landscape. A new family of tuneably rugged landscapes is
established, which arises by iterated smoothing of the random energy model.
It is shown that there are particular landscapes, characterized by auto
correlation functions that are eigenvectors of the collapsed adjacency matrix
-19 -
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
of the underlying configuration space, that are not affected by any such
"smoothing" operators. On Boolean Hypercubes the p-spin models, and in
particular the Sherrington-Kirkpatrick model are members of this class. An
other example is the landscape of the graph-bipartitioning problem.
The structure of a landscape is closely linked to the geometry of the
underlying configuration space. Iterated smoothing of the random energy
model leads to a landscape with linearly decreasing autocorrelation function
on generalized hypercubes with alphabets of size I', > 2, while on Boolean
Hypercubes (I', = 2) one obtaines a linear combination of a linear function
and (_l)d. On a circle graph the same procedure leads to an autocorrelation
function of the form 1 - ad2 + ....The theory is not applicable for the TSP problem for the following rea
sons: Firstly, natural configuration spaces for the TSP are Cayley graphs
of the Symmetric group with the set of all transposition, or the set of all
inversions (2opt-moves), as generators. It can be easily shown that these
graphs are not distance transitive, and of course the symmetric group is non
commutative. Secondly, the landscape of the TSP is not isotropic, e.g., on
the graph obtained for transpositions. There the correlation of two tours dif
fering by an exchange of two neighboring cities is larger than the correlation
of two tours differing by any other transposition. For details see Stadler and
Schnabl (1992).
It should be kept in mind that the results presented here depend on the
assumption of isotropic landscapes. While this requirement is often fulfilled
for simple statistical models, it is unlikely to hold for real biophysical or
physical systems. It has been shown, for instance, that the landscapes of
RNA free energies are not isotropic for natural sequences (Stadler and Gruner
1993).
- 20-
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
Acknowledgements
I thank the Max Planck Society, Germany, for supporting my stay at the
Santa Fe Institute. Stimulating discussions with W. Fontana, M. Millonas,
and S.A. Kauffman are gratefully acknowledged.
References
[1] Amitrano C., L. Peliti, and M. Saber. Population Dynamics in Spin
Glass Model of Chemical Evolution. J. Mol. Evol. 29, 513-525 (1989).
[2] Bak P., H. Flyvbjerg, and B. Lautrup. Coevolution in a rugged fitness
landscape. Phys. Rev. A[lS} 46, 6724-6730 (1992).
[3] Biggs N. Algebraic Graph Theory. Cambridge University Press, Cam
bridge, 1974.
[4] Biggs N.L. and A.T. White Permutation Groups and Combinatorial
Structures. Cambridge Univ. Press, Cambridge, 1979.
[5J Binder K. and A.P. Young. Spin Glasses: Experimental Facts, Theo
retical Concepts, and Open Questions. Rev. Mod. Phys. 58, 801-976
(1986).
[6] Bollobas B. Graph Theory - An Introductory Course. Springer-Verlag,
New York, 1979.
[7] Buckley F. and F. Harary. Distance in Graphs. Addison-Wesley,
Reading, MA, 1990.
[7J Cvetkovic D.M., M. Doob, and H. Sachs. Spectra of Graphs - Theory
and Applications, Academic Press, New York, 1980.
[9] Dunkl C.F. Orthogonal Functions on Some Permutation Groups. In:
Ray-Chaudhuri D.K. (ed.) Proceeding of Symposis in Pure Mathe
matic 34. American Mathematical Society, New York 1979.
[10] Eigen M., J.S. McCaskill, and P. Schuster. The Molecular Quasis
pecies. Adv. Phys. Chem. 75, 149-263 (1989).
- 21-
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
[11] Flyvbjerg H. and B. Lautrup. Evolution in a rugged fitness landscape.
Phys. Rev. AI15] 46,6714-6723 (1992).
[12J Fontana W, P.F. Stadler, E.G. Bomberg-Bauer, T. Griesmacher, 1.L.
Hofacker, M. Tacker, P. Tarazona, E.D. Weinberger, and P. Schuster.
RNA Folding and Combinatory Maps Phys. Rev. E 47, 2083-2099
(1993a).
[13J Fontana W., D.A.M. Konings, P.F. Stadler, P. Schuster. Statistics of
RNA Secondary Structures. Biopolymers in press 1993b.
[14] Hotelling H. Analysis of a complex of statistical variables into principal
components. J. Educ. Psych. 24, 417-441 and 498-520 (1933).
[15] Kauffman S.A. and S. Levin. Towards a General Theory of Adaptive
Walks on Rugged Landscapes. J. Theor. Bioi. 128, 11-45 (1987).
[16] Kauffman S.A~, E.D. Weinberger, and S.A. Perelson. Maturation of
the Immune Response Via Adaptive Walks on Affinity Landscapes. In:
Theoretical Immunology, Part I, ed.: Alan Perelson. Addison-Wesley,
Redwood City, 1983.
[17] Lovasz L. Spectra of graphs with transitive groups. Periodica Math.
Hung. 6, 191-195 (1975).
[18] Macken C.A., P.S. Hagan, and A.S. Perelson. Evolutionary Walks on
Rugged Landscapes. SIAM J. Appl. Math. 51, 799-827 (1991).
[19J Rao C.R. Linear Statistical Interference and Its Applications. 2nd ed.,
Wiley, New York (1973).
[20] Rumschitzky D. Spectral Properties of Eigen's evolution matrices. J.
Math. Bio!. 24, 667-680 (1987).
[21] Sherrington D., and S. Kirkpatrick. Solvable Model of a Spin Glass.
Phys. Rev. Lett. 35, 402-416 (1975).
[22] Stadler P.F. and W. Schnabl. The Landscape of the Travelling Sales
man Problem. Phys. Lett. 1,161, 337-344 (1992).
- 22-
P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES
[23J Stadler P.F. and R. Happel. Correlation Structure of the Graph Bi
partitioning Problem. J. Phys. A: Math. Gen. 25, 3103-3110 (1992).
[24] Stadler P.F. Correlation in Landscapes of Combinatorial Optimization
Problems Europhys. Lett. 20479-482 (1992).
[25] Stadler P.F. and W. Gruner. Anisotropy in Fitness Landscapes. J.
Theor. Bioi. in press 1993.
[26J Stadler P.F. Random Walks and Orthogonal Functions Associated
with Highly Symmetric Graphs. SFI preprint 93-06-31 (1993).
[27] Weinberger E.D. Correlated and Uncorrelated Fitness Landscapes and
How to Tell the Difference. Bioi. Cybern. 63, 325-336 (1990).
[28] Weinberger E.D. Local Properties of the Nk-Model, a Tuneably Rugged
Landscape. Phys. Rev. A 44, 6399-6813 (1991a).
[29] Weinberger E.D. Fourier and Taylor Series on Landscapes Bioi. Cy
bern. 65, 321-330 (1991b).
[30] Weinberger E.D. and P.F. Stadler. Why Some Fitness Landscapes are
Fractal. J. Theor. Bioi. in press 1993.
- 23-