Laboratoire de l’Informatique du ParallélismeEcole Normale Supérieure de LyonUnité de recherche associée au CNRS n°1398

Kolmogorov complexity and cellularautomata classi�cationJ.-C. Dubacq, B. Durand, E.Formenti September 1997Research Report No 97-29

Ecole Normale Supérieure de Lyon

Adresse électronique : lip@lip.ens−lyon.fr Téléphone : (+33) (0)4.72.72.80.00 Télécopieur : (+33) (0)4.72.72.80.80

46 Allée d’Italie, 69364 Lyon Cedex 07, France

Kolmogorov complexity and cellular automataclassi�cationJ.-C. Dubacq, B. Durand, E. FormentiSeptember 1997AbstractWe present a new approach to cellular automata (CA for short) classi�ca-tion based on algorithmic complexity. We construct a parameter � which isbased only on the transition table of CA and measures the \randomness" ofevolutions; � is better, in a certain sense, than any other parameter de�nedon rule tables. We investigate the relations between the classical approachbased on topology and ours based on algorithmic randomness. We also com-pare our parameter with Langton's one: we prove that ours is theoreticallybetter and also agrees with some practical evidences reported in literature.Finally we propose a protocol to approximate � and to make experimentson CA dynamical behavior.Keywords: Kolmogorov complexity, topological chaos, cellular automataclassi�cation R�esum�eNous pr�esentons une nouvelle approche �a la classi�cation des automatescellulaires, approche fond�ee sur la complexit�e algorithmique. Nous constru-isons un param�etre � prenant en compte la table de transitions de l'automatecellulaire consid�er�e. Notre th�ese est que ce param�etre mesure le caract�ereal�eatoire des �evolutions de l'automate cellulaire consid�er�e. Nous prouvonsqu'il est meilleur que tout autre param�etre d�e�ni sur les tables de tran-sitions. Nous nous int�eressons aux liens entre notre approche et l'analyseclassique qui utilise des arguments topologiques. Nous comparons aussi notreparam�etre �a celui de Langton et prouvons non seulement que le notre estmeilleur d'un point de vue th�eorique mais en plus qu'il rend mieux comptede r�esultats exp�erimentaux qu'on peut trouver dans la litt�erature. Finale-ment nous pr�esentons un protocole d'exp�erimentations approchant de fa�consatisfaisante notre param�etre �.Mots-cl�es: Complexit�e de Kolmogorov, chaos topologique, classi�cationdu automates cellulaires

Kolmogorov complexity and cellular automataclassi�cationJ.-C. Dubacq, B. Durand, E. FormentiLIP, ENS Lyon,46 all�ee d'Italie, 69364 Lyon Cedex 07, France,e-mail: fjcdubacq,bdurand,eformentg@ens-lyon.frOctober 8, 1997AbstractWe present a new approach to cellular automata (CA for short)classi�cation based on algorithmic complexity. We construct a param-eter � which is based only on the transition table of CA and measuresthe \randomness" of evolutions; � is better, in a certain sense, than anyother parameter de�ned on rule tables. We investigate the relationsbetween the classical approach based on topology and ours based onalgorithmic randomness. We also compare our parameter with Lang-ton's one: we prove that ours is theoretically better and also agreeswith some practical evidences reported in literature. Finally we pro-pose a protocol to approximate � and to make experiments on CAdynamical behavior.1 IntroductionCellular automata (CA for short) are often used for modeling systems con-sisting of many elementary cells interacting locally with each other. Thememory of CA cells is �nite, their interactions are synchronous and occursat discrete time steps.Notwithstanding the apparent simplicity of the formal de�nition of CA,a great variety of dynamical behaviors can be observed as in natural systems.This fact makes the problem of classi�cation a central topic in CA theory.In [14], Wolfram heuristically observes the following behaviors:W1. evolution to a homogeneous state;1

W2. evolution to a set of space-time patterns which are stable or periodic;W3. evolution to an \aperiodic" or \chaotic" space-time pattern;W4. evolution to complex localized structures, sometimes long-lived.Many successive works on CA are an attempt to give a mathematicalconsistence to this classi�cation scheme or to �nd other more satisfying ones.The standard approach to the classi�cation problem is within the frame-work of dynamical systems theory [2, 6, 3]. Classi�cations obtained in thismanner have a common drawback: they are undecidable. Moreover theytake into account neither the \information" content nor the algorithmiccomplexity of the evolutions. We propose an alternative approach which issupposed to recover this gap. In order to measure the information contentof CA evolutions, we de�ne a new parameter � which depends exclusivelyon CA tables. Its de�nition is based on concepts taken from Kolmogorovcomplexity. Theorem 2 proves that this parameter is \better" than anyother parameter based on CA transition tables.An important property of � is that it can be used to study CA on re-stricted sets of initial con�gurations. For instance, if we use CA as models ofco�ee percolation, only a small subset of all con�gurations can be consideredas admissible initial con�gurations: those representing porous media. Theonly request is that the set of admissible con�gurations has to be recursive.The idea of using a parameter based on CA tables for classifying CAbehavior is common. As far as we know it was �rst issued by Langton in [9].Section 3 discusses Langton's approach; we prove that it is too rough todiscriminate chaotic behavior from simple periodic behavior. The discussionis further developed in Section 4.7 where our theory justi�es some critics onLangton's parameter coming from experimental studies.One of the most studied dynamical behavior is chaoticity. Here we provethat topological chaos implies non randomness of CA tables (Theorem 4).This fact underlines one of the di�erences between randomness and topo-logical chaos, even if the lack of randomness tends to 0 when the size of thetables increases.Our study is theoretical. In order to give qualitative and quantitativeevaluations of CA evolutions, we propose in Section 5 an experimental pro-tocol which uses our parameter �. This protocol may be e�ectively exploitedby people which intend to analyze particular CA; its main advantage in thiscontext is that it is theoretically sound.As a �nal remark we underline that our approach is easily extensible toarbitrary dimension at the cost of small changes in the formalization.2

2 Background on CAA one-dimensional CA is a triple hS;N; fi, where S= f0; 1; : : : ; S � 1g isthe set of states, N = f�r; : : : ; 0; : : : ; rg is the neighborhood structure withradius r, and f :S2r+1! Sis the local rule. A con�guration is a \snapshot"of the state of cells, i.e. a mapping fromZtoS. The state of cells is updatedsynchronously by a local interaction rule f . The global rule represents theevolution of the system from time t to t + 1. It is de�ned by8c 2 SZ; F (ct+1)i = f(cti�r; : : : ; cti; : : : ; cti+r):A space-time diagram of an initial con�guration is a 2-dimensional rep-resentation of its iterated images by the considered CA. Observations ofspace-time diagrams may give an idea of the qualitative behavior of CA.In practical simulations we cannot represent in�nite space-time diagrams,so it is a common use to �x the length of the initial con�guration and toimpose some constraints on the cell at boundaries (e.g. periodic boundaryconditions). It turns out that the portion of space-time diagram which is nota�ected by \arti�cial" constraints (and thus it re ects the real computation)is a triangle T, called computation triangle of basis 2rt+ 1 and height t+1.3 Chaos has no edgesThe computer scientists' dream is to have a parameter which makes a one-to-one correspondence between classes of behaviors and intervals in the pa-rameter space. It should also have some e�ectiveness and therefore be basedon the de�ning data of the CA.The �rst approach of this kind (as far as we know) is due to Langton [9].Let us choose a symbol s 2 Sand let n be the number of transitions of f withoutput s. Langton's parameter � for f is the fraction of transitions of f withvalue s with respect of the total number of transitions, i.e. �(f) = S2r+1�nS2r+1(where S denotes the cardinality of S).Analyzing data from CA evolutions, Langton remarked that there exists\critical values" of � in which chaotic behavior has to be found. Thereforehe made the famous hypothesis of the edge of chaos (EOC). In order toexplain this hypothesis, we prefer to cite him directly [8]:\In its basic form this is the hypothesis that in the space of dynamicalsystems of a given type, there will generically exist regions in whichsystems with simple behavior are likely to be found, and other regions3

in which systems with chaotic behavior are to be found. Near theboundaries of these regions more interesting behavior, neither simplenor chaotic, may be expected."The EOC hypothesis is very appealing for scientists but the results ofLangton have been criticized by many researchers (e.g. [5], although we donot completely agree with the methodology and some claims of this paper).Below, we mathematically prove that Langton's parameter is too naive.The notion of deterministic chaos has intrigued many scientists from themost di�erent scienti�c �elds. Notwithstanding there is no standard de�ni-tion. One of the most popular de�nitions that can be found in literature isdue to Devaney [7]. He devises three main components of chaotic behavior,namely sensitivity to initial conditions, regularity and transitivity.De�nition 1 A discrete time dynamical system (DTDS) hX ; F i is a con-tinuous mapping F from a metric space X to itself. It is sensitive to initialconditions if there exists � > 0 such that for all x 2 X for all � > 0 thereexists y 2 B�(x) and n 2 N such that d(Fn(x); Fn(y)) � �. It is transi-tive if for all non-void open sets A;B � X there exists n 2 N such thatFn(A) \ B 6= ;. It is regular if its set of periodic points is dense in thephase space.Intuitively the meaning of sensitivity is that for every point x we can�nd another point y, arbitrary near to x, such that their orbits will divergeof at last � under the iteration of the map. For the physicists this meanthat small errors in experimental measurements are magni�ed in successivetime steps of the system. For all practical purposes sensitive systems defynumerical computation. A DTDS is transitive if it cannot be divided intotwo independent subsystems that do not interact each other.The above properties are not independent and are also linked with sur-jectivity:Proposition 1 Regularity and transitivity imply sensitivity [1]. If a CA istransitive or regular then it is surjective.This proposition justi�es that surjectivity is necessary for chaoticity.Proof. The proof that transitivity imply surjectivity is in [7]. Here weprove that if a CA is not surjective then it is also not regular. Let P bethe set of periodic points of a non surjective CA. Let us suppose that the4

system is regular, that is to say cl(P) = SZ(where cl(�) is the topologicalclosure operator). We have the following chain of relations SZ= cl(P) �cl(F (SZ)) = F (SZ) � SZ. We therefore conclude that F (SZ) = SZwhich iscontrary to our hypothesis. �Proposition 2 (Maruoka and Kimura [11]) A 1-D CA is surjective i�it is k-balanced for all k 2 N. A CA is k-balanced if 8y 2 S2r(k�1)+1;���x 2 S2rk+1 j f(x) = y�� = S2r; where r is the radius.By Proposition 2, we can say that if a CA is chaotic then it has � = 1� 1S .The converse is not true: as a trivial example, the identity CA has � = 1� 1Sbut it is far from being chaotic.Another argument can be used against Langton's approach. For any�xed � 2]0; 1[ we can construct CA with many di�erent properties, suchas the existence of �xed points, periodicity, universality and so on. Theidea is to use CA with many states and de�ne the expected properties onlyon a subset of them via a Cartesian product. From these facts we deducethat Langton's parameter is too rough to discriminate chaos from the pe-riodic behavior. Therefore the idea that \Life is at the edge of Chaos" ismeaningless: Chaos has no edges.4 The algorithmic complexity approachCA tables can be seen as words of length S2r+1 over the alphabet S, whichare the images of all blocks of size 2r+ 1 over the alphabet Sordered lex-icographically. The tables completely handle the description of the CA:assuming that Sis known, there is no ambiguity in identifying a CA withits table.4.1 Some basic results on Kolmogorov complexityThe Kolmogorov complexity theory, also known as the algorithmic informa-tion theory, studies the shortest description of words. Some words can bedescribed by very short programs, while other o�er no regularities and needto be fully spelled.Using Kolmogorov complexity, we implicitly study programs which com-pute the transition tables rather than the tables themselves. This approachis very natural in the context of CA because tables are usually not given5

extensively. For instance, Conway's Game of Life is de�ned by a few simplealgorithmic rules rather than by its 29 transitions.Let us recall that the Kolmogorov complexity of x knowing y, accordingto a machine ', is K'(xjy) = minfl(p); 'hp; yi = xg where l(p) denotesthe length of the word p. In particular, when y does not depend on x, wecan drop the \knowing y" part and just write K'(x). More intuitively theKolmogorov complexity of x knowing y according to a machine ' is the sizeof the smallest program that produces x when applied on y via the machine'. A fundamental result of Kolmogorov complexity theory is that thereexists a speci�c Turing machine that yields optimal results (Kolmogorov-Solomonov theorem). More precisely, there exists a machine '0, called ad-ditively optimal such that 8'0; 9c'0 2 N; 8x; y;K'0(xjy) � K'0(xjy) + c'0 :This result allows us to drop the subscript ' from K'(xjy). It is importantto note thatK(x) is approximable from above, but not computable (see [10]for further explanations).For our purposes, any variety of Kolmogorov complexity can be used.Pre�x Kolmogorov complexity (also called self-delimited) can be used as wellas the original version and gives the same results. This variety is essentialwhen working with in�nite words. This is not our case: we are concernedwith tables which are �nite objects; thus, we can use the pre�x variety oreven the monotonic one without any change in the results (see [13]).4.2 Martin-L�of tests for randomnessMartin-L�of introduced the notion tests for randomness as a tool to measurethe randomness of an individual in�nite sequence. Another equivalent de�-nition can be given in terms of Kolmogorov complexity but in this case, thepre�x or the monotonic variant are necessary (see [12, 4, 15, 10]).A Martin-L�of test for randomness (a test for short) is a function thatattributes marks to strings, qualifying their level of regularity. There aremany possible tests, e.g. looking at 0's in even places, looking for runs ofidentical bits. A test should not give too many high marks; it has to keepat most one string for S initial strings. For example, the test of being oflength 12 is not an acceptable test, because it discards too many strings.More formally, a function � : S?! N is aMartin-L�of test for randomness6

if it is approximable by below and �lls the following condition:8n;m 2 N X�(x)�ml(x)=n S�n � S�m;where x 2 S?.Theorem 1 There exists a Martin-L�of test �0, such that for all test �, thereexists a positive constant c, such that 8x 2 S?; �0(x) � �(x)�c. All the testsverifying these conditions are called universal.This theorem provides a parallel between a universal test and additivelyoptimal Turing machines. Both of them use a notion of optimality up to aconstant. In Section 4.4 we present a theorem of the same kind (Theorem 2).A string x is called c-incompressible i� K(xjl(x)) � l(x)� c. Levin andSchnorr proved that �0(x) = l(x)�K(xjl(x))�1 is a universal test. This re-sult is very important, because it conciliates two approaches on randomnessthat seem a priori very di�erent (see [10, 12]). In fact, any c-incompressiblestring x is c-random (i.e. �0(x) � c) and conversely.4.3 Classi�cation parametersLet us present now the basic properties required for reasonable parameters.In Section 4.4 we prove that among them, some are \better" than others;we shall choose our classi�cation parameter � among the \best" ones.A classi�cation parameter � should be a function taking a CA table asan input, and giving as an output a mark (a real number) that quanti�esthe degree of simplicity of the table. If a table has many regularities, thenit should get a low mark and conversely.Obviously, the larger the table, the more precise is the parameter. Anyparameter should have a minimum amount of e�ectiveness. More formallythe set fhx; p; qi; �(x) � p=qg should be enumerable.We should use only parameters that give a low score to a small num-ber of tables otherwise the parameters would be too rough. A reasonableassumption is to set this rate to at most Sl(x)�m tables x getting a markof at most m=l(x). More formally, an algorithmic classi�cation parameter� is a function from SS2r+1 into [0; 1] such that fhx; p; qi; �(x) � p=qg isenumerable and 8m 2 N jfx; �(x)� m=l(x)gj � Sl(x)�m.Let us take an example with S= f0; 1g. With little transformation, wecan turn Langton's parameter � into an algorithmic classi�cation parameter7

�� that ful�lls our requirements. Let �� = 1� j2�� 1j, it is maximal whenthe number of 0's and 1's are equal, and minimal for a table which is only0's or only 1's.4.4 Our parameter �Algorithmic classi�cation parameters are somehow linked with Martin-L�oftests. They can be transformed into Martin-L�of tests: �(x) = l(x)(1��(x)).As a consequence, Martin-L�of theorem (Theorem 1) about the existence ofa universal test can be reformulated in our context.Theorem 2 There exists a universal algorithmic classi�cation parameter�, such that for all �:9c 2 N; 8x 2 SS2r+1; �(x) � �(x) + cl(x):This theorem formalizes the claim that \� is better than any other pa-rameter". Practically, those tables which get high marks with �, also gethigh marks with any other parameter �. The thesis of this paper is that � isa good parameter for analyzing CA and we explain why in the sequel. Theidea is that tables with many regularities can be described by rather shortprograms. Thus we can take one of these programs as a representation ofthe table and consider its length as a measure of the complexity of the tableitself.Using the construction of Levin and Schnorr we can give a precise ex-pression of � in terms of Kolmogorov complexity.Theorem 3 A universal classi�cation parameter (that we denote by �) canbe expressed by �(x) = K(xjl(x))+1l(x) .In other terms, we can say that � captures all recursive regularities ofthe tables. For c-random tables, � is near 1; conversely, simple tables havelow �.4.5 Comparison with Wolfram's approachThe aim of this section is to analyze the relations between the complexity ofthe table of a CA and the degree of randomness of its computation triangles.8

Proposition 3 For any CA of table x the complexities of its computationtriangles are bounded by l(x)�(x) + � , where � is the width of the triangle.Proof. A triangle of computation can be obtained by the table of the CAand the initial con�guration. We have to process information coming fromtwo di�erent sources. The complexity of a pair isK(x; y) = K(x) +K(yjx) +O(log(min(K(x);K(y)))):In the following we drop the logarithmic term because it can always beenneglected in our context. Note also that this term depends on the variety ofKolmogorov complexity that is used.The complexity of T is bounded by the sum of the complexity of thetable and the one of the initial con�guration. If we pick some random tableof length l and some random initial con�guration of length l0, we get:K(Tjl(T)) � l�(table) + K(con�gjl0)� S2r+1 + 2tr + 1 �By Levin-Schnorr theorem, for any c-random triangle T, K(Tjl(T)) =(1 + t)(1 + tr) � c. Therefore for any �xed CA, the progression of thecomplexity ofTcompared with a random triangle of same size is t comparedwith t2. Hence, the claim \This computation is chaotic because it seemsrandom" is meaningless. No CA computation is random. However, we candiscuss the di�erent factors that interact to give the �nal complexity of atriangle of computation. These factors depend on the size of Tand the sizeof the table of the CA. Thus, we should make a trade-o� between thesequantities.A good idea is to minimize the e�ect of the initial con�guration on theglobal complexity of the computation triangle, and at the same time givethe CA enough room to distinguish itself from other CA. So we shoulduse con�gurations of height k = O �pS2r+1=r�, and of width rt. Asa consequence, the order of growth of such observations is in the rangeO��qS2r+1r ; S2r+1r ��. We remark that almost all practical observationsthat we can see in literature are in this range.Our approach is also robust when we restrict to a recursive set of initialcon�gurations. The formalization requires in this case some easy changes9

on the Kolmogorov complexity of the initial con�guration. We could alsoincrease the dimension of the working space. These modi�cations only af-fect the range of observations but there are no substantial changes in theformalization.4.6 An example of application: balanced CAIn Section 3, we proved that balance is a necessary property for chaotic CA.Let us recall that surjectivity is equivalent to be k-balanced for all k. Thisimplies in particular that surjective CA are 1-balanced, i.e. that the numberof occurrences of all outputs a 2 Sare the same: S2r+1S = S2r.Theorem 4 Topological chaos implies non-randomness of tables.Proof. A chaotic CA is necessarily 1-balanced (see Proposition 1). In thiscase, a description of the table shorter than S2r+1 log S can be given. Weneed to give a description of the table which is valid for all CA. The setof CA can be described �rst by giving for each state a 2 Sthe di�erencebetween the number of inputs giving this character jxja and S2r+1S = S2r.We use self-delimited notations for these numbers. Then we add in thedescription the index of the CA in the set containing all CA that are un-balanced the same way. In the case of a 1-balanced CA, the �rst part ofthe description uses only K(S) +K(r) bits (as the excess is always 0). Letus now compute the cardinality of the set of 1-balanced CA. We have tochoose exactly S2r places in a range of S2r+1 for the �rst element of S.Then we have to place S2r occurrences of the second element of S in theremaining places, and so on. This yields a number of 1-balanced CA ta-bles that is exactly S�2Qi=0 � (S � i)S2rS2r �. An approximation can be foundthrough Stirling's Formula. If we denote S2r by A, we obtain:S�2Yi=0 � (S � i)AA � = S�2Yi=0 ((S � i)A)!((S � i� 1)A)!A! � SSAs S(2�A)S�1 :After a tedious computation, we obtain that, the randomness de�ciencyis greater than O � rS2r �. �Therefore topologically chaotic CA have a � which is reduced by a certainamount, and hence is not maximal. This means that, compared with themaximal complexity of CA, those that are surjective are not random. This10

amount is relatively signi�cant when r and s are small but tends to 0 whenr and S grow.4.7 Relations with Langton's parameterIn this section, we investigate which CA have high Kolmogorov complexity.The preceding section shows that cellular automata with � = 1� 1S are notrandom, because their randomness de�ciency is lower bounded by O � rS2r �.However, in a random string the number of occurrences of each charactershould be about equal (but not exactly equal) [12, 10].A precise evaluation of Langton's �(x) for a c-random table x gives1� ��(x) = (1� 1S )� �(x) � O 1pl(x)! :In Langton's original point of view, � was used to measure the intrinsiclevel of chaos in CA. Crutch�eld et al. pointed out in [5] that experimentally,� = 1 � 1S seems to be a local minimum (instead of global maximum) forchaoticity in CA, but they agree that most chaotic CA have � not far from1� 1S . This result is compatible with our parameter, since it corresponds tothe fact that 1-balanced CA have a � that is not maximum. At the sametime, CA with maximum � are not far from being 1-balanced.If the experimental study reported in the previously cited paper [5] issound, then the behavior of � is a possible mathematical explanation ofthese outcomes.5 Conclusion and protocol propositionWe have approached the CA classi�cation problem by the tools of algo-rithmic randomness and Kolmogorov complexity theory. We have proposeda parameter � which measures the intrinsic complexity of the CA tablesand we have established a link between � and the degree of randomnessof computations. In Theorem 2 we proved that � is better than any otherparameter that could be de�ned on the tables.Unfortunately, � is not computable but only approximable by above.We suggest below to overcome this problem using an approximation of �.This way, we think that the qualitative informations obtained by � can beexperimentally observed. 11

An interesting open problem is to further understand the relationshipsbetween topological chaos and the degrees of randomness in CA computa-tions.Experimental protocol We suggest to replace the evaluation of � (de-�ned via Kolmogorov complexity) by the compression ratio of its table; wepropose to use a practically e�cient compression algorithm.We plan to apply this method in some practical cases, and check if theabove approximation of algorithmic chaos agrees with intuitive observations.AcknowledgementsWe are very grateful to Pr. Vladimir Andreevich Uspensky for many helpfuldiscussions on varieties of Kolmogorov complexity and randomness.References[1] J. Banks, J. Brooks, G. Davis G. Cairns, and P. Stacey. On Devaney'sde�nition of chaos. Am. Math. Monthly, 99:332{334, 1992.[2] F. Blanchard, A. Maas, and P. Kurka. Topological and measure-theoretic properties of one-dimensional cellular automata. Physica D,103:86{99, 1997.[3] G. Braga, G. Cattaneo, P. Flocchini, and C. Quaranta Vogliotti. Pat-tern growth in elementary cellular automata. Theor. Comp. Sci., 45:1{26, 1995.[4] C. Calude. Information and Randomness. Springer-Verlag, 1994.[5] J. P. Crutch�eld, P. T. Haber, and M. Mitchell. Revisiting the edge ofchaos: evolving cellular automata to perform computations. ComplexSystems, 7:89{130, 1993.[6] K. �Culik, J. Pachl, and S. Yu. On the limit set of cellular automata.SIAM Journal on Computing, 18:167{175, 1989.[7] R. L. Devaney. Introduction to chaotic dynamical systems. Addison-Wesley, second edition, 1989. 12

[8] H. Gutowitz and C. Langton. Mean �eld theory and the edge of chaos.In Proc. of 3rd Europ. Conf. on Art. Life, 1995.[9] C. Langton. Computation at the edge of chaos: phase transitions andemergent computation. Physica D, 42:12{37, 1990.[10] M. Li and P. Vit�anyi. An Introduction to Kolmogorov complexity andits applications. Springer-Verlag, second edition, 1997.[11] A. Maruoka and M. Kimura. Conditions for injectivity of global mapsfor tessellation automata. Information & control, 32:158{162, 1976.[12] V. A. Uspensky, A. L. Semenov, and A. Kh. Shen. Can individualsequences of zeros and ones be random? Russ. Math. Surveys, 45:121{189, 1990.[13] V. A. Uspensky and Kh. Shen. Relations between varieties of Kol-mogorov complexities. Math. Syst. Theory, 29(3):270{291, 1996.[14] S. Wolfram. Computation theory of cellular automata. Comm. in Math.Phys., 96:15{57, 1984.[15] A. K. Zvonkin and L. A. Levin. The complexity of �nite objects and thedevelopment of the concepts of information and randomness by meansof the theory of algorithms. Russ. Math. Surveys, 25:83{124, 1970.13