Research ArticleAn Information-Based Classification of ElementaryCellular Automata

Enrico Borriello12 and Sara Imari Walker1234

1Beyond Center for Fundamental Concepts in Science Arizona State University Tempe AZ USA2ASU-SFI Center for Biosocial Complex Systems Arizona State University Tempe AZ USA3School of Earth and Space Exploration Arizona State University Tempe AZ USA4Blue Marble Space Institute of Science Seattle WA USA

Correspondence should be addressed to Enrico Borriello enricoborrielloasuedu

Received 7 March 2017 Revised 11 June 2017 Accepted 27 July 2017 Published 7 September 2017

Academic Editor Sergio Gomez

Copyright copy 2017 Enrico Borriello and Sara Imari Walker This is an open access article distributed under the Creative CommonsAttribution License which permits unrestricted use distribution and reproduction in any medium provided the original work isproperly cited

We propose a novel information-based classification of elementary cellular automata The classification scheme proposedcircumvents the problems associated with isolating whether complexity is in fact intrinsic to a dynamical rule or if it arises merelyas a product of a complex initial state Transfer entropy variations processed by cellular automata split the 256 elementary rulesinto three information classes based on sensitivity to initial conditions These classes form a hierarchy such that coarse-grainingtransitions observed among elementary rules predominately occur within each information-based class or muchmore rarely downthe hierarchy

1 Introduction

Complexity is easily identified but difficult to quantifyAttempts to classify dynamical systems in terms of their com-plexity have therefore so far relied primarily on qualitativecriteria A good example is the classification of cellularautomaton (CA) rules CA are discrete dynamical systems ofgreat interest in complexity science because they capture twokey features of many physical systems they evolve accordingto a local uniform rule and can exhibit rich emergent behavioreven from very simple rules [1] Studying the dynamics ofCA therefore can provide insights into the harder problem ofhow it is that the natural world appears so complex given thatthe known laws of physics are local and (relatively) simpleHowever in the space of CA rules interesting emergentbehavior is the exception rather than the ruleThis has gener-ated wide-spread interest in understanding how to segregatethose local rules that generate rich emergent behavior forexample gliders and particles computational universalityand so on from those that do not (for the seminal paper onConwayrsquosGame of Life cf [2]The first outlines of proof of theuniversality of a two-dimensional Game of Life can be found

in [3 4] Proofs of the universality of nonelementary one-dimensional cellular automata are instead in [5 6] For thefamous proof of the universality of elementary rule 110 cf [7])A complication arises in that the complexity of the output ofa CA rule is often highly dependent on that of the input stateThismakes it difficult to disentangle emergent behavior that isa product of the initial state from that which is intrinsic to theruleThis has resulted in ambiguity in classifying the intrinsiccomplexity of CA rules as one must inevitably execute a CArule with a particular initial state in order to express its com-plexity (or lack thereof)

One of the first attempts to classify CA rules was pro-vided by Wolfram in his classification of elementary cellularautomata (ECA) [8] ECAare someof the simplest CAand are1-dimensional with nearest-neighbor update rules operatingon the two-bit alphabet ◼ ◻ Despite their simplicity ECAare known to be capable of complex emergent behaviorInitializing an ECA in a random state leads some rules toconverge on fixed point or oscillatory attractors while otherslead to chaotic patterns that are computationally irreducible(such that their dynamics are impossible to predict from theinitial state and chosen rule without actually executing the

HindawiComplexityVolume 2017 Article ID 1280351 8 pageshttpsdoiorg10115520171280351

2 Complexity

Table 1 Wolframrsquos classification of ECA with only the 88 rule equivalence classes shown (only the lowest number rule of a given equivalenceclass is listed) [28]

Wolframrsquos Class I0 8 32 40 128 136 160 168

Wolframrsquos Class II1 2 3 4 5 6 7 9 10 1112 13 14 15 19 23 24 25 26 2728 29 33 34 35 36 37 38 41 4243 44 46 50 51 56 57 58 62 7273 74 76 77 78 94 104 108 130 132134 138 140 142 152 154 156 162 164 170172 178 184 200 204 232

Wolframrsquos Class III18 22 30 45 60 90 105 122 126 146150

Wolframrsquos Class IV54 106 110

full simulation) Based on these diverse behaviors Wolframidentified four distinct complexity classes shown in Table 1Class I CA are those whose evolution eventually leads to cellsof only one kind Class II CA lead to scattered stable or oscil-lating behaviors Class III CA show an irreducibly chaoticpattern Class IVCAcan exhibit any of the previous behaviorssimultaneously and seem to possess the kind of complexitythat lies at the interface between mathematical modeling andlife studies [9] Wolframrsquos classification stands as a milestonein the understanding of CA properties and still represents themost widely adopted classification scheme Nonetheless itsqualitative nature represents its main limitation In fact asCulik II andYu showed formally redefiningWolframrsquos classes[10] reveals that determiningwhich class a given ECAbelongsto is actually undecidable

Despite this no-go result efforts to better classify CAhavenot diminished Li and Packard [11] introduced a classifica-tion scheme refining Wolframrsquos originally proposed schemewith the explicit goal to better distinguish between locallyand globally chaotic behaviors Langton introduced the firstattempt at a quantitative classification [12] His classificationimplemented a projection of the CA rule space over the 1Dclosed interval [0 1] CA rules with similar qualitative fea-tures are then roughly arranged in terms of comparable valuesof the Langton parameter with the most complex behav-iors found at the boundary separating periodic rules fromchaotic ones Alternative approaches to quantitative classifi-cation include generalizations of concepts from continuousdynamical systems applied to CA such as the maximumLyapunov Exponent (MLE) [13] For CA MLE is defined interms of the Boolean Jacobian matrix [14] and captures themain properties of its continuous counterpart encoding thetendency of a CA to reabsorb damage in its initial configura-tion or to let it spread as a consequence of chaotic dynamicsThis approach proved useful in more recent analysis of thestability and the shift in complexity of CA in response tochanges in topology [15ndash18] A number of other classificationschemes have been proposed over the years based on indexcomplexity [19] density parameter with d-spectrum [20]

communication complexity [21] algorithmic complexity[22] integrated information [23] and so on A comprehen-sive review is outside of the scope of this paperThe interestedreader is directed to the recent review by MartInez [24] andthe literature cited therein

In this paper we report on experiments demonstratingnew quantitative classification of the intrinsic complexity ofECAs which differs from earlier attempts by exploiting themain weakness plaguing many approaches to quantitativeclassification That is we explicitly utilize the sensitivity ofthe expressed complexity of ECA rules to the initial inputstate In recent years there has been increasing interest inusing information-theoretic tools to quantify the complexityof dynamical systems particularly in the context of under-standing biological systems [25 26] A promising tool in thiscontext is transfer entropy (TE) Schreiberrsquos measure of thedirected exchange of information between two interactingparts of a dynamical system [27] In what follows we adoptTE as a candidate quantitative selection criterion to classifythe intrinsic complexity of ECAs by exploring the sensitivityof TE to changes in the initial state of a CA

The paper is structured as follows In Section 2 we startfrom the simplest nontrivial initial configuration of an ECAand use it to identify the dynamical rules able to producea complex output (as quantified by TE) by virtue of theirintrinsic complexity In Section 3 we repeat our analysis for-more general inputs and identify those outputs whose com-plexity is instead inherited by the complexity of the input asopposed to the rule We then classify ECA rules according tothemaximumdegree of variability of the output they producefor varying inputs As we will show three quantitativelyand qualitatively distinct classes naturally emerge from thisanalysis In Section 4 we show that this classification inducesa partially ordered hierarchy among the rules such thatcoarse-graining an ECA of a given class yields an ECA of thesame class or simpler [1] We conclude by proposing furtherapplications of the classification method presented

Complexity 3

2 Intrinsic Complexity

In what follows we identify the complexity of a CA with theamount of information it processes during its time evolutionThe concept of information processing presented herein isadopted from information dynamics [29ndash32] a formalismthat quantifies computation in dynamical systems usingmethods from information theory In information dynam-ics Schreiberrsquos transfer entropy [27] is identified with theinformation processed between two elements of a dynamicalsystem In Schreiberrsquos measure the transfer of informationbetween a source 119884 and a target119883 is quantified as the reduc-tion in uncertainty of the future state of 119883 due to knowledgeof the state of 119884 In a CA the source 119884 and target 119883 are bothcells of the CA

Formally TE is defined as the mutual information be-tween the source119884 at a certain time 119905 and the target119883 at time119905 + 1 conditioned on a given number 119896 of previous states of119883

119879119884rarr119883 = sum(x(119896)119905 119909119905+1 119910119905)isinS

119901 (x(119896)119905 119909119905+1 119910119905)

times log2119901 (119909119905+1 | x(119896)119905 119910119905)119901 (119909119905+1 | x(119896)119905 )

(1)

where (see Figure 1)

(i) 119909119905+1 represents the state of cell119883 at time 119905 + 1(ii) 119910119905 is the state of cell 119884 at the previous time 119905(iii) x(119896)119905 is the vector (119909119905minus119896+1 119909119905) of the previous 119896

states of cell119883(iv) S represents the set of all possible patterns of sets of

states (x(119896)119905 119909119905+1 119910119905)To calculate TE one must start with the time series of

states of a CA Given the time series (see Figure 1) the occur-rence numbers of patterns of states (x(119896)119905 119909119905+1 119910119905) are countedfor each combination of cells 119883 and 119884 in the array Oncenormalized to the total number of (not necessarily distinct)patterns appearing in the time series these frequencies areidentified with the probabilities 119901(x(119896)119905 119909119905+1 119910119905) appearingin (1) The conditional probabilities 119901(119909119905+1 | x(119896)119905 119910119905) and119901(119909119905+1 | x(119896)119905 ) are calculated analogously

Changing the value of the history length can affect thevalue of TE calculated for the same time series data We findvery similar values of TE for 119896 = 4 and 6 and observe a con-siderable decrease in TE for 119896 lt 4 and 119896 gt 6 (cf Section 42 of[32]) relative to the value of TE for 119896 = 5 We thereforeconsider 119896 = 5 as the optimal value of 119896 that properly cap-tures the past history of 119883 for the results presented here (seeSection 3 for a visual explanation of why 119896 sim 5 represents theoptimal history length)

We first generated time series for the simplest initial statefor each of the 256 possible ECA rules which we numericallylabel following Wolframrsquos numbering scheme [33 34] Hereand in the following periodic boundary conditions are

ECA array

t

Tim

e

t + 1

t minus k + 1

CellCell XY

x(k)t

xt+1

yt

Figure 1 Time series of an ECA with the pattern of cells relevantto the evaluation of the transfer entropy from 119884 to 119883 119879119884rarr119883 high-lighted

enforcedTherefore the specific location of the different colorcell in the input array is irrelevant Our initial state is either

◻ sdot sdot sdot ◻⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟50-times

◼◻ sdot sdot sdot ◻⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟50-times

(2)

or the equivalent state obtained through a ◻ harr ◼ conju-gation For example rules 18 and 183 are equivalent undera ◻ harr ◼ conjugation The input ◻ sdot sdot sdot ◻◼◻ sdot sdot sdot ◻ is updatedusing rule 18 and the conjugated input ◼ sdot sdot sdot ◼◻◼ sdot sdot sdot ◼ usingrule 183 A comprehensive study of the symmetry propertiesof ECA can be found in [11] where the 256 possible ECArules are grouped into 88 different equivalence classes Eachclass contains no more than four rules which show the samebehavior under ◻ harr ◼ exchange left-right transformationsor the combination of both transformations The interestedreader can find such classification summarized in Table 1 ofthe cited work For Wolfram Classes III and IV (Wolframrsquosmost recent ECA classification scheme is now implementedin the Wolfram Alpha computational engine [28] and repro-duced here (Table 1) for the convenience of the reader) theequivalence classes are also shown in the legend of Figure 2

We calculate the amount of information processed by a CAin a space-time patch as the average of 119879119884rarr119883 calculated overthat region designated as ⟨TE⟩ To do so we evolve theCA for250 time steps and then remove the first 50 time steps fromeach generated time seriesThis allows us to evaluate TE overa region of the CA in causal contact with the initial input ateach point in space and time This ensures that the observeddynamics over the relevant space-time patch are driven byboth the chosen rule and the initial state The resulting timeseries is then used to evaluate the 1012 values TE119884rarr119883 Foreach rule the average value ⟨TE⟩ is shown in Figure 2(a)Equivalent rules like rules 30 86 135 and 149 producethe same values of ⟨TE⟩ and are represented by a commonmarker For clarity individual Wolfram Classes I and II rulesare not shown and are instead replaced by a dashed linecorresponding to the highest ⟨TE⟩ of any individual Class Ior II rule All Class III rules lie above the range of Classes Iand II Interestingly with only the exception of rule 110 andits equivalent rules all Class IV rules lie within the range ofClasses I and II rules

4 Complexity

0 50 100 150 200 250

Rule number

000

005

010

015

020Av

erag

ed tr

ansfe

r ent

ropy

(bits

)

18 18322 15130 86 135 14945 75 89 10160 102 153 19590 165105

122 161126 129146 18215054 147106 120 169 225110 124 137 193

Clas

ses I

amp II

rang

e

(a) Single-cell input

0 50 100 150 200 250

Rule number

Clas

ses I

amp II

rang

e

000

005

010

015

020

Aver

aged

tran

sfer e

ntro

py (b

its)

18 18322 15130 86 135 14945 75 89 10160 102 153 19590 165105

122 161126 129146 18215054 147106 120 169 225110 124 137 193

(b) Random input

Figure 2 ⟨TE⟩ for each rule shown for two different inputs single cell (a) (either one black cell or one white cell) and random (b) Equivalentrules are represented by a common marker The association of equivalent inputs with equivalent rules is chosen heuristically as the one thatmaximizes the averaged TE The dashed line corresponds to the highest value of ⟨TE⟩ for any Wolfram Class I or II rule Individual markersare shown only for Wolfram Classes III (black) and IV (orange) rules

The single-cell input considered here is extremely rarewithin the space of all possible initial inputs The numberof black cells in a state randomly extracted among the 2101different possible inputs follows a binomial distributionmeaning that states containing 50 or 51 black and white cellsare about 2times1027 times more likely than our initially selectedinput Our motivation for considering the single-cell inputfirst is that it automatically excludes many trivial rules fromour search for the complex ones Rules that duplicate theinput in a trivial way or annihilate it to produce all ◼ or all◻ naturally yield ⟨TE⟩ ≃ 0 This is the same approach thathas been recently assumed in algorithmic complexity basedclassification of CAs [22] It has the advantage of selectingmany rules according to their intrinsic complexity and notthe complexity of the input However a major shortcomingof choosing the single-cell input is that many Class IV ruleslook simpler than what they truly are Class IV rule 106 isa good example For the simple input of a single black bitrule 106 functions to shift the black cell one bit to the leftat each time-step generating a trivial trajectory However incases where the input allows two black cells to interact thefull potential of rule 106 can be expressed This is an explicitexample of the sensitivity of the behavior of some ECA rules

to the complexity of their input as discussed in the introduc-tion To isolate the complexity of the rule from that inheritedby the input we must therefore consider a random sample ofinitial states as we do next

3 Inherited Complexity andInformation-Based Classification

We next consider a more generic input randomly selectedamong the 2101 ≃ 25times1030 different possibilities As the inputis no longer symmetric we now need to consider reflec-tions in selecting the equivalent input rarr rule associationsThe scenario changes completely in this case as shown inFigure 2(b) where the highest values of ⟨TE⟩ correspondto Class II rules including 15 85 170 and 240 whichgenerate trivial wave-like behaviors These rules behavelike rule 106 (especially 170 and 240) when initialized witha single-cell input but they do not contribute any newemergent nontrivial features when nearby cells interact Forall purposes they appear less intrinsically complex With onlythe exception of rule 110 and its equivalent rules Class IVrules behave like many rules of Class II and exhibit a largeincrease in complexity as qualitatively observed and also

Complexity 5

as quantitatively captured by ⟨TE⟩ in response to a morecomplex input

It is worth noting that all of the rules whose initial valueof ⟨TE⟩ lay above the upper limit for Classes I and II of⟨TE⟩ = 0024 bits in the simplest input scenario still havecalculated ⟨TE⟩ above this value under a change of input Inparticular the rules with the highest values of ⟨TE⟩ are notsignificantly affected by the change in the input Let us naivelyuse the upper limit for Classes I and II rules emerging fromFigure 2(a) as the border line between what we call low andhigh values of ⟨TE⟩ We can summarize the changes undervarying the input as follows There are rules whose value of⟨TE⟩ is low for both the inputs There are rules whose valueof ⟨TE⟩ is low in the simplest case but high in the morecomplex one And finally there are also rules with a highvalue of ⟨TE⟩ in both cases The interesting point is that wefind no rule whose value of ⟨119879119864⟩ is high for the simplest inputand low for the random input This is equivalent to say thatthere are no rules generating complexity for simple inputsbut annihilating it for complex ones The significance of thisobservation lies in the fact that it enables classification interms of the shift of ⟨TE⟩ over a space-time region in responseto a change in its input by taking advantage of the mainlimitation that makes quantitative classifications of ECAs sodifficult that is ECA sensitivity to their initial state andexploits it in order to achieve such classification

To confirm this is indeed a viable method for quantitativeclassificationwemust considermore than just two inputsWetherefore first randomly selected twenty different ones Beinginterested in how much ⟨TE⟩ can vary as we vary the inputfor each rule we selected the maximum absolute value of thepercent change of ⟨TE⟩ between the random inputs (⟨TE⟩119903)and the single-cell input (⟨TE⟩1) considered before

max1003816100381610038161003816⟨TE⟩119903 minus ⟨TE⟩11003816100381610038161003816⟨TE⟩119903 randominputs

(3)

The results are shown in Figure 3 where the maximum per-cent change is plotted as a function of ⟨TE⟩1 Equivalent rulesare represented by a single marker and share the same valuefor the maximum relative change of ⟨TE⟩ In fact given tworules 1199031 and 1199032 sharing a symmetry it is always possible to findtwo inputs 1198941 and 1198942 such that rule 1199031 initialized to 1198941 and rule1199032 initialized to 1198942 give rise to the same value of ⟨TE⟩

The horizontal dashed line separates rules whose maxi-mum ⟨TE⟩ change is less than one order of magnitude fromthose that can undergo a change in ⟨TE⟩ of at least a factor 10The vertical dashed line denotes the highest value of ⟨TE⟩1 forWolframClasses I and II exactly as in Figure 2(a)The regionto the right of the vertical line and above the horizontal line isvoid of anyECA rules Points in that regionwould correspondto values of ⟨TE⟩ that are both high for the simplest input andcapable of high variation for exampleCArules that can anni-hilate the complexity of the input which we do not observeThis feature yields a distinctive L-shape in the distribu-tion of ruleswith the rules in each region sharingwell-definedproperties

As a result a natural information-based classification ofECAs can be given as follows

Class I1 ⟨TE⟩ is very small for the simplest input andstays so for complex inputsThis is themost populatedclass including almost all Wolfram Classes I and IIrules rules 18 and 146 and their equivalent Class IIIrules

Class I2 ⟨TE⟩ is small for the simplest input but itexperiences a drastic change (one order of magnitudeor more) when the input is complex This is the casefor many Wolfram Class II and some Class IV rules(eg 54 and 106 and their equivalent rules)

Class I3 ⟨TE⟩ has a high value for the simplest inputand this value is approximately unaffected by a changein the input Most Wolfram Class III rules belong tothis class as well as Class IV rule 110 and its equivalentrules

Our classification is summarized in Table 2 Despite thearbitrary placement of our boundary lines at the border ofthe I1 region it shows an unambiguous separation betweenthe most majority of rules classified as I2 and I3

Randomly sampling inputs leads to a bias favoring nearlyfifty-fifty distributions of black and white cells due to theirbinomial distribution We therefore also verified this clas-sification using a different distribution of inputs where thenumber of black cells is increased in a regular way from 2to 50 while keeping the specific positions of black cells inthe input array random We considered 20 different inputseach containing exactly 2 5 7 10 12 47 and 50 blackcells (higher numbers are not considered due to the ◻ harr◼ conjugation) Apart from minor shifts of the data pointsapplying the same procedure as above yields exactly the sameclassification as Figure 3 indicating that our classificationscheme is robust

We stress the importance of considering a large system(order sim 100 cells) as opposed to a much smaller one (egordersim 10)While for the latter a scan over the entire space ofinputs is computationally feasible (and indeed we performedthese experiments) it hides one of the main features enablinginformation-based classification the existence of Class I3rules which form themost stable class with respect to our TEbased complexity measure Class I3 rules produce time serieslargely independent of the initial state exemplified by rule30 in Figure 3 a feature evident only in larger systems (beingthe result of combinatorial calculus TE computational timegrows exponentially with the number of cells considered AnECA array of about 100 cells is both the smallest size that wecould safely consider much larger than that of the domainsof Figure 3 and for which a complete scan over the rule spaceand a large enough ensemble of inputs could be made Wealso considered individual runs with 200 and 400 cells andnoticed that our numerical results are stable for CA with 100or more cells but are fluctuating for CA with 50 or less cells)

The typical order 10 cell length of the domains appearingin Figure 3 is also the reason why 119896 sim 5 is the most optimalvalue for the history length parameter Smaller and largervalues of 119896 would not be able to resolve this lattice sizeindependent feature of Class I3 rules [32] (Figure 4)

6 Complexity

Table 2 Information based classification of ECA with only the 88 rule equivalence classes shown (represented by the lowest number rule)Wolfram Class III rules are denoted in black boldface type and Class IV in italic

Class I10 1 3 4 5 8 9 12 13 1819 23 25 26 28 29 32 33 35 3637 40 41 44 50 51 57 58 62 7273 76 77 78 94 104 108 128 130 132134 136 140 146 152 154 156 160 162 164178 200 204 232

Class I22 6 7 10 11 14 15 24 27 3438 42 43 46 54 56 74 106 138 142168 170 172 184

Class I322 30 45 60 90 105 110 122 126 150

0

20

40

60

80

100

Max

imum

per

cent

chan

ge o

f ⟨TE

⟩

005 010 015 020000

⟨TE⟩ (single-cell input bits)

Class I1 Class I3

Class I2Random inputs

Figure 3 Maximum percent change of ⟨TE⟩ as a function of ⟨TE⟩1(3) Equivalent rules are represented by a singlemarker colored as inFigure 2 but with Wolfram Classes I and II in gray Rules above thehorizontal dashed line can undergo a change of at least one orderof magnitude in ⟨TE⟩ The vertical dashed line is the same as inFigure 2(a)

4 Coarse-Graining andthe Information Hierarchy

Perhaps the most interesting feature of our quantitativeclassification is that Wolfram Class III and Class IV rulesare distributed over different information-based classes Thisbehavior looks less surprising in the light of the coarse-graining transitions among ECA uncovered by Israeli andGoldenfeld [1] One important aspect of the physical worldis that coarse-grained descriptions often suffice to make

Rule 30single cell input

Rule 30random input

Figure 4 Time series for rule 30 generated by the two initial statesof Figure 2 The similarity of the two outputs each yielding colordomains with a typical linear size of about sim10 blocks does notdepend on the size of the CA and is responsible for the stability ofClass I3 rules to changes in input

predictions about relevant behavior Noting this Israeli andGoldenfeld adopted a systematic procedure to try to coarsegrain ECA rules Their prescription consists in groupingtogether nearby cells in theCAarray into a supercell of a givensize 119873 according to some specified supercell rarr single-cellcorrespondence (for Boolean CAs 22119873 possible applicationsof this kind exist) and a given time constant 119879 The searchfor a coarse-graining rule consists in looking for a new CArule such that running the original CA for 119879119905 time steps andthen grouping the supercells produces the same output asgrouping the initial array and then adopting the new CA rulefor 119905 time steps The new CA rule the time constant 119879 andthe size of the supercell 119873 are all unknown therefore thissearch requires a scan over all the possible combinations ofthese parameters Israeli and Goldenfeld successfully coarse-grained 240 of the 256 ECA rules many to other ECA rulesperforming a complete scan of all the possibilities compatiblewith 119873 le 4 Importantly the rule complexity was neverobserved to increase under coarse-graining introducing apartial ordering among CA rules

Complexity 7

Class I1

Class I2

Class I3 60 90 105 150 126

2 42170

6 7 11 14 24 27 5674 168 184

15

34

5

51

140

13

152136

156

0

25 26 37 41 5762 73 94 104154 164 204

128 44 130 132 162

1 4 12 1929 72 76

23 77 134 146160 178 232

200

204

3 8 9 18 26 28 3233 35 36 40 50 5878 108

172 10 38 46 138

22 30 45 110 122

43 54 106 142

Figure 5 Coarse-graining transitions within ECA found in [1] mapped onto a hierarchy of the information-based classes identified hereOnly the 88 rule equivalency classes are shown (represented by the lowest number rule) For simplicity (nontrivial) transitions to rule zeroare not shown (isolated boxed with continuous border) Boxes with a dashed border include ECA rules for which a coarse-graining transitionto another ECA rule with supercell of size 4 or less does not exist [1]

The same ordering emerges from our information-basedclassification as shown in Figure 5 where arrows indicatecoarse-graining transitions uncovered in [1] These transi-tions introduce a fully ordered hierarchy I3 rarr I2 rarr I1such that coarse-graining is never observed to move up thehierarchy and the vast majority of rules may only undergocoarse-graining transitions within the same information classThis ordering is sometimes expected like in the case of manyof our Class I2 rules where wave-like patterns are coarse-grained to either self-similar or very similar patterns Othertimes it looks more profound like in the case of WolframClass III rule 146 Rule 146 is in I1 and it can be nontriviallycoarse-grained to I1 rule 128 (a Wolfram Class I rule) dueto a shared sensitivity of the information processed in agiven space-time patch to its input state As noted in [1] thefact that potentially (a conclusive proof of the computationalirreducibility of ECA rule 146 is stillmissing) computationallyirreducible rules [8] like elementary rule 146 can be coarse-grained to predictable dynamics (eg rule 128) shows thatcomputational irreducibility might lack the characteristicsof a good measure of physical complexity We are led toconclude that the coarse-graining hierarchy is more likelydefined by conserved informational properties with morecomplex rules by Wolframrsquos classification appearing lower inthe hierarchy if they can be coarse-grained to less complexones with common sensitivity to the input state

5 Conclusions

Physical systems evolve in time according to local uniformrules but can nonetheless exhibit complex emergent behav-ior Complexity arises either as a result of the initial state orrule or some combination of the twoThe ambiguity of deter-mining whether complexity is generated by rules or states hasconfounded previous efforts to isolate complexity intrinsicto a given rule In this work we introduced a quantitativeinformation-based classification of ECA The classificationscheme proposed circumvents the difficulties arising due to

the sensitivity of the expressed complexity of ECA rules totheir initial state The (averaged) directed exchange of infor-mation (TE) between the individual parts of an ECA is usedas ameasure of its complexityThe identification of the single-cell input (Section 2) as the nontrivial state with the leastcomplexity is assumed as a working hypothesis and providesa reference point for our analysis of the degree of variabilityof the complexity of ECA rules for varying inputs Weidentified three distinct classes based on our analysis whichvary in their sensitivity to initial conditions Class I1 ECAalways process little information Class I3 always processeshigh information and Class I2 can be low or high dependingon the input It is only for Class I3 that the expressed com-plexity is intrinsic and not a product of the complexity carriedby the input

The most complex rules by our analysis are in Class I3which includes the majority of Wolframrsquos Class III rules andClass IV rule 110 and its equivalent rules These rules form aclosed group under the coarse-graining transitions found in[1] The truly complex rules are those that remain complexeven at a macrolevel of description with behavior that is notsensitive to the initial state

We are currently investigating the possibility of extendingour measure to less elementary dynamical systems as well asto idealized biological systems (eg networks of regulatorygenes) Inspired by Nursersquos idea [35] that the optimization ofinformation storage and flow might represent the distinctivefeature of biological functions we find interesting that infor-mation dynamics identifies the most complex ECA dynamicswith the ones that least depend on their initial state Thisfeature evocative of the redundancy so ubiquitous in biologymight represent a link between biology and informationdynamics [36] that we consider worth of further study

Disclosure

The opinions expressed in this publication are those of theauthors and do not necessarily reflect the views of TempletonWorld Charity Foundation

8 Complexity

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This publication was made possible through support ofa grant from Templeton World Charity Foundation Theauthors thank Douglas Moore and Hector Zenil for theirhelpful comments

References

[1] N Israeli and N Goldenfeld ldquoCoarse-graining of cellularautomata emergence and the predictability of complex sys-temsrdquo Physical Review E Statistical Nonlinear and Soft MatterPhysics vol 73 no 2 026203 17 pages 2006

[2] M Gardner ldquoMathematical gamesrdquo Scientific American vol223 pp 120ndash123 1970

[3] E R Berlekamp J H Conway and R K Guy Winning Waysfor Your Mathematical Plays Vol 2 Games in Particular vol 2Academic Press London UK 1982

[4] M GardnerWheels Life and other Mathematical AmusementsFreeman amp Company NY USA 1983

[5] I Smith ldquoSimple computation-universal cellular spacesrdquo Jour-nal of the Association for ComputingMachinery vol 18 pp 339ndash353 1971

[6] K Lindgren and M G Nordahl Complex Systems vol 4 pp299ndash318 1990

[7] M Cook ldquoUniversality in elementary cellular automatardquo Com-plex Systems vol 15 no 1 pp 1ndash40 2004

[8] S Wolfram ldquoCellular automata as models of complexityrdquoNature vol 311 no 5985 pp 419ndash424 1984

[9] J von Neumann Theory of Self-Reproducing Automata A WBurks University of Illinois Press Urbana ILL USA 1966

[10] K Culik and S Yu ldquoUndecidability of CA classificationschemesrdquo Complex Systems vol 2 pp 177ndash190 1988

[11] W Li and N Packard ldquoErratum ldquoThe structure of the elemen-tary cellular automata rule spacerdquordquo Complex Systems vol 4 no1 pp 281ndash297 1990

[12] C G Langton ldquoComputation at the edge of chaos phase transi-tions and emergent computationrdquo Physica D Nonlinear Phe-nomena vol 42 no 1ndash3 pp 12ndash37 1990

[13] F Bagnoli R Rechtman and S Ruffo ldquoDamage spreading andLyapunov exponents in cellular automatardquoPhysics Letters A vol172 no 1-2 pp 34ndash38 1992

[14] G Y Vichniac ldquoBoolean derivatives on cellular automatardquoPhysica D Nonlinear Phenomena vol 45 no 1-3 pp 63ndash741990

[15] J M Baetens and B De Baets ldquoPhenomenological study ofirregular cellular automata based on Lyapunov exponents andJacobiansrdquo Chaos An Interdisciplinary Journal of NonlinearScience vol 20 no 3 Article ID 033112 033112 15 pages 2010

[16] J M Baetens P Van Der Weeen and B De Baets ldquoEffect ofasynchronous updating on the stability of cellular automatardquoChaos Solitons and Fractals vol 45 no 4 pp 383ndash394 2012

[17] J M Baetens and B De Baets ldquoTopology-induced phase transi-tions in totalistic cellular automatardquo Physica D NonlinearPhenomena vol 249 pp 16ndash24 2013

[18] J M Baetens and J Gravner ldquoStability of cellular automatatrajectories revisited branching walks and Lyapunov profilesrdquoJournal of Nonlinear Science vol 26 no 5 pp 1329ndash1367 2016

[19] L O Chua S Yoon and R Dogaru ldquoA nonlinear dynamicsperspective of Wolframrsquos new kind of science I Threshold ofcomplexityrdquo International Journal of Bifurcation and Chaos inApplied Sciences and Engineering vol 12 no 12 pp 2655ndash27662002

[20] N Fates ldquoExperimental study of elementary cellular automatadynamics using the density parameterrdquo Discrete Mathematicsand Theoretical Computer Science pp 15ndash166 2003

[21] C Durr I Rapaport and G Theyssier Theoretical ComputerScience vol 322 pp 355ndash368 2004

[22] H Zenil and E Villarreal-Zapata ldquoAsymptotic behavior andratios of complexity in cellular automatardquo International Journalof Bifurcation and Chaos vol 23 no 9 Article ID 1350159 2013

[23] L Albantakis and G Tononi ldquoThe intrinsic cause-effect powerof discrete dynamical systems-from elementary cellular autom-ata to adapting animatsrdquo Entropy vol 17 no 8 pp 5472ndash55022015

[24] G J MartInez ldquoA note on elementary cellular automata classi-ficationrdquo Journal of Cellular Automata vol 8 no 3-4 pp 233ndash259 2013

[25] W Bialek Biophysics searching for principles Princeton Univer-sity Press 2012

[26] S I Walker H Kim and P C W Davies ldquoThe informationalarchitecture of the cellrdquo Philosophical Transactions of the RoyalSociety A vol 374 Article ID 20150057 2016

[27] T Schreiber ldquoMeasuring information transferrdquo Physical ReviewLetters vol 85 no 2 pp 461ndash464 2000

[28] Wolfram Alpha httpswwwwolframalphacom[29] J T LizierM Prokopenko andA Y Zomaya ldquoThe information

dynamics of phase transitions in random boolean networksrdquoin Proceedings of the 11th International Conference on theSimulation and Synthesis of Living Systems Artificial Life XIALIFE 2008 pp 374ndash381 2008

[30] J T Lizier and M Prokopenko The European Physical JournalB vol 73 2010

[31] X R Wang J M Miller J T Lizier M Prokopenko andL F Rossi ldquoMeasuring information storage and transfer inswarmsrdquo in Proceedings of the Eleventh European Conference onthe Synthesis and Simulation of Living Systems pp 838ndash8452011

[32] T Bossomaier L Barnett M Harre and J T Lizier AnIntroduction to Transfer Entropy Information Flow in ComplexSystems Springer 2016

[33] S Wolfram ldquoStatistical mechanics of cellular automatardquo Re-views of Modern Physics vol 55 no 3 pp 601ndash644 1983

[34] S Wolfram A New Kind of Science Wolfram Media Cham-paign ILL USA 2002

[35] P Nurse ldquoLife logic and informationrdquo Nature vol 454 no7203 pp 424ndash426 2008

[36] P C W Davies and S I Walker ldquoThe hidden simplicity ofbiologyrdquo Reports on Progress in Physics vol 79 no 10 ArticleID 102601 2016

Complexity 3

2 Intrinsic Complexity

In what follows we identify the complexity of a CA with theamount of information it processes during its time evolutionThe concept of information processing presented herein isadopted from information dynamics [29ndash32] a formalismthat quantifies computation in dynamical systems usingmethods from information theory In information dynam-ics Schreiberrsquos transfer entropy [27] is identified with theinformation processed between two elements of a dynamicalsystem In Schreiberrsquos measure the transfer of informationbetween a source 119884 and a target119883 is quantified as the reduc-tion in uncertainty of the future state of 119883 due to knowledgeof the state of 119884 In a CA the source 119884 and target 119883 are bothcells of the CA

Formally TE is defined as the mutual information be-tween the source119884 at a certain time 119905 and the target119883 at time119905 + 1 conditioned on a given number 119896 of previous states of119883

119879119884rarr119883 = sum(x(119896)119905 119909119905+1 119910119905)isinS

119901 (x(119896)119905 119909119905+1 119910119905)

times log2119901 (119909119905+1 | x(119896)119905 119910119905)119901 (119909119905+1 | x(119896)119905 )

(1)

where (see Figure 1)

(i) 119909119905+1 represents the state of cell119883 at time 119905 + 1(ii) 119910119905 is the state of cell 119884 at the previous time 119905(iii) x(119896)119905 is the vector (119909119905minus119896+1 119909119905) of the previous 119896

states of cell119883(iv) S represents the set of all possible patterns of sets of

states (x(119896)119905 119909119905+1 119910119905)To calculate TE one must start with the time series of

states of a CA Given the time series (see Figure 1) the occur-rence numbers of patterns of states (x(119896)119905 119909119905+1 119910119905) are countedfor each combination of cells 119883 and 119884 in the array Oncenormalized to the total number of (not necessarily distinct)patterns appearing in the time series these frequencies areidentified with the probabilities 119901(x(119896)119905 119909119905+1 119910119905) appearingin (1) The conditional probabilities 119901(119909119905+1 | x(119896)119905 119910119905) and119901(119909119905+1 | x(119896)119905 ) are calculated analogously

Changing the value of the history length can affect thevalue of TE calculated for the same time series data We findvery similar values of TE for 119896 = 4 and 6 and observe a con-siderable decrease in TE for 119896 lt 4 and 119896 gt 6 (cf Section 42 of[32]) relative to the value of TE for 119896 = 5 We thereforeconsider 119896 = 5 as the optimal value of 119896 that properly cap-tures the past history of 119883 for the results presented here (seeSection 3 for a visual explanation of why 119896 sim 5 represents theoptimal history length)

We first generated time series for the simplest initial statefor each of the 256 possible ECA rules which we numericallylabel following Wolframrsquos numbering scheme [33 34] Hereand in the following periodic boundary conditions are

ECA array

t

Tim

e

t + 1

t minus k + 1

CellCell XY

x(k)t

xt+1

yt

Figure 1 Time series of an ECA with the pattern of cells relevantto the evaluation of the transfer entropy from 119884 to 119883 119879119884rarr119883 high-lighted

enforcedTherefore the specific location of the different colorcell in the input array is irrelevant Our initial state is either

◻ sdot sdot sdot ◻⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟50-times

◼◻ sdot sdot sdot ◻⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟50-times

(2)

or the equivalent state obtained through a ◻ harr ◼ conju-gation For example rules 18 and 183 are equivalent undera ◻ harr ◼ conjugation The input ◻ sdot sdot sdot ◻◼◻ sdot sdot sdot ◻ is updatedusing rule 18 and the conjugated input ◼ sdot sdot sdot ◼◻◼ sdot sdot sdot ◼ usingrule 183 A comprehensive study of the symmetry propertiesof ECA can be found in [11] where the 256 possible ECArules are grouped into 88 different equivalence classes Eachclass contains no more than four rules which show the samebehavior under ◻ harr ◼ exchange left-right transformationsor the combination of both transformations The interestedreader can find such classification summarized in Table 1 ofthe cited work For Wolfram Classes III and IV (Wolframrsquosmost recent ECA classification scheme is now implementedin the Wolfram Alpha computational engine [28] and repro-duced here (Table 1) for the convenience of the reader) theequivalence classes are also shown in the legend of Figure 2

We calculate the amount of information processed by a CAin a space-time patch as the average of 119879119884rarr119883 calculated overthat region designated as ⟨TE⟩ To do so we evolve theCA for250 time steps and then remove the first 50 time steps fromeach generated time seriesThis allows us to evaluate TE overa region of the CA in causal contact with the initial input ateach point in space and time This ensures that the observeddynamics over the relevant space-time patch are driven byboth the chosen rule and the initial state The resulting timeseries is then used to evaluate the 1012 values TE119884rarr119883 Foreach rule the average value ⟨TE⟩ is shown in Figure 2(a)Equivalent rules like rules 30 86 135 and 149 producethe same values of ⟨TE⟩ and are represented by a commonmarker For clarity individual Wolfram Classes I and II rulesare not shown and are instead replaced by a dashed linecorresponding to the highest ⟨TE⟩ of any individual Class Ior II rule All Class III rules lie above the range of Classes Iand II Interestingly with only the exception of rule 110 andits equivalent rules all Class IV rules lie within the range ofClasses I and II rules

4 Complexity

0 50 100 150 200 250

Rule number

000

005

010

015

020Av

erag

ed tr

ansfe

r ent

ropy

(bits

)

18 18322 15130 86 135 14945 75 89 10160 102 153 19590 165105

122 161126 129146 18215054 147106 120 169 225110 124 137 193

Clas

ses I

amp II

rang

e

(a) Single-cell input

0 50 100 150 200 250

Rule number

Clas

ses I

amp II

rang

e

000

005

010

015

020

Aver

aged

tran

sfer e

ntro

py (b

its)

18 18322 15130 86 135 14945 75 89 10160 102 153 19590 165105

122 161126 129146 18215054 147106 120 169 225110 124 137 193

(b) Random input

Figure 2 ⟨TE⟩ for each rule shown for two different inputs single cell (a) (either one black cell or one white cell) and random (b) Equivalentrules are represented by a common marker The association of equivalent inputs with equivalent rules is chosen heuristically as the one thatmaximizes the averaged TE The dashed line corresponds to the highest value of ⟨TE⟩ for any Wolfram Class I or II rule Individual markersare shown only for Wolfram Classes III (black) and IV (orange) rules

The single-cell input considered here is extremely rarewithin the space of all possible initial inputs The numberof black cells in a state randomly extracted among the 2101different possible inputs follows a binomial distributionmeaning that states containing 50 or 51 black and white cellsare about 2times1027 times more likely than our initially selectedinput Our motivation for considering the single-cell inputfirst is that it automatically excludes many trivial rules fromour search for the complex ones Rules that duplicate theinput in a trivial way or annihilate it to produce all ◼ or all◻ naturally yield ⟨TE⟩ ≃ 0 This is the same approach thathas been recently assumed in algorithmic complexity basedclassification of CAs [22] It has the advantage of selectingmany rules according to their intrinsic complexity and notthe complexity of the input However a major shortcomingof choosing the single-cell input is that many Class IV ruleslook simpler than what they truly are Class IV rule 106 isa good example For the simple input of a single black bitrule 106 functions to shift the black cell one bit to the leftat each time-step generating a trivial trajectory However incases where the input allows two black cells to interact thefull potential of rule 106 can be expressed This is an explicitexample of the sensitivity of the behavior of some ECA rules

to the complexity of their input as discussed in the introduc-tion To isolate the complexity of the rule from that inheritedby the input we must therefore consider a random sample ofinitial states as we do next

3 Inherited Complexity andInformation-Based Classification

We next consider a more generic input randomly selectedamong the 2101 ≃ 25times1030 different possibilities As the inputis no longer symmetric we now need to consider reflec-tions in selecting the equivalent input rarr rule associationsThe scenario changes completely in this case as shown inFigure 2(b) where the highest values of ⟨TE⟩ correspondto Class II rules including 15 85 170 and 240 whichgenerate trivial wave-like behaviors These rules behavelike rule 106 (especially 170 and 240) when initialized witha single-cell input but they do not contribute any newemergent nontrivial features when nearby cells interact Forall purposes they appear less intrinsically complex With onlythe exception of rule 110 and its equivalent rules Class IVrules behave like many rules of Class II and exhibit a largeincrease in complexity as qualitatively observed and also

Complexity 5

as quantitatively captured by ⟨TE⟩ in response to a morecomplex input

It is worth noting that all of the rules whose initial valueof ⟨TE⟩ lay above the upper limit for Classes I and II of⟨TE⟩ = 0024 bits in the simplest input scenario still havecalculated ⟨TE⟩ above this value under a change of input Inparticular the rules with the highest values of ⟨TE⟩ are notsignificantly affected by the change in the input Let us naivelyuse the upper limit for Classes I and II rules emerging fromFigure 2(a) as the border line between what we call low andhigh values of ⟨TE⟩ We can summarize the changes undervarying the input as follows There are rules whose value of⟨TE⟩ is low for both the inputs There are rules whose valueof ⟨TE⟩ is low in the simplest case but high in the morecomplex one And finally there are also rules with a highvalue of ⟨TE⟩ in both cases The interesting point is that wefind no rule whose value of ⟨119879119864⟩ is high for the simplest inputand low for the random input This is equivalent to say thatthere are no rules generating complexity for simple inputsbut annihilating it for complex ones The significance of thisobservation lies in the fact that it enables classification interms of the shift of ⟨TE⟩ over a space-time region in responseto a change in its input by taking advantage of the mainlimitation that makes quantitative classifications of ECAs sodifficult that is ECA sensitivity to their initial state andexploits it in order to achieve such classification

To confirm this is indeed a viable method for quantitativeclassificationwemust considermore than just two inputsWetherefore first randomly selected twenty different ones Beinginterested in how much ⟨TE⟩ can vary as we vary the inputfor each rule we selected the maximum absolute value of thepercent change of ⟨TE⟩ between the random inputs (⟨TE⟩119903)and the single-cell input (⟨TE⟩1) considered before

max1003816100381610038161003816⟨TE⟩119903 minus ⟨TE⟩11003816100381610038161003816⟨TE⟩119903 randominputs

(3)

The results are shown in Figure 3 where the maximum per-cent change is plotted as a function of ⟨TE⟩1 Equivalent rulesare represented by a single marker and share the same valuefor the maximum relative change of ⟨TE⟩ In fact given tworules 1199031 and 1199032 sharing a symmetry it is always possible to findtwo inputs 1198941 and 1198942 such that rule 1199031 initialized to 1198941 and rule1199032 initialized to 1198942 give rise to the same value of ⟨TE⟩

The horizontal dashed line separates rules whose maxi-mum ⟨TE⟩ change is less than one order of magnitude fromthose that can undergo a change in ⟨TE⟩ of at least a factor 10The vertical dashed line denotes the highest value of ⟨TE⟩1 forWolframClasses I and II exactly as in Figure 2(a)The regionto the right of the vertical line and above the horizontal line isvoid of anyECA rules Points in that regionwould correspondto values of ⟨TE⟩ that are both high for the simplest input andcapable of high variation for exampleCArules that can anni-hilate the complexity of the input which we do not observeThis feature yields a distinctive L-shape in the distribu-tion of ruleswith the rules in each region sharingwell-definedproperties

As a result a natural information-based classification ofECAs can be given as follows

Class I1 ⟨TE⟩ is very small for the simplest input andstays so for complex inputsThis is themost populatedclass including almost all Wolfram Classes I and IIrules rules 18 and 146 and their equivalent Class IIIrules

Class I2 ⟨TE⟩ is small for the simplest input but itexperiences a drastic change (one order of magnitudeor more) when the input is complex This is the casefor many Wolfram Class II and some Class IV rules(eg 54 and 106 and their equivalent rules)

Class I3 ⟨TE⟩ has a high value for the simplest inputand this value is approximately unaffected by a changein the input Most Wolfram Class III rules belong tothis class as well as Class IV rule 110 and its equivalentrules

Our classification is summarized in Table 2 Despite thearbitrary placement of our boundary lines at the border ofthe I1 region it shows an unambiguous separation betweenthe most majority of rules classified as I2 and I3

Randomly sampling inputs leads to a bias favoring nearlyfifty-fifty distributions of black and white cells due to theirbinomial distribution We therefore also verified this clas-sification using a different distribution of inputs where thenumber of black cells is increased in a regular way from 2to 50 while keeping the specific positions of black cells inthe input array random We considered 20 different inputseach containing exactly 2 5 7 10 12 47 and 50 blackcells (higher numbers are not considered due to the ◻ harr◼ conjugation) Apart from minor shifts of the data pointsapplying the same procedure as above yields exactly the sameclassification as Figure 3 indicating that our classificationscheme is robust

We stress the importance of considering a large system(order sim 100 cells) as opposed to a much smaller one (egordersim 10)While for the latter a scan over the entire space ofinputs is computationally feasible (and indeed we performedthese experiments) it hides one of the main features enablinginformation-based classification the existence of Class I3rules which form themost stable class with respect to our TEbased complexity measure Class I3 rules produce time serieslargely independent of the initial state exemplified by rule30 in Figure 3 a feature evident only in larger systems (beingthe result of combinatorial calculus TE computational timegrows exponentially with the number of cells considered AnECA array of about 100 cells is both the smallest size that wecould safely consider much larger than that of the domainsof Figure 3 and for which a complete scan over the rule spaceand a large enough ensemble of inputs could be made Wealso considered individual runs with 200 and 400 cells andnoticed that our numerical results are stable for CA with 100or more cells but are fluctuating for CA with 50 or less cells)

The typical order 10 cell length of the domains appearingin Figure 3 is also the reason why 119896 sim 5 is the most optimalvalue for the history length parameter Smaller and largervalues of 119896 would not be able to resolve this lattice sizeindependent feature of Class I3 rules [32] (Figure 4)

6 Complexity

Table 2 Information based classification of ECA with only the 88 rule equivalence classes shown (represented by the lowest number rule)Wolfram Class III rules are denoted in black boldface type and Class IV in italic

Class I10 1 3 4 5 8 9 12 13 1819 23 25 26 28 29 32 33 35 3637 40 41 44 50 51 57 58 62 7273 76 77 78 94 104 108 128 130 132134 136 140 146 152 154 156 160 162 164178 200 204 232

Class I22 6 7 10 11 14 15 24 27 3438 42 43 46 54 56 74 106 138 142168 170 172 184

Class I322 30 45 60 90 105 110 122 126 150

0

20

40

60

80

100

Max

imum

per

cent

chan

ge o

f ⟨TE

⟩

005 010 015 020000

⟨TE⟩ (single-cell input bits)

Class I1 Class I3

Class I2Random inputs

Figure 3 Maximum percent change of ⟨TE⟩ as a function of ⟨TE⟩1(3) Equivalent rules are represented by a singlemarker colored as inFigure 2 but with Wolfram Classes I and II in gray Rules above thehorizontal dashed line can undergo a change of at least one orderof magnitude in ⟨TE⟩ The vertical dashed line is the same as inFigure 2(a)

4 Coarse-Graining andthe Information Hierarchy

Perhaps the most interesting feature of our quantitativeclassification is that Wolfram Class III and Class IV rulesare distributed over different information-based classes Thisbehavior looks less surprising in the light of the coarse-graining transitions among ECA uncovered by Israeli andGoldenfeld [1] One important aspect of the physical worldis that coarse-grained descriptions often suffice to make

Rule 30single cell input

Rule 30random input

Figure 4 Time series for rule 30 generated by the two initial statesof Figure 2 The similarity of the two outputs each yielding colordomains with a typical linear size of about sim10 blocks does notdepend on the size of the CA and is responsible for the stability ofClass I3 rules to changes in input

predictions about relevant behavior Noting this Israeli andGoldenfeld adopted a systematic procedure to try to coarsegrain ECA rules Their prescription consists in groupingtogether nearby cells in theCAarray into a supercell of a givensize 119873 according to some specified supercell rarr single-cellcorrespondence (for Boolean CAs 22119873 possible applicationsof this kind exist) and a given time constant 119879 The searchfor a coarse-graining rule consists in looking for a new CArule such that running the original CA for 119879119905 time steps andthen grouping the supercells produces the same output asgrouping the initial array and then adopting the new CA rulefor 119905 time steps The new CA rule the time constant 119879 andthe size of the supercell 119873 are all unknown therefore thissearch requires a scan over all the possible combinations ofthese parameters Israeli and Goldenfeld successfully coarse-grained 240 of the 256 ECA rules many to other ECA rulesperforming a complete scan of all the possibilities compatiblewith 119873 le 4 Importantly the rule complexity was neverobserved to increase under coarse-graining introducing apartial ordering among CA rules

Complexity 7

Class I1

Class I2

Class I3 60 90 105 150 126

2 42170

6 7 11 14 24 27 5674 168 184

15

34

5

51

140

13

152136

156

0

25 26 37 41 5762 73 94 104154 164 204

128 44 130 132 162

1 4 12 1929 72 76

23 77 134 146160 178 232

200

204

3 8 9 18 26 28 3233 35 36 40 50 5878 108

172 10 38 46 138

22 30 45 110 122

43 54 106 142

Figure 5 Coarse-graining transitions within ECA found in [1] mapped onto a hierarchy of the information-based classes identified hereOnly the 88 rule equivalency classes are shown (represented by the lowest number rule) For simplicity (nontrivial) transitions to rule zeroare not shown (isolated boxed with continuous border) Boxes with a dashed border include ECA rules for which a coarse-graining transitionto another ECA rule with supercell of size 4 or less does not exist [1]

The same ordering emerges from our information-basedclassification as shown in Figure 5 where arrows indicatecoarse-graining transitions uncovered in [1] These transi-tions introduce a fully ordered hierarchy I3 rarr I2 rarr I1such that coarse-graining is never observed to move up thehierarchy and the vast majority of rules may only undergocoarse-graining transitions within the same information classThis ordering is sometimes expected like in the case of manyof our Class I2 rules where wave-like patterns are coarse-grained to either self-similar or very similar patterns Othertimes it looks more profound like in the case of WolframClass III rule 146 Rule 146 is in I1 and it can be nontriviallycoarse-grained to I1 rule 128 (a Wolfram Class I rule) dueto a shared sensitivity of the information processed in agiven space-time patch to its input state As noted in [1] thefact that potentially (a conclusive proof of the computationalirreducibility of ECA rule 146 is stillmissing) computationallyirreducible rules [8] like elementary rule 146 can be coarse-grained to predictable dynamics (eg rule 128) shows thatcomputational irreducibility might lack the characteristicsof a good measure of physical complexity We are led toconclude that the coarse-graining hierarchy is more likelydefined by conserved informational properties with morecomplex rules by Wolframrsquos classification appearing lower inthe hierarchy if they can be coarse-grained to less complexones with common sensitivity to the input state

5 Conclusions

Physical systems evolve in time according to local uniformrules but can nonetheless exhibit complex emergent behav-ior Complexity arises either as a result of the initial state orrule or some combination of the twoThe ambiguity of deter-mining whether complexity is generated by rules or states hasconfounded previous efforts to isolate complexity intrinsicto a given rule In this work we introduced a quantitativeinformation-based classification of ECA The classificationscheme proposed circumvents the difficulties arising due to

the sensitivity of the expressed complexity of ECA rules totheir initial state The (averaged) directed exchange of infor-mation (TE) between the individual parts of an ECA is usedas ameasure of its complexityThe identification of the single-cell input (Section 2) as the nontrivial state with the leastcomplexity is assumed as a working hypothesis and providesa reference point for our analysis of the degree of variabilityof the complexity of ECA rules for varying inputs Weidentified three distinct classes based on our analysis whichvary in their sensitivity to initial conditions Class I1 ECAalways process little information Class I3 always processeshigh information and Class I2 can be low or high dependingon the input It is only for Class I3 that the expressed com-plexity is intrinsic and not a product of the complexity carriedby the input

The most complex rules by our analysis are in Class I3which includes the majority of Wolframrsquos Class III rules andClass IV rule 110 and its equivalent rules These rules form aclosed group under the coarse-graining transitions found in[1] The truly complex rules are those that remain complexeven at a macrolevel of description with behavior that is notsensitive to the initial state

We are currently investigating the possibility of extendingour measure to less elementary dynamical systems as well asto idealized biological systems (eg networks of regulatorygenes) Inspired by Nursersquos idea [35] that the optimization ofinformation storage and flow might represent the distinctivefeature of biological functions we find interesting that infor-mation dynamics identifies the most complex ECA dynamicswith the ones that least depend on their initial state Thisfeature evocative of the redundancy so ubiquitous in biologymight represent a link between biology and informationdynamics [36] that we consider worth of further study

Disclosure

The opinions expressed in this publication are those of theauthors and do not necessarily reflect the views of TempletonWorld Charity Foundation

8 Complexity

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This publication was made possible through support ofa grant from Templeton World Charity Foundation Theauthors thank Douglas Moore and Hector Zenil for theirhelpful comments

References

[1] N Israeli and N Goldenfeld ldquoCoarse-graining of cellularautomata emergence and the predictability of complex sys-temsrdquo Physical Review E Statistical Nonlinear and Soft MatterPhysics vol 73 no 2 026203 17 pages 2006

[2] M Gardner ldquoMathematical gamesrdquo Scientific American vol223 pp 120ndash123 1970

[3] E R Berlekamp J H Conway and R K Guy Winning Waysfor Your Mathematical Plays Vol 2 Games in Particular vol 2Academic Press London UK 1982

[4] M GardnerWheels Life and other Mathematical AmusementsFreeman amp Company NY USA 1983

[5] I Smith ldquoSimple computation-universal cellular spacesrdquo Jour-nal of the Association for ComputingMachinery vol 18 pp 339ndash353 1971

[6] K Lindgren and M G Nordahl Complex Systems vol 4 pp299ndash318 1990

[7] M Cook ldquoUniversality in elementary cellular automatardquo Com-plex Systems vol 15 no 1 pp 1ndash40 2004

[8] S Wolfram ldquoCellular automata as models of complexityrdquoNature vol 311 no 5985 pp 419ndash424 1984

[9] J von Neumann Theory of Self-Reproducing Automata A WBurks University of Illinois Press Urbana ILL USA 1966

[10] K Culik and S Yu ldquoUndecidability of CA classificationschemesrdquo Complex Systems vol 2 pp 177ndash190 1988

[11] W Li and N Packard ldquoErratum ldquoThe structure of the elemen-tary cellular automata rule spacerdquordquo Complex Systems vol 4 no1 pp 281ndash297 1990

[12] C G Langton ldquoComputation at the edge of chaos phase transi-tions and emergent computationrdquo Physica D Nonlinear Phe-nomena vol 42 no 1ndash3 pp 12ndash37 1990

[13] F Bagnoli R Rechtman and S Ruffo ldquoDamage spreading andLyapunov exponents in cellular automatardquoPhysics Letters A vol172 no 1-2 pp 34ndash38 1992

[14] G Y Vichniac ldquoBoolean derivatives on cellular automatardquoPhysica D Nonlinear Phenomena vol 45 no 1-3 pp 63ndash741990

[15] J M Baetens and B De Baets ldquoPhenomenological study ofirregular cellular automata based on Lyapunov exponents andJacobiansrdquo Chaos An Interdisciplinary Journal of NonlinearScience vol 20 no 3 Article ID 033112 033112 15 pages 2010

[16] J M Baetens P Van Der Weeen and B De Baets ldquoEffect ofasynchronous updating on the stability of cellular automatardquoChaos Solitons and Fractals vol 45 no 4 pp 383ndash394 2012

[17] J M Baetens and B De Baets ldquoTopology-induced phase transi-tions in totalistic cellular automatardquo Physica D NonlinearPhenomena vol 249 pp 16ndash24 2013

[18] J M Baetens and J Gravner ldquoStability of cellular automatatrajectories revisited branching walks and Lyapunov profilesrdquoJournal of Nonlinear Science vol 26 no 5 pp 1329ndash1367 2016

[19] L O Chua S Yoon and R Dogaru ldquoA nonlinear dynamicsperspective of Wolframrsquos new kind of science I Threshold ofcomplexityrdquo International Journal of Bifurcation and Chaos inApplied Sciences and Engineering vol 12 no 12 pp 2655ndash27662002

[20] N Fates ldquoExperimental study of elementary cellular automatadynamics using the density parameterrdquo Discrete Mathematicsand Theoretical Computer Science pp 15ndash166 2003

[21] C Durr I Rapaport and G Theyssier Theoretical ComputerScience vol 322 pp 355ndash368 2004

[22] H Zenil and E Villarreal-Zapata ldquoAsymptotic behavior andratios of complexity in cellular automatardquo International Journalof Bifurcation and Chaos vol 23 no 9 Article ID 1350159 2013

[23] L Albantakis and G Tononi ldquoThe intrinsic cause-effect powerof discrete dynamical systems-from elementary cellular autom-ata to adapting animatsrdquo Entropy vol 17 no 8 pp 5472ndash55022015

[24] G J MartInez ldquoA note on elementary cellular automata classi-ficationrdquo Journal of Cellular Automata vol 8 no 3-4 pp 233ndash259 2013

[25] W Bialek Biophysics searching for principles Princeton Univer-sity Press 2012

[26] S I Walker H Kim and P C W Davies ldquoThe informationalarchitecture of the cellrdquo Philosophical Transactions of the RoyalSociety A vol 374 Article ID 20150057 2016

[27] T Schreiber ldquoMeasuring information transferrdquo Physical ReviewLetters vol 85 no 2 pp 461ndash464 2000

[28] Wolfram Alpha httpswwwwolframalphacom[29] J T LizierM Prokopenko andA Y Zomaya ldquoThe information

dynamics of phase transitions in random boolean networksrdquoin Proceedings of the 11th International Conference on theSimulation and Synthesis of Living Systems Artificial Life XIALIFE 2008 pp 374ndash381 2008

[30] J T Lizier and M Prokopenko The European Physical JournalB vol 73 2010

[31] X R Wang J M Miller J T Lizier M Prokopenko andL F Rossi ldquoMeasuring information storage and transfer inswarmsrdquo in Proceedings of the Eleventh European Conference onthe Synthesis and Simulation of Living Systems pp 838ndash8452011

[32] T Bossomaier L Barnett M Harre and J T Lizier AnIntroduction to Transfer Entropy Information Flow in ComplexSystems Springer 2016

[33] S Wolfram ldquoStatistical mechanics of cellular automatardquo Re-views of Modern Physics vol 55 no 3 pp 601ndash644 1983

[34] S Wolfram A New Kind of Science Wolfram Media Cham-paign ILL USA 2002

[35] P Nurse ldquoLife logic and informationrdquo Nature vol 454 no7203 pp 424ndash426 2008

[36] P C W Davies and S I Walker ldquoThe hidden simplicity ofbiologyrdquo Reports on Progress in Physics vol 79 no 10 ArticleID 102601 2016

4 Complexity

0 50 100 150 200 250

Rule number

000

005

010

015

020Av

erag

ed tr

ansfe

r ent

ropy

(bits

)

18 18322 15130 86 135 14945 75 89 10160 102 153 19590 165105

122 161126 129146 18215054 147106 120 169 225110 124 137 193

Clas

ses I

amp II

rang

e

(a) Single-cell input

0 50 100 150 200 250

Rule number

Clas

ses I

amp II

rang

e

000

005

010

015

020

Aver

aged

tran

sfer e

ntro

py (b

its)

18 18322 15130 86 135 14945 75 89 10160 102 153 19590 165105

122 161126 129146 18215054 147106 120 169 225110 124 137 193

(b) Random input

Figure 2 ⟨TE⟩ for each rule shown for two different inputs single cell (a) (either one black cell or one white cell) and random (b) Equivalentrules are represented by a common marker The association of equivalent inputs with equivalent rules is chosen heuristically as the one thatmaximizes the averaged TE The dashed line corresponds to the highest value of ⟨TE⟩ for any Wolfram Class I or II rule Individual markersare shown only for Wolfram Classes III (black) and IV (orange) rules

The single-cell input considered here is extremely rarewithin the space of all possible initial inputs The numberof black cells in a state randomly extracted among the 2101different possible inputs follows a binomial distributionmeaning that states containing 50 or 51 black and white cellsare about 2times1027 times more likely than our initially selectedinput Our motivation for considering the single-cell inputfirst is that it automatically excludes many trivial rules fromour search for the complex ones Rules that duplicate theinput in a trivial way or annihilate it to produce all ◼ or all◻ naturally yield ⟨TE⟩ ≃ 0 This is the same approach thathas been recently assumed in algorithmic complexity basedclassification of CAs [22] It has the advantage of selectingmany rules according to their intrinsic complexity and notthe complexity of the input However a major shortcomingof choosing the single-cell input is that many Class IV ruleslook simpler than what they truly are Class IV rule 106 isa good example For the simple input of a single black bitrule 106 functions to shift the black cell one bit to the leftat each time-step generating a trivial trajectory However incases where the input allows two black cells to interact thefull potential of rule 106 can be expressed This is an explicitexample of the sensitivity of the behavior of some ECA rules

to the complexity of their input as discussed in the introduc-tion To isolate the complexity of the rule from that inheritedby the input we must therefore consider a random sample ofinitial states as we do next

3 Inherited Complexity andInformation-Based Classification

We next consider a more generic input randomly selectedamong the 2101 ≃ 25times1030 different possibilities As the inputis no longer symmetric we now need to consider reflec-tions in selecting the equivalent input rarr rule associationsThe scenario changes completely in this case as shown inFigure 2(b) where the highest values of ⟨TE⟩ correspondto Class II rules including 15 85 170 and 240 whichgenerate trivial wave-like behaviors These rules behavelike rule 106 (especially 170 and 240) when initialized witha single-cell input but they do not contribute any newemergent nontrivial features when nearby cells interact Forall purposes they appear less intrinsically complex With onlythe exception of rule 110 and its equivalent rules Class IVrules behave like many rules of Class II and exhibit a largeincrease in complexity as qualitatively observed and also

Complexity 5

as quantitatively captured by ⟨TE⟩ in response to a morecomplex input

It is worth noting that all of the rules whose initial valueof ⟨TE⟩ lay above the upper limit for Classes I and II of⟨TE⟩ = 0024 bits in the simplest input scenario still havecalculated ⟨TE⟩ above this value under a change of input Inparticular the rules with the highest values of ⟨TE⟩ are notsignificantly affected by the change in the input Let us naivelyuse the upper limit for Classes I and II rules emerging fromFigure 2(a) as the border line between what we call low andhigh values of ⟨TE⟩ We can summarize the changes undervarying the input as follows There are rules whose value of⟨TE⟩ is low for both the inputs There are rules whose valueof ⟨TE⟩ is low in the simplest case but high in the morecomplex one And finally there are also rules with a highvalue of ⟨TE⟩ in both cases The interesting point is that wefind no rule whose value of ⟨119879119864⟩ is high for the simplest inputand low for the random input This is equivalent to say thatthere are no rules generating complexity for simple inputsbut annihilating it for complex ones The significance of thisobservation lies in the fact that it enables classification interms of the shift of ⟨TE⟩ over a space-time region in responseto a change in its input by taking advantage of the mainlimitation that makes quantitative classifications of ECAs sodifficult that is ECA sensitivity to their initial state andexploits it in order to achieve such classification

To confirm this is indeed a viable method for quantitativeclassificationwemust considermore than just two inputsWetherefore first randomly selected twenty different ones Beinginterested in how much ⟨TE⟩ can vary as we vary the inputfor each rule we selected the maximum absolute value of thepercent change of ⟨TE⟩ between the random inputs (⟨TE⟩119903)and the single-cell input (⟨TE⟩1) considered before

max1003816100381610038161003816⟨TE⟩119903 minus ⟨TE⟩11003816100381610038161003816⟨TE⟩119903 randominputs

(3)

The results are shown in Figure 3 where the maximum per-cent change is plotted as a function of ⟨TE⟩1 Equivalent rulesare represented by a single marker and share the same valuefor the maximum relative change of ⟨TE⟩ In fact given tworules 1199031 and 1199032 sharing a symmetry it is always possible to findtwo inputs 1198941 and 1198942 such that rule 1199031 initialized to 1198941 and rule1199032 initialized to 1198942 give rise to the same value of ⟨TE⟩

The horizontal dashed line separates rules whose maxi-mum ⟨TE⟩ change is less than one order of magnitude fromthose that can undergo a change in ⟨TE⟩ of at least a factor 10The vertical dashed line denotes the highest value of ⟨TE⟩1 forWolframClasses I and II exactly as in Figure 2(a)The regionto the right of the vertical line and above the horizontal line isvoid of anyECA rules Points in that regionwould correspondto values of ⟨TE⟩ that are both high for the simplest input andcapable of high variation for exampleCArules that can anni-hilate the complexity of the input which we do not observeThis feature yields a distinctive L-shape in the distribu-tion of ruleswith the rules in each region sharingwell-definedproperties

As a result a natural information-based classification ofECAs can be given as follows

Class I1 ⟨TE⟩ is very small for the simplest input andstays so for complex inputsThis is themost populatedclass including almost all Wolfram Classes I and IIrules rules 18 and 146 and their equivalent Class IIIrules

Class I2 ⟨TE⟩ is small for the simplest input but itexperiences a drastic change (one order of magnitudeor more) when the input is complex This is the casefor many Wolfram Class II and some Class IV rules(eg 54 and 106 and their equivalent rules)

Class I3 ⟨TE⟩ has a high value for the simplest inputand this value is approximately unaffected by a changein the input Most Wolfram Class III rules belong tothis class as well as Class IV rule 110 and its equivalentrules

Our classification is summarized in Table 2 Despite thearbitrary placement of our boundary lines at the border ofthe I1 region it shows an unambiguous separation betweenthe most majority of rules classified as I2 and I3

Randomly sampling inputs leads to a bias favoring nearlyfifty-fifty distributions of black and white cells due to theirbinomial distribution We therefore also verified this clas-sification using a different distribution of inputs where thenumber of black cells is increased in a regular way from 2to 50 while keeping the specific positions of black cells inthe input array random We considered 20 different inputseach containing exactly 2 5 7 10 12 47 and 50 blackcells (higher numbers are not considered due to the ◻ harr◼ conjugation) Apart from minor shifts of the data pointsapplying the same procedure as above yields exactly the sameclassification as Figure 3 indicating that our classificationscheme is robust

We stress the importance of considering a large system(order sim 100 cells) as opposed to a much smaller one (egordersim 10)While for the latter a scan over the entire space ofinputs is computationally feasible (and indeed we performedthese experiments) it hides one of the main features enablinginformation-based classification the existence of Class I3rules which form themost stable class with respect to our TEbased complexity measure Class I3 rules produce time serieslargely independent of the initial state exemplified by rule30 in Figure 3 a feature evident only in larger systems (beingthe result of combinatorial calculus TE computational timegrows exponentially with the number of cells considered AnECA array of about 100 cells is both the smallest size that wecould safely consider much larger than that of the domainsof Figure 3 and for which a complete scan over the rule spaceand a large enough ensemble of inputs could be made Wealso considered individual runs with 200 and 400 cells andnoticed that our numerical results are stable for CA with 100or more cells but are fluctuating for CA with 50 or less cells)

The typical order 10 cell length of the domains appearingin Figure 3 is also the reason why 119896 sim 5 is the most optimalvalue for the history length parameter Smaller and largervalues of 119896 would not be able to resolve this lattice sizeindependent feature of Class I3 rules [32] (Figure 4)

6 Complexity

Table 2 Information based classification of ECA with only the 88 rule equivalence classes shown (represented by the lowest number rule)Wolfram Class III rules are denoted in black boldface type and Class IV in italic

Class I10 1 3 4 5 8 9 12 13 1819 23 25 26 28 29 32 33 35 3637 40 41 44 50 51 57 58 62 7273 76 77 78 94 104 108 128 130 132134 136 140 146 152 154 156 160 162 164178 200 204 232

Class I22 6 7 10 11 14 15 24 27 3438 42 43 46 54 56 74 106 138 142168 170 172 184

Class I322 30 45 60 90 105 110 122 126 150

0

20

40

60

80

100

Max

imum

per

cent

chan

ge o

f ⟨TE

⟩

005 010 015 020000

⟨TE⟩ (single-cell input bits)

Class I1 Class I3

Class I2Random inputs

Figure 3 Maximum percent change of ⟨TE⟩ as a function of ⟨TE⟩1(3) Equivalent rules are represented by a singlemarker colored as inFigure 2 but with Wolfram Classes I and II in gray Rules above thehorizontal dashed line can undergo a change of at least one orderof magnitude in ⟨TE⟩ The vertical dashed line is the same as inFigure 2(a)

4 Coarse-Graining andthe Information Hierarchy

Perhaps the most interesting feature of our quantitativeclassification is that Wolfram Class III and Class IV rulesare distributed over different information-based classes Thisbehavior looks less surprising in the light of the coarse-graining transitions among ECA uncovered by Israeli andGoldenfeld [1] One important aspect of the physical worldis that coarse-grained descriptions often suffice to make

Rule 30single cell input

Rule 30random input

Figure 4 Time series for rule 30 generated by the two initial statesof Figure 2 The similarity of the two outputs each yielding colordomains with a typical linear size of about sim10 blocks does notdepend on the size of the CA and is responsible for the stability ofClass I3 rules to changes in input

predictions about relevant behavior Noting this Israeli andGoldenfeld adopted a systematic procedure to try to coarsegrain ECA rules Their prescription consists in groupingtogether nearby cells in theCAarray into a supercell of a givensize 119873 according to some specified supercell rarr single-cellcorrespondence (for Boolean CAs 22119873 possible applicationsof this kind exist) and a given time constant 119879 The searchfor a coarse-graining rule consists in looking for a new CArule such that running the original CA for 119879119905 time steps andthen grouping the supercells produces the same output asgrouping the initial array and then adopting the new CA rulefor 119905 time steps The new CA rule the time constant 119879 andthe size of the supercell 119873 are all unknown therefore thissearch requires a scan over all the possible combinations ofthese parameters Israeli and Goldenfeld successfully coarse-grained 240 of the 256 ECA rules many to other ECA rulesperforming a complete scan of all the possibilities compatiblewith 119873 le 4 Importantly the rule complexity was neverobserved to increase under coarse-graining introducing apartial ordering among CA rules

Complexity 7

Class I1

Class I2

Class I3 60 90 105 150 126

2 42170

6 7 11 14 24 27 5674 168 184

15

34

5

51

140

13

152136

156

0

25 26 37 41 5762 73 94 104154 164 204

128 44 130 132 162

1 4 12 1929 72 76

23 77 134 146160 178 232

200

204

3 8 9 18 26 28 3233 35 36 40 50 5878 108

172 10 38 46 138

22 30 45 110 122

43 54 106 142

Figure 5 Coarse-graining transitions within ECA found in [1] mapped onto a hierarchy of the information-based classes identified hereOnly the 88 rule equivalency classes are shown (represented by the lowest number rule) For simplicity (nontrivial) transitions to rule zeroare not shown (isolated boxed with continuous border) Boxes with a dashed border include ECA rules for which a coarse-graining transitionto another ECA rule with supercell of size 4 or less does not exist [1]

The same ordering emerges from our information-basedclassification as shown in Figure 5 where arrows indicatecoarse-graining transitions uncovered in [1] These transi-tions introduce a fully ordered hierarchy I3 rarr I2 rarr I1such that coarse-graining is never observed to move up thehierarchy and the vast majority of rules may only undergocoarse-graining transitions within the same information classThis ordering is sometimes expected like in the case of manyof our Class I2 rules where wave-like patterns are coarse-grained to either self-similar or very similar patterns Othertimes it looks more profound like in the case of WolframClass III rule 146 Rule 146 is in I1 and it can be nontriviallycoarse-grained to I1 rule 128 (a Wolfram Class I rule) dueto a shared sensitivity of the information processed in agiven space-time patch to its input state As noted in [1] thefact that potentially (a conclusive proof of the computationalirreducibility of ECA rule 146 is stillmissing) computationallyirreducible rules [8] like elementary rule 146 can be coarse-grained to predictable dynamics (eg rule 128) shows thatcomputational irreducibility might lack the characteristicsof a good measure of physical complexity We are led toconclude that the coarse-graining hierarchy is more likelydefined by conserved informational properties with morecomplex rules by Wolframrsquos classification appearing lower inthe hierarchy if they can be coarse-grained to less complexones with common sensitivity to the input state

5 Conclusions

Physical systems evolve in time according to local uniformrules but can nonetheless exhibit complex emergent behav-ior Complexity arises either as a result of the initial state orrule or some combination of the twoThe ambiguity of deter-mining whether complexity is generated by rules or states hasconfounded previous efforts to isolate complexity intrinsicto a given rule In this work we introduced a quantitativeinformation-based classification of ECA The classificationscheme proposed circumvents the difficulties arising due to

the sensitivity of the expressed complexity of ECA rules totheir initial state The (averaged) directed exchange of infor-mation (TE) between the individual parts of an ECA is usedas ameasure of its complexityThe identification of the single-cell input (Section 2) as the nontrivial state with the leastcomplexity is assumed as a working hypothesis and providesa reference point for our analysis of the degree of variabilityof the complexity of ECA rules for varying inputs Weidentified three distinct classes based on our analysis whichvary in their sensitivity to initial conditions Class I1 ECAalways process little information Class I3 always processeshigh information and Class I2 can be low or high dependingon the input It is only for Class I3 that the expressed com-plexity is intrinsic and not a product of the complexity carriedby the input

The most complex rules by our analysis are in Class I3which includes the majority of Wolframrsquos Class III rules andClass IV rule 110 and its equivalent rules These rules form aclosed group under the coarse-graining transitions found in[1] The truly complex rules are those that remain complexeven at a macrolevel of description with behavior that is notsensitive to the initial state

We are currently investigating the possibility of extendingour measure to less elementary dynamical systems as well asto idealized biological systems (eg networks of regulatorygenes) Inspired by Nursersquos idea [35] that the optimization ofinformation storage and flow might represent the distinctivefeature of biological functions we find interesting that infor-mation dynamics identifies the most complex ECA dynamicswith the ones that least depend on their initial state Thisfeature evocative of the redundancy so ubiquitous in biologymight represent a link between biology and informationdynamics [36] that we consider worth of further study

Disclosure

The opinions expressed in this publication are those of theauthors and do not necessarily reflect the views of TempletonWorld Charity Foundation

8 Complexity

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This publication was made possible through support ofa grant from Templeton World Charity Foundation Theauthors thank Douglas Moore and Hector Zenil for theirhelpful comments

References

[1] N Israeli and N Goldenfeld ldquoCoarse-graining of cellularautomata emergence and the predictability of complex sys-temsrdquo Physical Review E Statistical Nonlinear and Soft MatterPhysics vol 73 no 2 026203 17 pages 2006

[2] M Gardner ldquoMathematical gamesrdquo Scientific American vol223 pp 120ndash123 1970

[3] E R Berlekamp J H Conway and R K Guy Winning Waysfor Your Mathematical Plays Vol 2 Games in Particular vol 2Academic Press London UK 1982

[4] M GardnerWheels Life and other Mathematical AmusementsFreeman amp Company NY USA 1983

[5] I Smith ldquoSimple computation-universal cellular spacesrdquo Jour-nal of the Association for ComputingMachinery vol 18 pp 339ndash353 1971

[6] K Lindgren and M G Nordahl Complex Systems vol 4 pp299ndash318 1990

[7] M Cook ldquoUniversality in elementary cellular automatardquo Com-plex Systems vol 15 no 1 pp 1ndash40 2004

[8] S Wolfram ldquoCellular automata as models of complexityrdquoNature vol 311 no 5985 pp 419ndash424 1984

[9] J von Neumann Theory of Self-Reproducing Automata A WBurks University of Illinois Press Urbana ILL USA 1966

[10] K Culik and S Yu ldquoUndecidability of CA classificationschemesrdquo Complex Systems vol 2 pp 177ndash190 1988

[11] W Li and N Packard ldquoErratum ldquoThe structure of the elemen-tary cellular automata rule spacerdquordquo Complex Systems vol 4 no1 pp 281ndash297 1990

[12] C G Langton ldquoComputation at the edge of chaos phase transi-tions and emergent computationrdquo Physica D Nonlinear Phe-nomena vol 42 no 1ndash3 pp 12ndash37 1990

[13] F Bagnoli R Rechtman and S Ruffo ldquoDamage spreading andLyapunov exponents in cellular automatardquoPhysics Letters A vol172 no 1-2 pp 34ndash38 1992

[14] G Y Vichniac ldquoBoolean derivatives on cellular automatardquoPhysica D Nonlinear Phenomena vol 45 no 1-3 pp 63ndash741990

[15] J M Baetens and B De Baets ldquoPhenomenological study ofirregular cellular automata based on Lyapunov exponents andJacobiansrdquo Chaos An Interdisciplinary Journal of NonlinearScience vol 20 no 3 Article ID 033112 033112 15 pages 2010

[16] J M Baetens P Van Der Weeen and B De Baets ldquoEffect ofasynchronous updating on the stability of cellular automatardquoChaos Solitons and Fractals vol 45 no 4 pp 383ndash394 2012

[17] J M Baetens and B De Baets ldquoTopology-induced phase transi-tions in totalistic cellular automatardquo Physica D NonlinearPhenomena vol 249 pp 16ndash24 2013

[18] J M Baetens and J Gravner ldquoStability of cellular automatatrajectories revisited branching walks and Lyapunov profilesrdquoJournal of Nonlinear Science vol 26 no 5 pp 1329ndash1367 2016

[19] L O Chua S Yoon and R Dogaru ldquoA nonlinear dynamicsperspective of Wolframrsquos new kind of science I Threshold ofcomplexityrdquo International Journal of Bifurcation and Chaos inApplied Sciences and Engineering vol 12 no 12 pp 2655ndash27662002

[20] N Fates ldquoExperimental study of elementary cellular automatadynamics using the density parameterrdquo Discrete Mathematicsand Theoretical Computer Science pp 15ndash166 2003

[21] C Durr I Rapaport and G Theyssier Theoretical ComputerScience vol 322 pp 355ndash368 2004

[22] H Zenil and E Villarreal-Zapata ldquoAsymptotic behavior andratios of complexity in cellular automatardquo International Journalof Bifurcation and Chaos vol 23 no 9 Article ID 1350159 2013

[23] L Albantakis and G Tononi ldquoThe intrinsic cause-effect powerof discrete dynamical systems-from elementary cellular autom-ata to adapting animatsrdquo Entropy vol 17 no 8 pp 5472ndash55022015

[24] G J MartInez ldquoA note on elementary cellular automata classi-ficationrdquo Journal of Cellular Automata vol 8 no 3-4 pp 233ndash259 2013

[25] W Bialek Biophysics searching for principles Princeton Univer-sity Press 2012

[26] S I Walker H Kim and P C W Davies ldquoThe informationalarchitecture of the cellrdquo Philosophical Transactions of the RoyalSociety A vol 374 Article ID 20150057 2016

[27] T Schreiber ldquoMeasuring information transferrdquo Physical ReviewLetters vol 85 no 2 pp 461ndash464 2000

[28] Wolfram Alpha httpswwwwolframalphacom[29] J T LizierM Prokopenko andA Y Zomaya ldquoThe information

dynamics of phase transitions in random boolean networksrdquoin Proceedings of the 11th International Conference on theSimulation and Synthesis of Living Systems Artificial Life XIALIFE 2008 pp 374ndash381 2008

[30] J T Lizier and M Prokopenko The European Physical JournalB vol 73 2010

[31] X R Wang J M Miller J T Lizier M Prokopenko andL F Rossi ldquoMeasuring information storage and transfer inswarmsrdquo in Proceedings of the Eleventh European Conference onthe Synthesis and Simulation of Living Systems pp 838ndash8452011

[32] T Bossomaier L Barnett M Harre and J T Lizier AnIntroduction to Transfer Entropy Information Flow in ComplexSystems Springer 2016

[33] S Wolfram ldquoStatistical mechanics of cellular automatardquo Re-views of Modern Physics vol 55 no 3 pp 601ndash644 1983

[34] S Wolfram A New Kind of Science Wolfram Media Cham-paign ILL USA 2002

[35] P Nurse ldquoLife logic and informationrdquo Nature vol 454 no7203 pp 424ndash426 2008

[36] P C W Davies and S I Walker ldquoThe hidden simplicity ofbiologyrdquo Reports on Progress in Physics vol 79 no 10 ArticleID 102601 2016

Complexity 5

as quantitatively captured by ⟨TE⟩ in response to a morecomplex input

It is worth noting that all of the rules whose initial valueof ⟨TE⟩ lay above the upper limit for Classes I and II of⟨TE⟩ = 0024 bits in the simplest input scenario still havecalculated ⟨TE⟩ above this value under a change of input Inparticular the rules with the highest values of ⟨TE⟩ are notsignificantly affected by the change in the input Let us naivelyuse the upper limit for Classes I and II rules emerging fromFigure 2(a) as the border line between what we call low andhigh values of ⟨TE⟩ We can summarize the changes undervarying the input as follows There are rules whose value of⟨TE⟩ is low for both the inputs There are rules whose valueof ⟨TE⟩ is low in the simplest case but high in the morecomplex one And finally there are also rules with a highvalue of ⟨TE⟩ in both cases The interesting point is that wefind no rule whose value of ⟨119879119864⟩ is high for the simplest inputand low for the random input This is equivalent to say thatthere are no rules generating complexity for simple inputsbut annihilating it for complex ones The significance of thisobservation lies in the fact that it enables classification interms of the shift of ⟨TE⟩ over a space-time region in responseto a change in its input by taking advantage of the mainlimitation that makes quantitative classifications of ECAs sodifficult that is ECA sensitivity to their initial state andexploits it in order to achieve such classification

To confirm this is indeed a viable method for quantitativeclassificationwemust considermore than just two inputsWetherefore first randomly selected twenty different ones Beinginterested in how much ⟨TE⟩ can vary as we vary the inputfor each rule we selected the maximum absolute value of thepercent change of ⟨TE⟩ between the random inputs (⟨TE⟩119903)and the single-cell input (⟨TE⟩1) considered before

max1003816100381610038161003816⟨TE⟩119903 minus ⟨TE⟩11003816100381610038161003816⟨TE⟩119903 randominputs

(3)

The results are shown in Figure 3 where the maximum per-cent change is plotted as a function of ⟨TE⟩1 Equivalent rulesare represented by a single marker and share the same valuefor the maximum relative change of ⟨TE⟩ In fact given tworules 1199031 and 1199032 sharing a symmetry it is always possible to findtwo inputs 1198941 and 1198942 such that rule 1199031 initialized to 1198941 and rule1199032 initialized to 1198942 give rise to the same value of ⟨TE⟩

The horizontal dashed line separates rules whose maxi-mum ⟨TE⟩ change is less than one order of magnitude fromthose that can undergo a change in ⟨TE⟩ of at least a factor 10The vertical dashed line denotes the highest value of ⟨TE⟩1 forWolframClasses I and II exactly as in Figure 2(a)The regionto the right of the vertical line and above the horizontal line isvoid of anyECA rules Points in that regionwould correspondto values of ⟨TE⟩ that are both high for the simplest input andcapable of high variation for exampleCArules that can anni-hilate the complexity of the input which we do not observeThis feature yields a distinctive L-shape in the distribu-tion of ruleswith the rules in each region sharingwell-definedproperties

As a result a natural information-based classification ofECAs can be given as follows

Class I1 ⟨TE⟩ is very small for the simplest input andstays so for complex inputsThis is themost populatedclass including almost all Wolfram Classes I and IIrules rules 18 and 146 and their equivalent Class IIIrules

Class I2 ⟨TE⟩ is small for the simplest input but itexperiences a drastic change (one order of magnitudeor more) when the input is complex This is the casefor many Wolfram Class II and some Class IV rules(eg 54 and 106 and their equivalent rules)

Class I3 ⟨TE⟩ has a high value for the simplest inputand this value is approximately unaffected by a changein the input Most Wolfram Class III rules belong tothis class as well as Class IV rule 110 and its equivalentrules

Our classification is summarized in Table 2 Despite thearbitrary placement of our boundary lines at the border ofthe I1 region it shows an unambiguous separation betweenthe most majority of rules classified as I2 and I3

Randomly sampling inputs leads to a bias favoring nearlyfifty-fifty distributions of black and white cells due to theirbinomial distribution We therefore also verified this clas-sification using a different distribution of inputs where thenumber of black cells is increased in a regular way from 2to 50 while keeping the specific positions of black cells inthe input array random We considered 20 different inputseach containing exactly 2 5 7 10 12 47 and 50 blackcells (higher numbers are not considered due to the ◻ harr◼ conjugation) Apart from minor shifts of the data pointsapplying the same procedure as above yields exactly the sameclassification as Figure 3 indicating that our classificationscheme is robust

We stress the importance of considering a large system(order sim 100 cells) as opposed to a much smaller one (egordersim 10)While for the latter a scan over the entire space ofinputs is computationally feasible (and indeed we performedthese experiments) it hides one of the main features enablinginformation-based classification the existence of Class I3rules which form themost stable class with respect to our TEbased complexity measure Class I3 rules produce time serieslargely independent of the initial state exemplified by rule30 in Figure 3 a feature evident only in larger systems (beingthe result of combinatorial calculus TE computational timegrows exponentially with the number of cells considered AnECA array of about 100 cells is both the smallest size that wecould safely consider much larger than that of the domainsof Figure 3 and for which a complete scan over the rule spaceand a large enough ensemble of inputs could be made Wealso considered individual runs with 200 and 400 cells andnoticed that our numerical results are stable for CA with 100or more cells but are fluctuating for CA with 50 or less cells)

The typical order 10 cell length of the domains appearingin Figure 3 is also the reason why 119896 sim 5 is the most optimalvalue for the history length parameter Smaller and largervalues of 119896 would not be able to resolve this lattice sizeindependent feature of Class I3 rules [32] (Figure 4)

6 Complexity

Table 2 Information based classification of ECA with only the 88 rule equivalence classes shown (represented by the lowest number rule)Wolfram Class III rules are denoted in black boldface type and Class IV in italic

Class I10 1 3 4 5 8 9 12 13 1819 23 25 26 28 29 32 33 35 3637 40 41 44 50 51 57 58 62 7273 76 77 78 94 104 108 128 130 132134 136 140 146 152 154 156 160 162 164178 200 204 232

Class I22 6 7 10 11 14 15 24 27 3438 42 43 46 54 56 74 106 138 142168 170 172 184

Class I322 30 45 60 90 105 110 122 126 150

0

20

40

60

80

100

Max

imum

per

cent

chan

ge o

f ⟨TE

⟩

005 010 015 020000

⟨TE⟩ (single-cell input bits)

Class I1 Class I3

Class I2Random inputs

Figure 3 Maximum percent change of ⟨TE⟩ as a function of ⟨TE⟩1(3) Equivalent rules are represented by a singlemarker colored as inFigure 2 but with Wolfram Classes I and II in gray Rules above thehorizontal dashed line can undergo a change of at least one orderof magnitude in ⟨TE⟩ The vertical dashed line is the same as inFigure 2(a)

4 Coarse-Graining andthe Information Hierarchy

Perhaps the most interesting feature of our quantitativeclassification is that Wolfram Class III and Class IV rulesare distributed over different information-based classes Thisbehavior looks less surprising in the light of the coarse-graining transitions among ECA uncovered by Israeli andGoldenfeld [1] One important aspect of the physical worldis that coarse-grained descriptions often suffice to make

Rule 30single cell input

Rule 30random input

Figure 4 Time series for rule 30 generated by the two initial statesof Figure 2 The similarity of the two outputs each yielding colordomains with a typical linear size of about sim10 blocks does notdepend on the size of the CA and is responsible for the stability ofClass I3 rules to changes in input

predictions about relevant behavior Noting this Israeli andGoldenfeld adopted a systematic procedure to try to coarsegrain ECA rules Their prescription consists in groupingtogether nearby cells in theCAarray into a supercell of a givensize 119873 according to some specified supercell rarr single-cellcorrespondence (for Boolean CAs 22119873 possible applicationsof this kind exist) and a given time constant 119879 The searchfor a coarse-graining rule consists in looking for a new CArule such that running the original CA for 119879119905 time steps andthen grouping the supercells produces the same output asgrouping the initial array and then adopting the new CA rulefor 119905 time steps The new CA rule the time constant 119879 andthe size of the supercell 119873 are all unknown therefore thissearch requires a scan over all the possible combinations ofthese parameters Israeli and Goldenfeld successfully coarse-grained 240 of the 256 ECA rules many to other ECA rulesperforming a complete scan of all the possibilities compatiblewith 119873 le 4 Importantly the rule complexity was neverobserved to increase under coarse-graining introducing apartial ordering among CA rules

Complexity 7

Class I1

Class I2

Class I3 60 90 105 150 126

2 42170

6 7 11 14 24 27 5674 168 184

15

34

5

51

140

13

152136

156

0

25 26 37 41 5762 73 94 104154 164 204

128 44 130 132 162

1 4 12 1929 72 76

23 77 134 146160 178 232

200

204

3 8 9 18 26 28 3233 35 36 40 50 5878 108

172 10 38 46 138

22 30 45 110 122

43 54 106 142

Figure 5 Coarse-graining transitions within ECA found in [1] mapped onto a hierarchy of the information-based classes identified hereOnly the 88 rule equivalency classes are shown (represented by the lowest number rule) For simplicity (nontrivial) transitions to rule zeroare not shown (isolated boxed with continuous border) Boxes with a dashed border include ECA rules for which a coarse-graining transitionto another ECA rule with supercell of size 4 or less does not exist [1]

The same ordering emerges from our information-basedclassification as shown in Figure 5 where arrows indicatecoarse-graining transitions uncovered in [1] These transi-tions introduce a fully ordered hierarchy I3 rarr I2 rarr I1such that coarse-graining is never observed to move up thehierarchy and the vast majority of rules may only undergocoarse-graining transitions within the same information classThis ordering is sometimes expected like in the case of manyof our Class I2 rules where wave-like patterns are coarse-grained to either self-similar or very similar patterns Othertimes it looks more profound like in the case of WolframClass III rule 146 Rule 146 is in I1 and it can be nontriviallycoarse-grained to I1 rule 128 (a Wolfram Class I rule) dueto a shared sensitivity of the information processed in agiven space-time patch to its input state As noted in [1] thefact that potentially (a conclusive proof of the computationalirreducibility of ECA rule 146 is stillmissing) computationallyirreducible rules [8] like elementary rule 146 can be coarse-grained to predictable dynamics (eg rule 128) shows thatcomputational irreducibility might lack the characteristicsof a good measure of physical complexity We are led toconclude that the coarse-graining hierarchy is more likelydefined by conserved informational properties with morecomplex rules by Wolframrsquos classification appearing lower inthe hierarchy if they can be coarse-grained to less complexones with common sensitivity to the input state

5 Conclusions

Physical systems evolve in time according to local uniformrules but can nonetheless exhibit complex emergent behav-ior Complexity arises either as a result of the initial state orrule or some combination of the twoThe ambiguity of deter-mining whether complexity is generated by rules or states hasconfounded previous efforts to isolate complexity intrinsicto a given rule In this work we introduced a quantitativeinformation-based classification of ECA The classificationscheme proposed circumvents the difficulties arising due to

the sensitivity of the expressed complexity of ECA rules totheir initial state The (averaged) directed exchange of infor-mation (TE) between the individual parts of an ECA is usedas ameasure of its complexityThe identification of the single-cell input (Section 2) as the nontrivial state with the leastcomplexity is assumed as a working hypothesis and providesa reference point for our analysis of the degree of variabilityof the complexity of ECA rules for varying inputs Weidentified three distinct classes based on our analysis whichvary in their sensitivity to initial conditions Class I1 ECAalways process little information Class I3 always processeshigh information and Class I2 can be low or high dependingon the input It is only for Class I3 that the expressed com-plexity is intrinsic and not a product of the complexity carriedby the input

The most complex rules by our analysis are in Class I3which includes the majority of Wolframrsquos Class III rules andClass IV rule 110 and its equivalent rules These rules form aclosed group under the coarse-graining transitions found in[1] The truly complex rules are those that remain complexeven at a macrolevel of description with behavior that is notsensitive to the initial state

We are currently investigating the possibility of extendingour measure to less elementary dynamical systems as well asto idealized biological systems (eg networks of regulatorygenes) Inspired by Nursersquos idea [35] that the optimization ofinformation storage and flow might represent the distinctivefeature of biological functions we find interesting that infor-mation dynamics identifies the most complex ECA dynamicswith the ones that least depend on their initial state Thisfeature evocative of the redundancy so ubiquitous in biologymight represent a link between biology and informationdynamics [36] that we consider worth of further study

Disclosure

The opinions expressed in this publication are those of theauthors and do not necessarily reflect the views of TempletonWorld Charity Foundation

8 Complexity

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This publication was made possible through support ofa grant from Templeton World Charity Foundation Theauthors thank Douglas Moore and Hector Zenil for theirhelpful comments

References

[1] N Israeli and N Goldenfeld ldquoCoarse-graining of cellularautomata emergence and the predictability of complex sys-temsrdquo Physical Review E Statistical Nonlinear and Soft MatterPhysics vol 73 no 2 026203 17 pages 2006

[2] M Gardner ldquoMathematical gamesrdquo Scientific American vol223 pp 120ndash123 1970

[3] E R Berlekamp J H Conway and R K Guy Winning Waysfor Your Mathematical Plays Vol 2 Games in Particular vol 2Academic Press London UK 1982

[4] M GardnerWheels Life and other Mathematical AmusementsFreeman amp Company NY USA 1983

[5] I Smith ldquoSimple computation-universal cellular spacesrdquo Jour-nal of the Association for ComputingMachinery vol 18 pp 339ndash353 1971

[6] K Lindgren and M G Nordahl Complex Systems vol 4 pp299ndash318 1990

[7] M Cook ldquoUniversality in elementary cellular automatardquo Com-plex Systems vol 15 no 1 pp 1ndash40 2004

[8] S Wolfram ldquoCellular automata as models of complexityrdquoNature vol 311 no 5985 pp 419ndash424 1984

[9] J von Neumann Theory of Self-Reproducing Automata A WBurks University of Illinois Press Urbana ILL USA 1966

[10] K Culik and S Yu ldquoUndecidability of CA classificationschemesrdquo Complex Systems vol 2 pp 177ndash190 1988

[11] W Li and N Packard ldquoErratum ldquoThe structure of the elemen-tary cellular automata rule spacerdquordquo Complex Systems vol 4 no1 pp 281ndash297 1990

[12] C G Langton ldquoComputation at the edge of chaos phase transi-tions and emergent computationrdquo Physica D Nonlinear Phe-nomena vol 42 no 1ndash3 pp 12ndash37 1990

[13] F Bagnoli R Rechtman and S Ruffo ldquoDamage spreading andLyapunov exponents in cellular automatardquoPhysics Letters A vol172 no 1-2 pp 34ndash38 1992

[14] G Y Vichniac ldquoBoolean derivatives on cellular automatardquoPhysica D Nonlinear Phenomena vol 45 no 1-3 pp 63ndash741990

[15] J M Baetens and B De Baets ldquoPhenomenological study ofirregular cellular automata based on Lyapunov exponents andJacobiansrdquo Chaos An Interdisciplinary Journal of NonlinearScience vol 20 no 3 Article ID 033112 033112 15 pages 2010

[16] J M Baetens P Van Der Weeen and B De Baets ldquoEffect ofasynchronous updating on the stability of cellular automatardquoChaos Solitons and Fractals vol 45 no 4 pp 383ndash394 2012

[17] J M Baetens and B De Baets ldquoTopology-induced phase transi-tions in totalistic cellular automatardquo Physica D NonlinearPhenomena vol 249 pp 16ndash24 2013

[18] J M Baetens and J Gravner ldquoStability of cellular automatatrajectories revisited branching walks and Lyapunov profilesrdquoJournal of Nonlinear Science vol 26 no 5 pp 1329ndash1367 2016

[19] L O Chua S Yoon and R Dogaru ldquoA nonlinear dynamicsperspective of Wolframrsquos new kind of science I Threshold ofcomplexityrdquo International Journal of Bifurcation and Chaos inApplied Sciences and Engineering vol 12 no 12 pp 2655ndash27662002

[20] N Fates ldquoExperimental study of elementary cellular automatadynamics using the density parameterrdquo Discrete Mathematicsand Theoretical Computer Science pp 15ndash166 2003

[21] C Durr I Rapaport and G Theyssier Theoretical ComputerScience vol 322 pp 355ndash368 2004

[22] H Zenil and E Villarreal-Zapata ldquoAsymptotic behavior andratios of complexity in cellular automatardquo International Journalof Bifurcation and Chaos vol 23 no 9 Article ID 1350159 2013

[23] L Albantakis and G Tononi ldquoThe intrinsic cause-effect powerof discrete dynamical systems-from elementary cellular autom-ata to adapting animatsrdquo Entropy vol 17 no 8 pp 5472ndash55022015

[24] G J MartInez ldquoA note on elementary cellular automata classi-ficationrdquo Journal of Cellular Automata vol 8 no 3-4 pp 233ndash259 2013

[25] W Bialek Biophysics searching for principles Princeton Univer-sity Press 2012

[26] S I Walker H Kim and P C W Davies ldquoThe informationalarchitecture of the cellrdquo Philosophical Transactions of the RoyalSociety A vol 374 Article ID 20150057 2016

[27] T Schreiber ldquoMeasuring information transferrdquo Physical ReviewLetters vol 85 no 2 pp 461ndash464 2000

[28] Wolfram Alpha httpswwwwolframalphacom[29] J T LizierM Prokopenko andA Y Zomaya ldquoThe information

dynamics of phase transitions in random boolean networksrdquoin Proceedings of the 11th International Conference on theSimulation and Synthesis of Living Systems Artificial Life XIALIFE 2008 pp 374ndash381 2008

[30] J T Lizier and M Prokopenko The European Physical JournalB vol 73 2010

[31] X R Wang J M Miller J T Lizier M Prokopenko andL F Rossi ldquoMeasuring information storage and transfer inswarmsrdquo in Proceedings of the Eleventh European Conference onthe Synthesis and Simulation of Living Systems pp 838ndash8452011

[32] T Bossomaier L Barnett M Harre and J T Lizier AnIntroduction to Transfer Entropy Information Flow in ComplexSystems Springer 2016

[33] S Wolfram ldquoStatistical mechanics of cellular automatardquo Re-views of Modern Physics vol 55 no 3 pp 601ndash644 1983

[34] S Wolfram A New Kind of Science Wolfram Media Cham-paign ILL USA 2002

[35] P Nurse ldquoLife logic and informationrdquo Nature vol 454 no7203 pp 424ndash426 2008

[36] P C W Davies and S I Walker ldquoThe hidden simplicity ofbiologyrdquo Reports on Progress in Physics vol 79 no 10 ArticleID 102601 2016

6 Complexity

Table 2 Information based classification of ECA with only the 88 rule equivalence classes shown (represented by the lowest number rule)Wolfram Class III rules are denoted in black boldface type and Class IV in italic

Class I10 1 3 4 5 8 9 12 13 1819 23 25 26 28 29 32 33 35 3637 40 41 44 50 51 57 58 62 7273 76 77 78 94 104 108 128 130 132134 136 140 146 152 154 156 160 162 164178 200 204 232

Class I22 6 7 10 11 14 15 24 27 3438 42 43 46 54 56 74 106 138 142168 170 172 184

Class I322 30 45 60 90 105 110 122 126 150

0

20

40

60

80

100

Max

imum

per

cent

chan

ge o

f ⟨TE

⟩

005 010 015 020000

⟨TE⟩ (single-cell input bits)

Class I1 Class I3

Class I2Random inputs

Figure 3 Maximum percent change of ⟨TE⟩ as a function of ⟨TE⟩1(3) Equivalent rules are represented by a singlemarker colored as inFigure 2 but with Wolfram Classes I and II in gray Rules above thehorizontal dashed line can undergo a change of at least one orderof magnitude in ⟨TE⟩ The vertical dashed line is the same as inFigure 2(a)

4 Coarse-Graining andthe Information Hierarchy

Perhaps the most interesting feature of our quantitativeclassification is that Wolfram Class III and Class IV rulesare distributed over different information-based classes Thisbehavior looks less surprising in the light of the coarse-graining transitions among ECA uncovered by Israeli andGoldenfeld [1] One important aspect of the physical worldis that coarse-grained descriptions often suffice to make

Rule 30single cell input

Rule 30random input

Figure 4 Time series for rule 30 generated by the two initial statesof Figure 2 The similarity of the two outputs each yielding colordomains with a typical linear size of about sim10 blocks does notdepend on the size of the CA and is responsible for the stability ofClass I3 rules to changes in input

predictions about relevant behavior Noting this Israeli andGoldenfeld adopted a systematic procedure to try to coarsegrain ECA rules Their prescription consists in groupingtogether nearby cells in theCAarray into a supercell of a givensize 119873 according to some specified supercell rarr single-cellcorrespondence (for Boolean CAs 22119873 possible applicationsof this kind exist) and a given time constant 119879 The searchfor a coarse-graining rule consists in looking for a new CArule such that running the original CA for 119879119905 time steps andthen grouping the supercells produces the same output asgrouping the initial array and then adopting the new CA rulefor 119905 time steps The new CA rule the time constant 119879 andthe size of the supercell 119873 are all unknown therefore thissearch requires a scan over all the possible combinations ofthese parameters Israeli and Goldenfeld successfully coarse-grained 240 of the 256 ECA rules many to other ECA rulesperforming a complete scan of all the possibilities compatiblewith 119873 le 4 Importantly the rule complexity was neverobserved to increase under coarse-graining introducing apartial ordering among CA rules

Complexity 7

Class I1

Class I2

Class I3 60 90 105 150 126

2 42170

6 7 11 14 24 27 5674 168 184

15

34

5

51

140

13

152136

156

0

25 26 37 41 5762 73 94 104154 164 204

128 44 130 132 162

1 4 12 1929 72 76

23 77 134 146160 178 232

200

204

3 8 9 18 26 28 3233 35 36 40 50 5878 108

172 10 38 46 138

22 30 45 110 122

43 54 106 142

Figure 5 Coarse-graining transitions within ECA found in [1] mapped onto a hierarchy of the information-based classes identified hereOnly the 88 rule equivalency classes are shown (represented by the lowest number rule) For simplicity (nontrivial) transitions to rule zeroare not shown (isolated boxed with continuous border) Boxes with a dashed border include ECA rules for which a coarse-graining transitionto another ECA rule with supercell of size 4 or less does not exist [1]

The same ordering emerges from our information-basedclassification as shown in Figure 5 where arrows indicatecoarse-graining transitions uncovered in [1] These transi-tions introduce a fully ordered hierarchy I3 rarr I2 rarr I1such that coarse-graining is never observed to move up thehierarchy and the vast majority of rules may only undergocoarse-graining transitions within the same information classThis ordering is sometimes expected like in the case of manyof our Class I2 rules where wave-like patterns are coarse-grained to either self-similar or very similar patterns Othertimes it looks more profound like in the case of WolframClass III rule 146 Rule 146 is in I1 and it can be nontriviallycoarse-grained to I1 rule 128 (a Wolfram Class I rule) dueto a shared sensitivity of the information processed in agiven space-time patch to its input state As noted in [1] thefact that potentially (a conclusive proof of the computationalirreducibility of ECA rule 146 is stillmissing) computationallyirreducible rules [8] like elementary rule 146 can be coarse-grained to predictable dynamics (eg rule 128) shows thatcomputational irreducibility might lack the characteristicsof a good measure of physical complexity We are led toconclude that the coarse-graining hierarchy is more likelydefined by conserved informational properties with morecomplex rules by Wolframrsquos classification appearing lower inthe hierarchy if they can be coarse-grained to less complexones with common sensitivity to the input state

5 Conclusions

Physical systems evolve in time according to local uniformrules but can nonetheless exhibit complex emergent behav-ior Complexity arises either as a result of the initial state orrule or some combination of the twoThe ambiguity of deter-mining whether complexity is generated by rules or states hasconfounded previous efforts to isolate complexity intrinsicto a given rule In this work we introduced a quantitativeinformation-based classification of ECA The classificationscheme proposed circumvents the difficulties arising due to

the sensitivity of the expressed complexity of ECA rules totheir initial state The (averaged) directed exchange of infor-mation (TE) between the individual parts of an ECA is usedas ameasure of its complexityThe identification of the single-cell input (Section 2) as the nontrivial state with the leastcomplexity is assumed as a working hypothesis and providesa reference point for our analysis of the degree of variabilityof the complexity of ECA rules for varying inputs Weidentified three distinct classes based on our analysis whichvary in their sensitivity to initial conditions Class I1 ECAalways process little information Class I3 always processeshigh information and Class I2 can be low or high dependingon the input It is only for Class I3 that the expressed com-plexity is intrinsic and not a product of the complexity carriedby the input

The most complex rules by our analysis are in Class I3which includes the majority of Wolframrsquos Class III rules andClass IV rule 110 and its equivalent rules These rules form aclosed group under the coarse-graining transitions found in[1] The truly complex rules are those that remain complexeven at a macrolevel of description with behavior that is notsensitive to the initial state

We are currently investigating the possibility of extendingour measure to less elementary dynamical systems as well asto idealized biological systems (eg networks of regulatorygenes) Inspired by Nursersquos idea [35] that the optimization ofinformation storage and flow might represent the distinctivefeature of biological functions we find interesting that infor-mation dynamics identifies the most complex ECA dynamicswith the ones that least depend on their initial state Thisfeature evocative of the redundancy so ubiquitous in biologymight represent a link between biology and informationdynamics [36] that we consider worth of further study

Disclosure

The opinions expressed in this publication are those of theauthors and do not necessarily reflect the views of TempletonWorld Charity Foundation

8 Complexity

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This publication was made possible through support ofa grant from Templeton World Charity Foundation Theauthors thank Douglas Moore and Hector Zenil for theirhelpful comments

References

[1] N Israeli and N Goldenfeld ldquoCoarse-graining of cellularautomata emergence and the predictability of complex sys-temsrdquo Physical Review E Statistical Nonlinear and Soft MatterPhysics vol 73 no 2 026203 17 pages 2006

[2] M Gardner ldquoMathematical gamesrdquo Scientific American vol223 pp 120ndash123 1970

[3] E R Berlekamp J H Conway and R K Guy Winning Waysfor Your Mathematical Plays Vol 2 Games in Particular vol 2Academic Press London UK 1982

[4] M GardnerWheels Life and other Mathematical AmusementsFreeman amp Company NY USA 1983

[5] I Smith ldquoSimple computation-universal cellular spacesrdquo Jour-nal of the Association for ComputingMachinery vol 18 pp 339ndash353 1971

[6] K Lindgren and M G Nordahl Complex Systems vol 4 pp299ndash318 1990

[7] M Cook ldquoUniversality in elementary cellular automatardquo Com-plex Systems vol 15 no 1 pp 1ndash40 2004

[8] S Wolfram ldquoCellular automata as models of complexityrdquoNature vol 311 no 5985 pp 419ndash424 1984

[9] J von Neumann Theory of Self-Reproducing Automata A WBurks University of Illinois Press Urbana ILL USA 1966

[10] K Culik and S Yu ldquoUndecidability of CA classificationschemesrdquo Complex Systems vol 2 pp 177ndash190 1988

[11] W Li and N Packard ldquoErratum ldquoThe structure of the elemen-tary cellular automata rule spacerdquordquo Complex Systems vol 4 no1 pp 281ndash297 1990

[12] C G Langton ldquoComputation at the edge of chaos phase transi-tions and emergent computationrdquo Physica D Nonlinear Phe-nomena vol 42 no 1ndash3 pp 12ndash37 1990

[13] F Bagnoli R Rechtman and S Ruffo ldquoDamage spreading andLyapunov exponents in cellular automatardquoPhysics Letters A vol172 no 1-2 pp 34ndash38 1992

[14] G Y Vichniac ldquoBoolean derivatives on cellular automatardquoPhysica D Nonlinear Phenomena vol 45 no 1-3 pp 63ndash741990

[15] J M Baetens and B De Baets ldquoPhenomenological study ofirregular cellular automata based on Lyapunov exponents andJacobiansrdquo Chaos An Interdisciplinary Journal of NonlinearScience vol 20 no 3 Article ID 033112 033112 15 pages 2010

[16] J M Baetens P Van Der Weeen and B De Baets ldquoEffect ofasynchronous updating on the stability of cellular automatardquoChaos Solitons and Fractals vol 45 no 4 pp 383ndash394 2012

[17] J M Baetens and B De Baets ldquoTopology-induced phase transi-tions in totalistic cellular automatardquo Physica D NonlinearPhenomena vol 249 pp 16ndash24 2013

[18] J M Baetens and J Gravner ldquoStability of cellular automatatrajectories revisited branching walks and Lyapunov profilesrdquoJournal of Nonlinear Science vol 26 no 5 pp 1329ndash1367 2016

[19] L O Chua S Yoon and R Dogaru ldquoA nonlinear dynamicsperspective of Wolframrsquos new kind of science I Threshold ofcomplexityrdquo International Journal of Bifurcation and Chaos inApplied Sciences and Engineering vol 12 no 12 pp 2655ndash27662002

[20] N Fates ldquoExperimental study of elementary cellular automatadynamics using the density parameterrdquo Discrete Mathematicsand Theoretical Computer Science pp 15ndash166 2003

[21] C Durr I Rapaport and G Theyssier Theoretical ComputerScience vol 322 pp 355ndash368 2004

[22] H Zenil and E Villarreal-Zapata ldquoAsymptotic behavior andratios of complexity in cellular automatardquo International Journalof Bifurcation and Chaos vol 23 no 9 Article ID 1350159 2013

[23] L Albantakis and G Tononi ldquoThe intrinsic cause-effect powerof discrete dynamical systems-from elementary cellular autom-ata to adapting animatsrdquo Entropy vol 17 no 8 pp 5472ndash55022015

[24] G J MartInez ldquoA note on elementary cellular automata classi-ficationrdquo Journal of Cellular Automata vol 8 no 3-4 pp 233ndash259 2013

[25] W Bialek Biophysics searching for principles Princeton Univer-sity Press 2012

[26] S I Walker H Kim and P C W Davies ldquoThe informationalarchitecture of the cellrdquo Philosophical Transactions of the RoyalSociety A vol 374 Article ID 20150057 2016

[27] T Schreiber ldquoMeasuring information transferrdquo Physical ReviewLetters vol 85 no 2 pp 461ndash464 2000

[28] Wolfram Alpha httpswwwwolframalphacom[29] J T LizierM Prokopenko andA Y Zomaya ldquoThe information

dynamics of phase transitions in random boolean networksrdquoin Proceedings of the 11th International Conference on theSimulation and Synthesis of Living Systems Artificial Life XIALIFE 2008 pp 374ndash381 2008

[30] J T Lizier and M Prokopenko The European Physical JournalB vol 73 2010

[31] X R Wang J M Miller J T Lizier M Prokopenko andL F Rossi ldquoMeasuring information storage and transfer inswarmsrdquo in Proceedings of the Eleventh European Conference onthe Synthesis and Simulation of Living Systems pp 838ndash8452011

[32] T Bossomaier L Barnett M Harre and J T Lizier AnIntroduction to Transfer Entropy Information Flow in ComplexSystems Springer 2016

[33] S Wolfram ldquoStatistical mechanics of cellular automatardquo Re-views of Modern Physics vol 55 no 3 pp 601ndash644 1983

[34] S Wolfram A New Kind of Science Wolfram Media Cham-paign ILL USA 2002

[35] P Nurse ldquoLife logic and informationrdquo Nature vol 454 no7203 pp 424ndash426 2008

[36] P C W Davies and S I Walker ldquoThe hidden simplicity ofbiologyrdquo Reports on Progress in Physics vol 79 no 10 ArticleID 102601 2016

Complexity 7

Class I1

Class I2

Class I3 60 90 105 150 126

2 42170

6 7 11 14 24 27 5674 168 184

15

34

5

51

140

13

152136

156

0

25 26 37 41 5762 73 94 104154 164 204

128 44 130 132 162

1 4 12 1929 72 76

23 77 134 146160 178 232

200

204

3 8 9 18 26 28 3233 35 36 40 50 5878 108

172 10 38 46 138

22 30 45 110 122

43 54 106 142

Figure 5 Coarse-graining transitions within ECA found in [1] mapped onto a hierarchy of the information-based classes identified hereOnly the 88 rule equivalency classes are shown (represented by the lowest number rule) For simplicity (nontrivial) transitions to rule zeroare not shown (isolated boxed with continuous border) Boxes with a dashed border include ECA rules for which a coarse-graining transitionto another ECA rule with supercell of size 4 or less does not exist [1]

The same ordering emerges from our information-basedclassification as shown in Figure 5 where arrows indicatecoarse-graining transitions uncovered in [1] These transi-tions introduce a fully ordered hierarchy I3 rarr I2 rarr I1such that coarse-graining is never observed to move up thehierarchy and the vast majority of rules may only undergocoarse-graining transitions within the same information classThis ordering is sometimes expected like in the case of manyof our Class I2 rules where wave-like patterns are coarse-grained to either self-similar or very similar patterns Othertimes it looks more profound like in the case of WolframClass III rule 146 Rule 146 is in I1 and it can be nontriviallycoarse-grained to I1 rule 128 (a Wolfram Class I rule) dueto a shared sensitivity of the information processed in agiven space-time patch to its input state As noted in [1] thefact that potentially (a conclusive proof of the computationalirreducibility of ECA rule 146 is stillmissing) computationallyirreducible rules [8] like elementary rule 146 can be coarse-grained to predictable dynamics (eg rule 128) shows thatcomputational irreducibility might lack the characteristicsof a good measure of physical complexity We are led toconclude that the coarse-graining hierarchy is more likelydefined by conserved informational properties with morecomplex rules by Wolframrsquos classification appearing lower inthe hierarchy if they can be coarse-grained to less complexones with common sensitivity to the input state

5 Conclusions

Physical systems evolve in time according to local uniformrules but can nonetheless exhibit complex emergent behav-ior Complexity arises either as a result of the initial state orrule or some combination of the twoThe ambiguity of deter-mining whether complexity is generated by rules or states hasconfounded previous efforts to isolate complexity intrinsicto a given rule In this work we introduced a quantitativeinformation-based classification of ECA The classificationscheme proposed circumvents the difficulties arising due to

the sensitivity of the expressed complexity of ECA rules totheir initial state The (averaged) directed exchange of infor-mation (TE) between the individual parts of an ECA is usedas ameasure of its complexityThe identification of the single-cell input (Section 2) as the nontrivial state with the leastcomplexity is assumed as a working hypothesis and providesa reference point for our analysis of the degree of variabilityof the complexity of ECA rules for varying inputs Weidentified three distinct classes based on our analysis whichvary in their sensitivity to initial conditions Class I1 ECAalways process little information Class I3 always processeshigh information and Class I2 can be low or high dependingon the input It is only for Class I3 that the expressed com-plexity is intrinsic and not a product of the complexity carriedby the input

The most complex rules by our analysis are in Class I3which includes the majority of Wolframrsquos Class III rules andClass IV rule 110 and its equivalent rules These rules form aclosed group under the coarse-graining transitions found in[1] The truly complex rules are those that remain complexeven at a macrolevel of description with behavior that is notsensitive to the initial state

We are currently investigating the possibility of extendingour measure to less elementary dynamical systems as well asto idealized biological systems (eg networks of regulatorygenes) Inspired by Nursersquos idea [35] that the optimization ofinformation storage and flow might represent the distinctivefeature of biological functions we find interesting that infor-mation dynamics identifies the most complex ECA dynamicswith the ones that least depend on their initial state Thisfeature evocative of the redundancy so ubiquitous in biologymight represent a link between biology and informationdynamics [36] that we consider worth of further study

Disclosure

The opinions expressed in this publication are those of theauthors and do not necessarily reflect the views of TempletonWorld Charity Foundation

8 Complexity

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This publication was made possible through support ofa grant from Templeton World Charity Foundation Theauthors thank Douglas Moore and Hector Zenil for theirhelpful comments

References

[1] N Israeli and N Goldenfeld ldquoCoarse-graining of cellularautomata emergence and the predictability of complex sys-temsrdquo Physical Review E Statistical Nonlinear and Soft MatterPhysics vol 73 no 2 026203 17 pages 2006

[2] M Gardner ldquoMathematical gamesrdquo Scientific American vol223 pp 120ndash123 1970

[3] E R Berlekamp J H Conway and R K Guy Winning Waysfor Your Mathematical Plays Vol 2 Games in Particular vol 2Academic Press London UK 1982

[4] M GardnerWheels Life and other Mathematical AmusementsFreeman amp Company NY USA 1983

[5] I Smith ldquoSimple computation-universal cellular spacesrdquo Jour-nal of the Association for ComputingMachinery vol 18 pp 339ndash353 1971

[6] K Lindgren and M G Nordahl Complex Systems vol 4 pp299ndash318 1990

[7] M Cook ldquoUniversality in elementary cellular automatardquo Com-plex Systems vol 15 no 1 pp 1ndash40 2004

[8] S Wolfram ldquoCellular automata as models of complexityrdquoNature vol 311 no 5985 pp 419ndash424 1984

[9] J von Neumann Theory of Self-Reproducing Automata A WBurks University of Illinois Press Urbana ILL USA 1966

[10] K Culik and S Yu ldquoUndecidability of CA classificationschemesrdquo Complex Systems vol 2 pp 177ndash190 1988

[11] W Li and N Packard ldquoErratum ldquoThe structure of the elemen-tary cellular automata rule spacerdquordquo Complex Systems vol 4 no1 pp 281ndash297 1990

[12] C G Langton ldquoComputation at the edge of chaos phase transi-tions and emergent computationrdquo Physica D Nonlinear Phe-nomena vol 42 no 1ndash3 pp 12ndash37 1990

[13] F Bagnoli R Rechtman and S Ruffo ldquoDamage spreading andLyapunov exponents in cellular automatardquoPhysics Letters A vol172 no 1-2 pp 34ndash38 1992

[14] G Y Vichniac ldquoBoolean derivatives on cellular automatardquoPhysica D Nonlinear Phenomena vol 45 no 1-3 pp 63ndash741990

[15] J M Baetens and B De Baets ldquoPhenomenological study ofirregular cellular automata based on Lyapunov exponents andJacobiansrdquo Chaos An Interdisciplinary Journal of NonlinearScience vol 20 no 3 Article ID 033112 033112 15 pages 2010

[16] J M Baetens P Van Der Weeen and B De Baets ldquoEffect ofasynchronous updating on the stability of cellular automatardquoChaos Solitons and Fractals vol 45 no 4 pp 383ndash394 2012

[17] J M Baetens and B De Baets ldquoTopology-induced phase transi-tions in totalistic cellular automatardquo Physica D NonlinearPhenomena vol 249 pp 16ndash24 2013

[18] J M Baetens and J Gravner ldquoStability of cellular automatatrajectories revisited branching walks and Lyapunov profilesrdquoJournal of Nonlinear Science vol 26 no 5 pp 1329ndash1367 2016

[19] L O Chua S Yoon and R Dogaru ldquoA nonlinear dynamicsperspective of Wolframrsquos new kind of science I Threshold ofcomplexityrdquo International Journal of Bifurcation and Chaos inApplied Sciences and Engineering vol 12 no 12 pp 2655ndash27662002

[20] N Fates ldquoExperimental study of elementary cellular automatadynamics using the density parameterrdquo Discrete Mathematicsand Theoretical Computer Science pp 15ndash166 2003

[21] C Durr I Rapaport and G Theyssier Theoretical ComputerScience vol 322 pp 355ndash368 2004

[22] H Zenil and E Villarreal-Zapata ldquoAsymptotic behavior andratios of complexity in cellular automatardquo International Journalof Bifurcation and Chaos vol 23 no 9 Article ID 1350159 2013

[23] L Albantakis and G Tononi ldquoThe intrinsic cause-effect powerof discrete dynamical systems-from elementary cellular autom-ata to adapting animatsrdquo Entropy vol 17 no 8 pp 5472ndash55022015

[24] G J MartInez ldquoA note on elementary cellular automata classi-ficationrdquo Journal of Cellular Automata vol 8 no 3-4 pp 233ndash259 2013

[25] W Bialek Biophysics searching for principles Princeton Univer-sity Press 2012

[26] S I Walker H Kim and P C W Davies ldquoThe informationalarchitecture of the cellrdquo Philosophical Transactions of the RoyalSociety A vol 374 Article ID 20150057 2016

[27] T Schreiber ldquoMeasuring information transferrdquo Physical ReviewLetters vol 85 no 2 pp 461ndash464 2000

[28] Wolfram Alpha httpswwwwolframalphacom[29] J T LizierM Prokopenko andA Y Zomaya ldquoThe information

dynamics of phase transitions in random boolean networksrdquoin Proceedings of the 11th International Conference on theSimulation and Synthesis of Living Systems Artificial Life XIALIFE 2008 pp 374ndash381 2008

[30] J T Lizier and M Prokopenko The European Physical JournalB vol 73 2010

[31] X R Wang J M Miller J T Lizier M Prokopenko andL F Rossi ldquoMeasuring information storage and transfer inswarmsrdquo in Proceedings of the Eleventh European Conference onthe Synthesis and Simulation of Living Systems pp 838ndash8452011

[32] T Bossomaier L Barnett M Harre and J T Lizier AnIntroduction to Transfer Entropy Information Flow in ComplexSystems Springer 2016

[33] S Wolfram ldquoStatistical mechanics of cellular automatardquo Re-views of Modern Physics vol 55 no 3 pp 601ndash644 1983

[34] S Wolfram A New Kind of Science Wolfram Media Cham-paign ILL USA 2002

[35] P Nurse ldquoLife logic and informationrdquo Nature vol 454 no7203 pp 424ndash426 2008

[36] P C W Davies and S I Walker ldquoThe hidden simplicity ofbiologyrdquo Reports on Progress in Physics vol 79 no 10 ArticleID 102601 2016

8 Complexity

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This publication was made possible through support ofa grant from Templeton World Charity Foundation Theauthors thank Douglas Moore and Hector Zenil for theirhelpful comments

References

[1] N Israeli and N Goldenfeld ldquoCoarse-graining of cellularautomata emergence and the predictability of complex sys-temsrdquo Physical Review E Statistical Nonlinear and Soft MatterPhysics vol 73 no 2 026203 17 pages 2006

[2] M Gardner ldquoMathematical gamesrdquo Scientific American vol223 pp 120ndash123 1970

[3] E R Berlekamp J H Conway and R K Guy Winning Waysfor Your Mathematical Plays Vol 2 Games in Particular vol 2Academic Press London UK 1982

[4] M GardnerWheels Life and other Mathematical AmusementsFreeman amp Company NY USA 1983

[5] I Smith ldquoSimple computation-universal cellular spacesrdquo Jour-nal of the Association for ComputingMachinery vol 18 pp 339ndash353 1971

[6] K Lindgren and M G Nordahl Complex Systems vol 4 pp299ndash318 1990

[7] M Cook ldquoUniversality in elementary cellular automatardquo Com-plex Systems vol 15 no 1 pp 1ndash40 2004

[8] S Wolfram ldquoCellular automata as models of complexityrdquoNature vol 311 no 5985 pp 419ndash424 1984

[9] J von Neumann Theory of Self-Reproducing Automata A WBurks University of Illinois Press Urbana ILL USA 1966

[10] K Culik and S Yu ldquoUndecidability of CA classificationschemesrdquo Complex Systems vol 2 pp 177ndash190 1988

[11] W Li and N Packard ldquoErratum ldquoThe structure of the elemen-tary cellular automata rule spacerdquordquo Complex Systems vol 4 no1 pp 281ndash297 1990

[12] C G Langton ldquoComputation at the edge of chaos phase transi-tions and emergent computationrdquo Physica D Nonlinear Phe-nomena vol 42 no 1ndash3 pp 12ndash37 1990

[13] F Bagnoli R Rechtman and S Ruffo ldquoDamage spreading andLyapunov exponents in cellular automatardquoPhysics Letters A vol172 no 1-2 pp 34ndash38 1992

[14] G Y Vichniac ldquoBoolean derivatives on cellular automatardquoPhysica D Nonlinear Phenomena vol 45 no 1-3 pp 63ndash741990

[15] J M Baetens and B De Baets ldquoPhenomenological study ofirregular cellular automata based on Lyapunov exponents andJacobiansrdquo Chaos An Interdisciplinary Journal of NonlinearScience vol 20 no 3 Article ID 033112 033112 15 pages 2010

[16] J M Baetens P Van Der Weeen and B De Baets ldquoEffect ofasynchronous updating on the stability of cellular automatardquoChaos Solitons and Fractals vol 45 no 4 pp 383ndash394 2012

[17] J M Baetens and B De Baets ldquoTopology-induced phase transi-tions in totalistic cellular automatardquo Physica D NonlinearPhenomena vol 249 pp 16ndash24 2013

[18] J M Baetens and J Gravner ldquoStability of cellular automatatrajectories revisited branching walks and Lyapunov profilesrdquoJournal of Nonlinear Science vol 26 no 5 pp 1329ndash1367 2016

[19] L O Chua S Yoon and R Dogaru ldquoA nonlinear dynamicsperspective of Wolframrsquos new kind of science I Threshold ofcomplexityrdquo International Journal of Bifurcation and Chaos inApplied Sciences and Engineering vol 12 no 12 pp 2655ndash27662002

[20] N Fates ldquoExperimental study of elementary cellular automatadynamics using the density parameterrdquo Discrete Mathematicsand Theoretical Computer Science pp 15ndash166 2003

[21] C Durr I Rapaport and G Theyssier Theoretical ComputerScience vol 322 pp 355ndash368 2004

[22] H Zenil and E Villarreal-Zapata ldquoAsymptotic behavior andratios of complexity in cellular automatardquo International Journalof Bifurcation and Chaos vol 23 no 9 Article ID 1350159 2013

[23] L Albantakis and G Tononi ldquoThe intrinsic cause-effect powerof discrete dynamical systems-from elementary cellular autom-ata to adapting animatsrdquo Entropy vol 17 no 8 pp 5472ndash55022015

[24] G J MartInez ldquoA note on elementary cellular automata classi-ficationrdquo Journal of Cellular Automata vol 8 no 3-4 pp 233ndash259 2013

[25] W Bialek Biophysics searching for principles Princeton Univer-sity Press 2012

[26] S I Walker H Kim and P C W Davies ldquoThe informationalarchitecture of the cellrdquo Philosophical Transactions of the RoyalSociety A vol 374 Article ID 20150057 2016

[27] T Schreiber ldquoMeasuring information transferrdquo Physical ReviewLetters vol 85 no 2 pp 461ndash464 2000

[28] Wolfram Alpha httpswwwwolframalphacom[29] J T LizierM Prokopenko andA Y Zomaya ldquoThe information

dynamics of phase transitions in random boolean networksrdquoin Proceedings of the 11th International Conference on theSimulation and Synthesis of Living Systems Artificial Life XIALIFE 2008 pp 374ndash381 2008

[30] J T Lizier and M Prokopenko The European Physical JournalB vol 73 2010

[31] X R Wang J M Miller J T Lizier M Prokopenko andL F Rossi ldquoMeasuring information storage and transfer inswarmsrdquo in Proceedings of the Eleventh European Conference onthe Synthesis and Simulation of Living Systems pp 838ndash8452011

[32] T Bossomaier L Barnett M Harre and J T Lizier AnIntroduction to Transfer Entropy Information Flow in ComplexSystems Springer 2016

[33] S Wolfram ldquoStatistical mechanics of cellular automatardquo Re-views of Modern Physics vol 55 no 3 pp 601ndash644 1983

[34] S Wolfram A New Kind of Science Wolfram Media Cham-paign ILL USA 2002

[35] P Nurse ldquoLife logic and informationrdquo Nature vol 454 no7203 pp 424ndash426 2008

[36] P C W Davies and S I Walker ldquoThe hidden simplicity ofbiologyrdquo Reports on Progress in Physics vol 79 no 10 ArticleID 102601 2016

Submit your manuscripts athttpswwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of