Computational Emergence in Cellular Computing Systemsatm.neuro.pub.ro/radu_d/html/bn2007_2008/emergence_reason_2.pdfsystems is evolutionary, the emergence of new functions being sometimes

Copyright 2003, Radu Dogaru Polytechnic Univ. of Bucharest - Romania

Computational Emergence in Cellular Computing Systems

Radu Dogaru

Polytechnic University of Bucharest Department of Applied Electronics and

Information Engineering

1

Corrupted image fed into the GCA

GCA output after 8 iterations GCA output after thresholding

Unperturbed reference image

GC

A Processing

Thresholdingyij=sign(yij-0.12)

Additivenoise withuniform distribution

1


1. OVERVIEW

2

Copyright 2003, Radu Dogaru Polytechnic Univ. of Bucharest - Romania3

Motivation

The Cellular Neural Network (CNN), introduced by Chua and Yang from University ofCalifornia at Berkeley in the late 1980's is an attractive computational structure, particularlyform the implementation perspective in various micro and nanoelectronic technologies.

Computation in CNNs is brain-like rather than "classic" in the sense of the widespreadcomputing architectures based on microprocessors. Emergent computation, simply defined asthe class of both dynamic and static patterns of activity emerging in an array of interconnectedcells which are meaningful for various information processing tasks, is the equivalent of thesequences of instructions associated with classic computers.

It is of a paramount importance to find the equivalent of the"programming rules" for such cellular devices.

Massive parallel computing system, Regular structure and thus well suited for VLSI and nanotechnologies


Cellular vs. Classic Computing Systems

Assume that we have an array of memory cells (i.e. a Random Access Memory).- In a classic computer the cells are sequentially updated and located by the central processingunit via some external buses while in a cellular computer each memory cell exchangesinformation locally only with the neighbouring cells.- In a cellular computer all cells are updated in parallel and there is no central processing unit tocontrol the cells. Such as in a classic computer, the array of cells starts from an initial state,which contains the problem, and the solution will be found in the same array of cells after aperiod of time during which computation emerges.

While the designer of a classic computer focuses on the central processing unit, on data andaddress buses and instruction sets, the designer of a cellular computer has to focus only ondesigning the cell by itself. The "program" is now coded in what Chua called "cells' gene" (i.e.the entire set of parameters defining the cell)

4


Specific Design Problems

- What is the capability of cell to perform arbitrary localcomputations, i.e. the universality of a cell?

What is the choice of the cell parameters such thatemergent computation will occur?More sharply, one would like to find the exact values ofthe parameters for a given information processing task.


Why Emergent Computation in Cellular Systems ?

6

(i) The type of computation taking place in a cellular computer is the one used by living entities.Life itself is an emergent phenomenon and several examples will show that simple living-likeentities may emerge as a pattern of cellular activities. Brain-like computation could be bettermimicked by a compact and small CNN, rather than by a classic computer. Particularly when suchcomputations are required in micro-robotics, or in any circumstance requiring compact yetintelligent sensor systems, the CNN could be a better choice;

(ii) The cellular systems are highly parallel and consequently they perform several orders ofmagnitudes faster than classic (serial) computing ones. Tera-ops processing speed (1012

elementary operation per second) is common for the actual generation of CNNs;

(iii) Recent developments in the area of nanotechnology indicate that cellular structures made oflattices of interconnected active devices (for example, resonant tunneling diodes, quantum dotsor single electron transistors) could be easily developed. Characterized by a very high density ofcells, they can fully exploit the benefits of emergent computation for various tasks in informationprocessing.


Applications

Intelligent sensors and other compact computing systems (low power, easy tointegrate various information processing functions with sensing cells, greatpotential for pervasive computing and nanotechnologies)

Ciphering – emergence of unstable (chaotic) patterns in a cellular system ensures a huge statespace, so that the generating sequence can be considered random for all practical purposes.

Image processing (filtering, etc.) – emergence of stable patterns in a cellular system ensures –for instance edge detection, halftoning, hole filing, erosion, dilation, etc.

Signal compression – e.g. sound compression (see details next) or image compression usingCA transforms (cellular automata transforms)

Biometry – (details in this course) based on perceptual resonance of emergent patterns withhumans.

Biological systems modelling – e.g. the neural system driving the locomotion of a simpleartificial insect.


Example – Sound Compression R. Dogaru, T.A. Murgan, M. Glesner,”A compact solution for voice compression based on cellular neural networks”, in ProceedingsNSIP 2003 (IEEE-Eurasip Workshop on Nonlinear Signal and Image Processing (CD Proceedings)

encoder Initial state (continuous) ore.g. 16 bits per sample

Final state (binary) or 1 bit per sample

decoder

channel

Huffman encoder


Example – Sound Compression

9

Developmentplatform andtechnology is welldeveloped(See the CNNAWorkshops – oneat two years)

[ ]Ni xxx ,..,..1=x

A sequence of samples is an initial state for a recurrentcellular neural network with self feedback (a parameter )and lateral inhibition (b parameter)

)( 11

111

−+

−−− −+−= t

it

it

it

i byaybyfy

( )11)( 21 −−+== xxxfy

For 256 cells about 20 iterationssuffice to obtain a binary(compressed) signal

The gene parameters (a,b) have to beoptimized for the quality of thereconstructed sound !E.g. (a,b)=(1.8, 0.3)

A simple mixed-signalcircuit can be built toimplement the cell


Emergent computation as a universal phenomena

10

The process of developing computing machines is a universal and a natural one, which obeyscertain life laws according to which life itself is sustained by a continuous informationexchange with the external world. It appears that while life itself is the result of a continuousflow of information exchange (or computation in a wider sense), hierarchies of other (superior)“living” entities develop as a result of information exchange between similar entities. So,simple cells collaborate to produce and maintain functional organs, which then are linked in anetwork forming individuals. Such individuals then form families, villages, countries,federations, etc.

At a given level of the hierarchy, the actors (let us call them cells) are quite similar and they dousually exchange information only within a very small fraction of the entire population. Thisparticular fraction constitutes a neighborhood, which is essentially of informational naturealthough some topological constraints may have an influence on which members belong to aneighborhood (for example, a group of scientists usually collaborate through e-mail althoughthey are located geographically at very distances). The same structure is reflected in ourbrains. Our brain operates as an emergent computer where a network of concepts andrelationships are stored.


In digital systems: Novel functions (e.g. RS flip-flops, counters, automata) emerge whenproperly connecting simple identical parts called logical gates. Design for emergence in digitalsystems is evolutionary, the emergence of new functions being sometimes a matter of educatedguess or inspiration.

In biology: Novel structures and functions (e.g. tissues, bodies etc.) emerge from properlydecoding DNA strings. Life itself is an emergent phenomena .

In electronics: Various information processing functions (e.g. amplification, demodulation,oscillators etc.) emerge from properly interconnected cells (transistors, resistors, capacitors,inductors).

In societies: Competition and cooperation emerge in societies of humans or other living entities

Examples of Emergence

11


How is Emergence Defined?

Dictionary definition: to emerge, is defined as “to come out from inside or from being hidden”.

David Chalmers (1990, comp.ai.philosophy): “Emergence is the phenomenon wherein complex,interesting high-level function is produced as a result of combining simple low-level mechanismsin simple ways”, ... “emergence is a psychological property”.

“Emergence is the phenomenon wherein a system is designedaccording to certain principles, but interesting properties arise that are not included in the goals

of the designer“.

Ronald (1999): emergence test based on a surprise effect and somehow similar to the Turing testfor detecting intelligence. If using a (macro) language L2 to describe the global behavior of a system,designed according to a (micro) language L1 (usually providing information about the cell structure andits local interconnectivity) the behaviors observed in L2 are non-obvious to the observer – whotherefore experiences surprise – one may conclude that the global behavior is an emergent one.


Standish (2001) develops the ideas of Ronald observing that the surpriseeffect in the above test disappears after the first observation of anemergent behavior. Then he proposes the following definition ofemergence:

“An emergent phenomenon is simply one that is described by atomicconcepts available in the macro language, but cannot be so described inthe micro language”.

How is Emergence Defined?

Due to their regularity and homogeneity, cellular computing systems arebest suited for developing a systematic theory and methods for “design

for emergence”


The structure of a cellular computer iscomposed of a grid (often two-dimensional) of cells locallyinterconnected with their neighbors.Each cell can be in a number of states(ranging from 2 to infinity) and thestate of a cell depends by itself andby the states of its neighbors througha nonlinear functional, which can bedefined in different ways. Thisfunctional is associated with apractical implementation of the celland it includes a set of tunableparameters.

Cellular computing systems and emergence


In a cellular computer, computation can be considered any form of meaningful emergent globalphenomenon resulted from a proper design of the cell functional (gene).

Usually the initial state and the inputs code the problem to be solved while the answer to thisproblem is coded in an equilibrium steady state but it can be also coded in the emergentdynamics of the cellular system.

By meaningful I mean a subjective definition according to which one can effectively use theresults of the emergent phenomena for a purpose.For instance, loading an input image in a cellular computer is corresponds to setting the initialstate of each cell proportional to the brightness of its corresponding a pixel in the image. Thenusing a properly designed cell, the cellular computer will dynamically evolve towards a globalstate representing a transformation of the initial image (e.g. an image containing only edges).Or, a set of different waves could emerge in a cellular computer, each being formally assigned topa computational symbol. The dynamics of the cellular system will lead to collisions betweenwaves, which, when meaningful can be interpreted as computing.

Cellular computing systems and emergence


1940‘s: Stanislas Ulam suggested John von Neumann to use what he called "cellular spaces" tobuild his self-reproductive machine.Later cellular models, often called Cellular Automata (CA) were developed to explain various naturalphenomena. Choosing the proper genes in the form of local rules defining the behavior of each cell wasthe equivalent of programming in serial computers.

Conway (in the 70s) introduced the “Game of Life” rule. Many complex and diverse patterns emerge insuch a simply defined system. We should note that designing a proper set of local rules was then amatter of intuition and educated guess rather than the outcome of a well-defined procedure. It was provedthat such a simple machine (this is a 2 state per cell cellular automata with a very simple local rule) iscapable of universal computation (i.e. it is a universal Turing machine).

Last decade: A lot of research devoted to the study of cellular automata and local rules leading toemergent properties such as self-reproduction and artificial life. An overview of these non-conventionalcomputers can be found in [Toffoli, 1998]

History - Cellular Automata


Emergent behaviors in the simplest Cellular Automata

One dimensional CA Rule 30 – Random number generator

Rule 110 – Universal computing machine*

17

*According to Wolfram‘s book „A New Kind of Science“ (2002)

Rule 196 – Steady state pattern (nonlinear filter)


Birth of CNN: Chua and Yang [Chua & Yang, 1988] presented a novel cellular computing paradigmcalled Cellular Neural Network. It inherits the basic ideas of cellular computing and in addition bore someinteresting ideas from the field of neural computation.

History - Cellular Neural Networks

CNN was from the beginning circuit oriented i.e. intended for practical applications as an mixed-signal(analog and digital) integrated circuit

Roska and Chua [Roska & Chua, 1993] proposed a revolutionary framework called a CNN UniversalMachine, in fact a specialized programmable cellular computer which is capable to execute compleximage processing tasks and which found numerous applications in vision, robotics and remote sensing[Chua & Roska, 2001]. Its hardware implementations are called today visual microprocessors.

The CNN paradigm include Cellular Automata as a special case ([Chua, 1998] ). Therefore many of theresearch in the area of CA can be easily mapped into the CNN formalism with the advantage ofexploiting a range of powerful chip implementations that have been developed over the years [Roska &Rodriguez-Vazquez, 2000].


To date several types of emergent computation were identified as meaningful and useful either forcomputing applications (e.g. in the area of vision and image processing) or for modeling purposed(e.g. models of the cell membrane) by what I could call generically evolutionary strategies.An interesting example is the development of a relatively large library of CNN genes (templates) overthe last decade [AnalogicLAB, 2002].Many of these genes were discovered by chance, studying the dynamic evolution of the CNN andidentifying certain dynamic behaviors with meaningful computational primitives such as edge or cornerdetection, hole filling, motion detection, and so on.Although some theoretical approaches, mainly inspired from the techniques of filter design weresuccessfully employed to design new chromosomes, there is still much to do for a systematicdesign (design for emergence) of the cells and genes.

This course offers several novel solutions to this problem starting from the recent theory oflocal activity [Chua, 1998] and by employing mutations of genes for cells such as „Game ofLife“.

Emergent Computation in Cellular Neural Networks


Designing for emergence

An emergent phenomenon is one that cannot be described in terms of a micro language (e.g. byspecifying the cell as a nonlinear dynamic system or its gene). However, since the choice space ofgenes is usually huge, it is of practical interest to develop methods to locate in this space ofparameters the regions where emergent phenomena are likely to occur. This question is in factdifficult to answer since “the surprise effect” associated with emergence does not allow us to specify inadvance what we are we looking for i.e. the desired emergent behavior.

The main question is: under what conditions (described in L1) is the system emergent?

Emergent computation is an evolutionary process. For example, one may start with a cell definitionleading to a non-emergent behavior and then, applying successively genetic operators (e.g. simplemutations) followed by observations of the global behavior one can detect emergent phenomena thatmight be of a computational nature.

Method: Construct cellular systems with cells properly tuned (gene is chosen in regions of the cellparameter space such that emergent phenomena are likely to occur) and then evolutionary explore thespace around this initial guess solution. During this exploration some genes will be selected, theones that generate surprisingly behaviors which in addition may match certain computationalobjectives.


Steps in our design for emergence method

1. Introduce a cell parameter space, which can be effectively defined after a cellmathematical model is specified. Any possible cell is now specified only trough a genevector of real parameters [ ]m1 ggg ,...,, 2=G

2. Find a way (theoretical or algorithmic) to identify those regions in the cells parameter spacesuch that emergent phenomena are likely to occur in the array of coupled cells.

- For Reaction Diffusion type of continuous-time cellular systems, an analyticalmethod was developed based on the powerful local activity concept introduced byChua.- For the case of discrete time generalized cellular automata an algorithmicmethod is proposed in this course to identify regions in the cell parameter space suchthat emergent behaviors are likely to occur. It is based on a newly introducedmeasure of emergence.

3. Refine the outcome of the previous step in locating much finer sub-domains where applicationspecific genes will be found using evolutionary techniques.


A good “design foremergence” techniqueis capable to deriveboundaries in the cellparameter space suchthat emergence is likelyto occur for cellparameter pointslocated within theseboundaries.

Visual interpretation


Brief on the local activity theory

A cell could be passive or active. For example think of a seminar room. The lecturer provides the groupwith some ideas, consequently sending a flow of information.

Some of the students will receive the information but will simply absorb it without furtherprocessing and feedback. Such students are passive since they do not resonate with the flow ofinformation. If the students would all be passive after several time there will be nothing to be discussedin that group and there will be no information exchange at both local and global (i.e. entire group)level.

However there may be some students in the room interested in the lecturer’s speech. They willreact to the input flow of information by locally processing it and sending back information to theirneighbor cells (fellow students and their teacher). These are locally active cells in our cellular systemsframework and they may contribute significantly to the emergence of new ideas.

Note that emergence will produce surprisingly new global ideas starting from the interaction ofactive cells. Still a locally active cell (read “student” in our context) could be a noisy one, whichproduces a lot of information, some of which is not necessarily related to the content of the talk. Suchcells may be called active but unstable to differentiate from active and stable cells.


In a system made of passive cellsemergence is not possible

Refinements of this theory allow locating the emergence sub-domains in the gene parameterspace therefore giving an efficient procedure to locate initial gene solutions in designing foremergence problems.

The main result of the local activity theory

Passive cells cannot amplify small perturbations, while active cells do. A network of resistorsand capacitors is of no practical use since all the above devices are passive electronic devices.However, when operational amplifiers, transistors or other active electronic devices are addedto the network – useful computational information processing emerge (e.g. amplification,filtering, demodulation, etc.). Actual computers are in fact networks of coupled active devices(logical gates), each cell operating in a binary mode.


Design for emergence includes in fact the whole spectrum of design rules in the field of electronics andits evolutionary feature is obvious. Starting from simple tube schematics in the pioneering years of the20’th century generations of engineers evolved more and more complicated schematics allowingvarious types of information processing to emerge.With the advent of emerging technologies (e.g. nanotechnologies or molecular computing), the interestfor understanding and "programming" cellular computers is continuously growing.Among the most impressive results expected are computing systems capable of self-reproduction andself-repair while performing brain-like operations at a very high speed.Such kind of systems may easily find applications in intelligent sensors and autonomous micro robotsbut also in other various tasks requiring fast and multi-dimensional signal processing. In suchtechnologies, networks of similar coupled cells are easy to generate.

Therefore a consistent and universal theory of “designing for emergence” in cellular systems is likelyto provide the effective tools for designing applications for these new classes of devices. Moreover,such theories are likely to bring more light in understanding how complex life forms can emerge fromsimple egg cells.

On the potential and applications of designing for emergence


2. SEVERAL TYPES OFCELLULAR SYSTEMS,

MATHEMATICAL MODELS ANDCOMPUTER SIMULATION


Generalities on Cellular Systems

The main features of cellular systems are regularityand homogeneity. In fact a cellular system can bedefined as a structured collection of identicalelements called cells. The structure is given by thechoice of a lattice. Such lattices are 1-dimensional,2-dimensional and, less used, 3 or moredimensional. The following are examples ofcommon used 2-dimensional lattices:

In the above pictures, the central cell is depicted in red while cells in the neighborhood are depicted inmagenta. The neighborhood is another important concept which defines a cellular system and itrepresents the set of cells that are directly interacting with the central cell.The basic computational unit in a cellular automata is called a cell and such cells are in fact nonlineardynamic systems. The cell dynamics can be continuous in time and in this case they aremathematically described by Ordinary Differential Equations ODEs or discrete in time and in suchcases their dynamics is described by a difference equation.

Fig.1


( )∑∈

−⋅=Nk

kk tyaty 1)(1 defines the dynamics of the output of a discrete time autonomous cell.

It computes the output as a weighted summation of all neighboring cell outputs at the previous timeclock. The neighborhood is represented by the set N and a unique index k is chosen to identify theneighboring cell in a particular neighborhood. The assignment of different integer values to k isconventional and some examples are provided in Fig.1. where such indexes are depicted in white foreach type of neighborhood. The time variable t is here an integer.

The above cell is one of the simplest possible. It is autonomous since there is no external input to drivethe dynamics of the cell. In the most general case a cell is described by the following variables:

Example 1

Inputs – usually denoted by variable u (scalar) or u (vector of inputs);States – usually denoted by variable x (scalar) or x (vector of states);Initial states- a particular state variable at the initial moment, t=0.Outputs – usually denoted by variable y (scalar) or y (vector of outputs).

In terms of a computing machine, inputs and initial states are used to supply the cellular system with the input data (to be processed)while the result or output data is available in the form of a pattern (spatial, temporal or spatio-temporal) of the outputs



GenesObserve that in all previous examples the dynamics of the cell for the same input excitation and the sameinitial state is significantly influenced by the values given to certain parameters (denoted ,, and z in theabove example). In [Chua, 98] it was proposed to pack all these parameters in a unique vector Gcalled a gene since it determines the overall function of the cellular system much like the DNA – basedgene determines the functionality of the biological systems made of cells containing that DNA.

30

[ ],...,..,,..,,.., 221121 nn bbbaaa=G

Boundary conditions The cells located on the boundary of a lattice have a special regime. One should define the way inwhich these cells interact with their neighbors. This is called a boundary condition. The choice of theboundary condition influences the cellular system dynamics, therefore this is an issue that has to beclearly specified. The most common boundary conditions:

Periodic: Opposite borders of the lattice are "sticked together". A one dimensional "line” becomesfollowing that way a circle, a two dimensional lattice becomes a torus.Reflective: The border cells are mirrored: the consequence are symmetric border properties.

In the remainder of the course exclusively square lattices and periodic boundary conditions are used.


Discrete and Continuous states / outputsIn defining a cellular system one has to define the variation domain of the state and output variables.For example, one can use continuous state cells where the outputs are defined within a boundedinterval or one can also use discrete state cells where the states or/and the outputs belong to a finiteset of possible values.

A binary output cell implements Boolean functions. Each input also represents a truth value. The cellcan be again specified as a discrete-time dynamical system but it can be also specified using atransition table or a set of local rules. The last two modes of specifying a cell are specific to theCellular Automata formalism. A compact piecewise linear description (i.e. a discrete dynamical system)can be found for any Boolean or other type of input function.


Major cellular systems paradigms - The cellular automata (CA)

The Cellular Automata are widely studied today as a convenient paradigm for modeling physicalprocesses and for investigating emergence and complexity. Several good on-line tutorials are available,for example [Schatten A., 1999], [Rennard, 2000], [Weimar, 1996] to name just a few of the most recent.

A famos CA example is the Conway’s “Game of Life” [Gardner, 1970]. It is a binary, discrete time cellular automata wherethe cell is defined by some simple, common sense, local rules. It is assumed that each cell has 8 neighbors (Mooreneighborhood) and each cell is either DEAD (coded as state 0) or ALIVE (coded as state 1). There are two possibilities:- The cell is DEAD. Then it become ALIVE in the next cycle only if 3 of its neighbors are ALIVE. Otherwise it stays DEAD(This rule somehow suggests that a new being is generated if enough people are there around. Though, too many peoplemay compete for resources and there is no place for a “new life”).- The cell is ALIVE. Then except if it has 2 or 3 ALIVE neighbors it will become DEAD in the next cycle. Otherwise it willstay ALIVE. This rule suggests that a cell could die either by loneliness (only 1 living cell around) or by overpopulation(more than 3 cells around competing for resources makes life impossible).

It was proved [Berlekamp et al., 1982] that for certain configurations of initial states the “game of life”CA embeds a universal Turing machine and therefore, in principle, a CA with “game of life” cells iscapable of universal computation.

http://www.bitstorm.org/gameoflife/ - Game of Life simulator


Major cellular systems paradigms - Cellular Neural Network (CNN) model

The CNN cell is a continuous time and continuous state dynamical system with somesaturated nonlinearity (see equation (1) above) which is well suited for implementationusing analog circuits.

Several generations of microelectronic chips were reported so far [Roska & Rodriguez-Vazquez, 2000a], as well as development tools which allow an user to program the CNN asvisual microprocessor.There is a wide range of applications, mostly in the area of image processing. Suchapplication include image segmentation, image compression, fast halftoning, contourtracking, image fusion, pattern recognition, to name just a few.Although initially the equilibrium dynamics of CNNs was mostly exploited for applications,recently the non-equilibrium dynamics is employed for certain interesting applications inwhat is currently called “computing with waves” [Roska, 2001],[Roska, 2002].

33


The additional CNN loop is thus given by the discrete time equation There aretwo cases of interest:

34

Major cellular systems paradigms - The Generalized Cellular Automata

In [Chua, 1998], the idea of a Generalized Cellular Automata (GCA) was introduced as anextension of the CNN so that a GCA includes CAs as a special case. The main idea of theGCA is to use a CNN running for a period T in a discrete-time loop.

Uncoupled CNN cell: (the CNN cell has no connection with its neighbours). Then the nonlineardynamic system (1’) converges towards a steady state output solution where the output of eachcell can be described as In this case, the GCA can emulate any CA, providedthat there exist a method to map the local rules or transition table into a nonlinear function Fassociated with the CNN cell

Coupled CNN cells: The resulting GCA is more complex than a normal CA. The reason is thatan emergent computation already takes place during the period of time T in the CNN, therefore atthe discrete time moment when the output of the CNN is sampled, it does not represent only thecontribution of the neighbouring cells as in the case of a “classic” cellular automata but rather thecontribution of all CNN cells.

)()( Ttytu kk −=

( ) ( )( )tuFty k,1 G=


Major cellular systems paradigms -Reaction-Diffusion Cellular Nonlinear Networks

Fig.2 A cell is a m-port described by anonlinear ODE which models a physicalreaction. The coupling with neighboring cellsis done via resistive grids modeling thephysical process of diffusion

Reaction-Diffusion CNNs (RD-CNNs) were proposed in [Chua et al., 1995] as a particular case ofcontinuous-time autonomous CNN which are space-discretized models of Partial differential Equationsdescribing the Reaction-Diffusion physical processes.


Simulating the Uncoupled GCA (emulator for any CA type)

The nine inputs of this function can be either a scalar, a vector or an array. The program aboveimplements the Parity9 local logic function. However, by simply changing the gene parameters (in ourcase the values of the Z,B,s) many other local Boolean functions can be implemented.


The code GCA_U.M to implementthe GCA itself is a Matlab functionwith two input parameters. The firstis a number and represents thenumber of iteration steps until stop(a faster stop can be achieved bypressing the keys CTRL and C).The second is optional and itrepresents the name (a string) of afile containing the initial state. Theinitial state gives also informationabout the size (number of cells) ofthe GCA. The lattice is square witha Moore neighborhood indexed asexplained in the source code andthe boundary condition is periodic:


With these two simple programs you may begin to observe emergent patterns in generalizedcellular automata. First, running the program as they are (in fact you should run GCA_U(100)for a 100 steps run) a sequence of growing binary patterns is observed. The color code usedhere associates color red with +1 and blue with –1.

One can easily change the behavior by re-editing the GCA_U_CELL.M file. For example afterreplacing Z=[-1,-2,-4,-8,-7] with Z=[-1,-2,-4,-6,-7] the result will be a sequence of “snow-flake”like binary patterns. Moreover, if you use the initial Z but replace the initial B with B=[1 1 1 1 -8.5 1 1 1 1] and use the output function y=0.5*(abs(w+1)-abs(w-1)) the result is a sequence ofcolored patterns corresponding to a continuous-state cellular automaton.

Observe that much of the flexibility in programming different behaviors using the same code isdue to the use of a universal piecewise-linear parametric functional (the function with nestedabsolute values).

38

Experiments


Simulating the Coupled GCA ( the CNN kernel)

In the implementation of the cell for the uncoupled GCA there is no temporal dynamics involved and theoutput is just a nonlinear function of the 9 inputs.

In fact, when called from the mainfunction GCA_U the functionGCA_U_CELL updates all cellssimultaneously since it performsmatrix computation (all inputs, forexample u1, are matrices of the samesize as the CNN). We can viewGCA_U_CELL as a layer of CNN cells,but since the model is simplifiedthere is no temporal dynamicsincluded. In order to implement acoupled GCA we should change thecell function such that it willimplement the dynamic evolution of acontinuous time CNN. The resultingcode of the program GCA_C_CELL.Mis listed next:


This new cell is in fact a implementation of the standard (linear coupling) CNN model (of Chua andYang [Chua & Yang, 1988] but in fact can be easily developed into a nonstandard model by simplychanging LINE A and LINE B accordingly.The continuous time dynamics associated to Equation (1) is emulated on the discrete-time computerusing a simple integration based on the Euler’s method. Therefore, in addition to gene parameters thisnew cell has to specify two dynamic parameters. The first, Delta_T indicates the step size. Thesmallest the better will be the approximation of the continuous time dynamics. The second, T,represents the integration time period. The user may choose such a value that corresponds to a steadystate dynamics in the output.

In order to simulate a coupled GCA one should use the above file in conjunction with a file calledGCA_C.M which is simply obtained from a copy of the GCA_U.M file after changing the name of cellfunction from GCA_U_CELL to GCA_C_CELL.Using the values in the file above, the dynamics of the CNN will be first observed, and from time to timea figure will show the GCA output. The visualization of the CNN steps is useful for first experiments andit allows one to choose the two dynamic parameters. However, if one wants to see only the GCA output,the visualization lines in the above file should be ignored (by inserting % in front of them).

Simulating the Coupled GCA (discussion)


Simulating the Reaction-Diffusion CNN - The Cell

We will exemplify the simulation for the particular case of FitzHugh Nagumo cells (See next chapter fordetails). However, any Reaction-Diffusion system can be easily simulated after several minor re-editingoperations on the files. As in the previous cases we should first define the cell equation. In the case of theReaction-Diffusion systems this is done by the following function (RD_CELL.M):

the nonlinear functionsf1 and f2 in (3) areimplemented separatelybeing specific for theFitzHugh-Nagumo cellmodel. They are asfollows:


Simulating the Reaction-Diffusion CNN - explanations

Here a two-layer Reaction-Diffusion CNN is implemented. The Reaction-Diffusion systems in thenext chapter are all 2-layers systems. The function RD_CELL.M is prepared to be used in anMatlab specific integration routine based on the Runge-Kutta algorithm.Since the integration routine ODE23 requires a unique vector variable x , before preparing towrite down the cell equations as in (3) we have first to unpack the vector variable into the twovariables u (first layer) and v (second layer). The two variables are square matrices of dimensionN each while an element u(i,j) of each matrix corresponds to the CNN cell on the (i,j) position. The first line allows one to specify the gene parameters of the cell. Different dynamic behaviorscan be simulated by changing these values and saving the file. The next lines deal with theimplementation of (3).

Whenever a different type of cell (of order two) has to be implemented one needs only to changethe above function and to provide the adequate list of parameters in RD_CELL.M. No otherchange is required in the file. Next let us examine the source code of the main simulatorfunction called RD_CNN.M


Simulating the Reaction-Diffusion CNN - Main Simulator


This program allows the integration of the ODE description of the cell in file RD_CELL.M and itallows also to display the dynamic evolution as a sequence of 5 or 42 snapshots depending onthe selection of the input variable MODE. The method of integration is the one specific of thefunction ODE23, but one can simply replace it with other ODE integration routines available inMatlab such as ODE45, ODE113, etc.The only parameter which has to be specified is the vector ts which specifies the moments oftime for display, e.g. ts=1:5:67 means that there will be displayed snapshots taken at themoments 1,6,11,..67 i.e. from 5 to 5 time periods.Each snapshot is a colored image and the color of each pixel codes the amplitude of the statevariable u using the colormap code assigned to the display. The colormap code used by defaultis JET but it can be easily changed with one of the other available colormaps . During thesimulation, the maximum and minimum values of the state variables are recorded and typed tothe screen. When the snapshots are displayed these extreme values are assigned to the colorsrepresenting the minimum and respectively the maximum amplitude. For JET, the colorassigned to minimum is dark blue and for maximum dark red.

Simulating the Reaction-Diffusion CNN - explanations


Using the default parameters and typing RD_CNN(1) a dynamic evolution towards anon-homogeneous stable state is observed. If the parameters are changed as follows:a=0.0; b=1.3; e=-0.1; alfa=1; d1=.1; d2=0; in RD_CELL.M and ts=0:5:200 in RD_CNN.Mthe result of running RD_CNN(1) is an emergence of colliding waves.

As shown in next chapter, a precise location of the sets of parameters producinginteresting and potentially useful emergent behaviors can be done employing the localactivity theory.

EXPERIMENTS


3. EMERGENCE IN CONTINUOUS-TIMESYSTEMS:

REACTION-DIFFUSIONCELLULAR NEURAL NETWORKS


The local activity theory [Chua, 1998], [Chua, 1999] offers a constructive analytical toolfor predicting whether a non-linear system composed of coupled cells, such as reaction-diffusion and lattice dynamical systems, can exhibit complexity.

The fundamental result of the local activity theory asserts that a system cannot exhibitemergence and complexity unless its cells are locally active.

This chapter presents an application of this new theory to several Cellular NonlinearNetwork (CNN) models of some very basic physical and biological phenomena. The firstis a model of nerve membrane due to FitzHugh-Nagumo, the second is the Brusselatormodel used by Nobel laureate Ilya Prigogine to substantiate on his “far from equilibrium”systems theory and the last is the morphogenesis model of Gierer and Meinhardt toexplain pattern formation in living systems.

Generalities


Explicit inequalities defining uniquely the local activity parameter domain for eachmodel are presented. It is shown that when the cell parameters are chosen within asubset of the local activity parameter domain where at least one of the equilibriumstates of the decoupled cells is stable, the probability of emergence increasessubstantially. This precisely defined parameter domain is called the “edge of chaos”, aterminology previously used loosely in the literature to define a related but much moreambiguous concept.

Numerical simulations of the CNN dynamics corresponding to a large varietyof cell parameters chosen on, or nearby, the “edge of chaos” confirm the existence ofa wide spectrum of complex behaviors, many of them with computational potential inimage processing and other applications. Several examples are selected in this chapter todemonstrate the potential of the local activity theory as a novel tool in nonlinear dynamics.It is not only an instrument for understanding the genesis and emergence of complexity,but also an efficient tool for choosing cell parameters in the spirit of „designing foremergence“.


While it is generally recognized that nonlinearity is a condition for complexity [Schrödinger, 1967],[Prigogine 1980], [Nicolis, G. & Prigogine, I. 1989],[Haken 1994], it was shown in [Chua 1998], forthe first time, that nonlinearity is too crude a condition for complexity.

A complete theory for “Reaction-Diffusion” CNNs reveals that the necessary condition for a non-conservative system to exhibit complexity is to have its cells locally active in a precise sense tobe defined in Section 3.3.3. In other words, unless the cells are locally active (or equivalently, arenot locally passive) no complexity can emerge.

This theory offers a constructive analytical method for uncovering local activity. In particular,given the mathematical model of the cells (the kinetic term in reaction-diffusion equations), onecan determine the domain of the cell parameters in order for the cells to be locally active, andthus potentially capable of exhibiting complexity.

The theory of local activity provides a definitive answer to the fundamental question: What are thevalues of the cell parameters for which the interconnected system may exhibit complexity?


A large variety of complex dynamical phenomena from reaction-diffusion systems, includingchaos, spiral waves, Turing patterns, etc., have been described in the literature [Murray 1989].

The CNN paradigm provides a unified treatment of all such systems in terms of the associatedcells, and couplings.

Until the recent development of the local activity theory [Chua, 1998], to obtain a desireddynamical behavior, the cell parameters are usually determined empirically by trial-and error.

By restricting the cell parameter space to the local activity domain, a major reduction in thecomputing time required by the parameter search algorithm is achieved.

Another important aspect of the local activity theory for reaction-diffusion systems is that itreduces the complexity determining procedure to only the cell parameter domain, therebyignoring the “couplings”; namely, the diffusion coefficients.


Locating Points of Interest in the Cell Parameter Space

Using tools and methods to be described next in detail, several behavioraldomains (labeled with different colors in the figure) and the boundaries betweenthem are defined.

Then, parameter points are chosenwithin the „edge of chaos“ domain tomaximize the likelihood of detectingemergent phenomena.

As the particular example in the figureshows, the „red“ domain is quitenarrow in a 2-dimensional sectionwithin the (a,b) cell parameter domain.This gives a glimpse on the efficiencyof the method. No emergentphenomena will appear if the cellparameter points are located in the„blue“ (passive) regions


Next it is shown how to apply the theory of local activity in general with examples for theFitzHugh-Nagumo CNN model of nerve membranes

Choosing cell parameters within, or nearby the edge of chaos, complex behaviors didemerge with a high probability for various choices of the coupling parameters (i.e., thediffusion coefficients). The task of determining those cell parameters for which complexitycan emerge is not trivial. Indeed, our previous results on both models indicate that theedge of chaos domain is usually a relatively small subset of the entire cell parameterspace. Finding this subset without applying the local activity theory is like fishing for aneedle in a haystack and would have been impractical.

The following methods can be applied only to Reaction-Diffusion CNN systems with twostate variables per cell. In the following we will exemplify the steps in determining the“edge of chaos” domains for the FitzHugh-Nagumo Reaction-Diffusion CNN.

52


"Edge of chaos" is a narrow subset of the local activity domain

Passive Active (potential for emergence)Edge of chaos

(strong potential for emergence)

• Test for local activity• Add supplementary constraints to identify the „edge of chaos“


Starting with a model

A Reaction-Diffusion system is mathematically described by a system of Partial DifferentialEquations (PDE) with one or two positive diffusion coefficients, respectively:

PDE of the Reaction-Diffusion system

with one diffusion coefficient D1

PDE of the Reaction-Diffusion system

with two diffusion coefficients D1, D2


Four steps precluding the testing for local activity

Step ¶¶ To apply the concept of local activity, the first step is to map the above partialdifferential equations (PDE) model into the following associated discrete-space version, called aReaction-Diffusion Cellular Neural Network (CNN) in [Chua, 1998] :



Step ·· The second step in applying the theory of local activity is to find the equilibrium pointsof Eqs. (1’) and (2’) (we will henceforth delete the spatial coordinates to avoid clutter) :( )j k,


Step ··






Step ¹ The fourth step in applying the theory of local activity is to calculate the trace and the determinant of the Jacobian matrix (13) about each equilibrium point:T Q i( ) ∆( )Qi


The concept of local activity


The concept of local activity


Refining the local activity domains

A methodology was described before (the 4 steps) such that at each cell parameter point and for eachequilibrium point of the uncoupled CNN cell one can compute three coefficients and Accordingto corollary 4.4.1 in [Chua, 1998], for any reaction-diffusion system with two state variables and onediffusion coefficient these three scalars suffice to classify the emergent dynamics into one of thefollowing three categories:

,,∆T 22a

Locally active and stable (usually points in the cell parameter space are labeled with color redwhen belonging to this class);Locally active and unstable (usually points in the cell parameter space are labeled with colorgreen when belonging to this class);Locally passive (usually points in the cell parameter space are labeled with color blue whenbelonging to this class);

A similar classification can be done using the four coefficients obtained in Step 3, Equation (13’)above, for the case of two-diffusion coefficients. As shown next, the local activity domain for thecase of two diffusion coefficients includes the local activity domain for the case of a singlediffusion coefficient



T

∆∆

a22<0

T

∆∆

a22>0

Unstable and locally active

A sub-region of the local activity domain where the equilibrium point isboth stable and locally active has a special physical significance since itcorresponds to the operating mode of all active electronic devices used ininformation processing (e.g. transistors, logic gates, etc.). We willhenceforth define this region an edge of chaos domain.


LEGEND

T = a22

a22

T = a11 + a22

1011

12

13

14 15

16

9

7,11,10

6,8

x

x

13,14

x

15

xx

x

x

4

1

2

3-5 5

x19

x

16

x9

N c<-1

N c=1

N c=0.1

FitzHugh-Nagumo Cell Parameter Projection Profile (CPPP), within the edge of chaos ( c=1 )

N Labeled cell parameter point “N”; the background color corresponds to different values of the c parameter

a 2 2

∆∆ > 0T < a 2 2< 0

∆∆ = 0

∆ ∆ > 0a 2 2 < T < 0

∆ ∆ > 0T < 0 , a 2 2 > 0

1 3

1 21 1 1 0

9

1 4 1 51 6

43 2

1

876

5

F i t z H u g h -N a g u m o C e l l P a r a m e t e r P r o j e c t i o n P r o f i l e ( C P P P ) , w i t h i n t h e e d g e o f c h a o s ( c = 1 )

N L a b e l e d c e l l p a r a m e t e r p o i n t “ N ” ; t h e b a c k g r o u n d c o l o r c o r r e s p o n d s t o d i f f e r e n t v a l u e s o f t h e c p a r a m e t e r

65

Passivity and Activity (stable and unstable) in the abstract T-∆∆-a22 space

The exact shape of this sub-domain, called a cell parameter projection profile (CPPP) depends on thenonlinear functions defining the cell and on its specific set of tunable parameters. In particular, a CPPP isdefined as the projection of any prescribed subset of the cell parameter domain of interest into the T-∆∆-a22space



For the particular case of the FitzHugh-Nagumo cell

„edge of chaos“

„edge of chaos“

„passivity region“

„passivity region“

1V Is the „d.c.“ voltage at the given equilibrium point

The boundaries between domains depend on the cell parameters and its equilibria !!


Unrestricted versus restricted local activity / passivity

( )∞∞−∈ ,, 21 II

Unrestricted

Although correct from a puretheoretical point of view, it leadsto a situation where most of thecell parameter points are locallyactive. However, the simulationsshow that many cell parameterpoints within such local activitydomains do not lead toemergent behaviors

Determining the unrestricted (theoretical) localpassivity and local activity domains bynumerical methods can sometimes lead toerrors in the general case because it isimpractical to conduct the test for all equilibria.Fortunately, for cells with two state variables,analytical techniques can be applied so thatthe local passivity and local activity domainscan be determined analytically.


Partially motivated by the fact that in practice both actually would vary withina limited domain (otherwise the power within the coupling resistors in a physicalrealization would become infinity),

The above justify why we will narrow even more the domains where emergence islikely to occur. First, restricted local activity domains will be defined by taking theassumption and , then additional constrains will be imposed leading tothe definition of the “edge of chaos” domain and a methodology for identifying it.

69

Restricted Local Activity / Passivity, a Significant Step Towards Locating Emergent Behaviors

01 =I 02 =I

1I 2I

021 == II

Restricted


Several cross sections identifying the local activity domain(coded in red) and local passivity domain (coded in blue), ofthe FitzHugh-Nagumo Equation. The locally passiveregions in (a) and (b) are printed in non-uniformly faded bluein order to enhance their 3-dimensional perspective.FitzHugh-Nagumo cells from these bluish regions can notexhibit any form of complexity regardless of thecoupling coefficients:

(a) Three-dimensional domain: one diffusion coefficient case;(b) Three-dimensional domain: two diffusion coefficientscase; (c)Two-dimensional cross-sections; The left sidecorresponds to the one diffusion case and the right side tothe two diffusion case, respectively; (d) Two-dimensionalcross-sections. Each point marked on these cross-sectionsis associated with a set of cell parameters shown in theyellow “caption box”. Dynamic simulations of thecorresponding CNNs are shown next where the number inparenthesis is the same as the label shown in the light blue“caption box” attached to each parameter point at the tip ofan arrowhead.

Example


Dynamic simulations

Many other points in the local passivity regions have beenrandomly chosen and for all of them the dynamic simulation ofthe corresponding CNN had confirmed the predicted relaxationbehavior towards a homogeneous equilibrium state for all cells.On the other hand, by randomly picking points in the unrestrictedlocally active domain, some complex phenomena were indeedfound but with a very, very small probability. This observationslet to the conclusion that although theoretically correct, theunrestricted local activity domain is too large and we may findsome additional constraints to narrow the search within a muchsmaller sub-domain of the cell parameter space called an “edgeof chaos domain”.


The Edge of Chaos

The jargon “edge of chaos” has been used loosely in the literature to mean a region in the parameterspace of a dynamical system, such as cellular automata or lattices, where complex phenomena andinformation processing can emerge [Packard, 1988], [Langton, 1990]. Still it remains an ambiguousconcept without a rigorous foundation. In the light of the local actvity theory, it is possible to propose aprecise definition and to illustrate its scientific significance as a “complexity” predicting tool.


The algorithm for finding the edge of chaos C

Slides 63,66


Restricted local activity (red) and local passivity(blue) domains in the parameter spacefor . The coupling inputs , and havebeen restricted to

(a) The one diffusion coefficient case;

(b) The two diffusion coefficients case. Since thered region here is larger (especially clear nearthe origin) than the red region from above, thelocal activity domain in this case includes thelocal activity domain shown in (a) as a propersubset. This property holds in general.

74

(a)

(b)

locally activelocally passive

locally activelocally passive

( )a b, , εc = 1 I1 2I

I I1 2 0= =

EXAMPLE


(a) (b)

(c) (d)

Legend Edge of chaos

Locally active and unstable at all equilibrium points

No equilibrium point is both stable and locally active

Locally passive

7

(a) (b)

(c) (d)

Edge of chaos

Locally active and unstable at all equilibrium points


Locally passive

Legend

8

Details on „Edge of Chaos“ and dependence of coupling currents


Getting Into More Details

In the next, the c parameter will be fixed and we will generate bifurcation diagrams within aspecified bi-dimensional domain (e.g. the plane of the a,b parameters)

Next we will present examples of such bifurcation diagrams when the cell parameters arerestricted to [ ]a ∈ −3 3, [ ]b ∈ −3 3, c = 1 ε = −010.

Recall that the FitzHugh-Nagumo Equation can have at most 3 equilibrium points•The regions in the parameter domain having one or 3 equilibrium points will be depicted using light or dark color,respectively.•The color blue was chosen to code the restricted local passivity domain.•The orange color corresponds to the region where there is one active and stable equilibrium point.•Within the local activity domain, the red and the orange colors denote regions where there is at least one equilibriumpoint, which is both stable and locally active, and hence they correspond to the edge of chaos domain.•The yellow regions correspond to domains where even though there is no equilibrium point, which is both stable andlocally active, there is at least a locally active but unstable equilibrium point.•Finally, the color green is assigned to regions where all equilibrium points are both active and unstable. The light greenregion corresponds to one unstable equilibrium point while dark green regions are assigned to more (three) unstableequilibrium points.


Examples of Detailed Bifurcation Diagrams

C3

C4

-3 0 3a

0

3

b

Edge of chaos ( S(Qi )A(Qi ) for at least one Qi)

Locally active and unstable ( A(Qi )U(Qi ) )

No. of Qi : 3 1

Locally passive


a

0

3

b

-3 0 3

Edge of chaos ( S(Qi )A(Qi ) for at least one Q i)

Locally active and unstable ( A(Qi )U(Q i ) )

No. of Qi : 3 1

Locally passive


One-diffusion coefficient Two-diffusion coefficient


Locating emergent behaviors near and within the “edge of chaos”

Particular points ofinterest in theparameter domain areassigned a numericallabel (e.g. 5 here) usedto identify the dynamicCNN simulation.

The label above (hereIC) identifies a type ofbehavior that can beidentifed after thedynamic simulation.

“C” - chaotic dynamic pattern, “P” - periodic dynamic pattern, “S” - spiral wave pattern, “T” - Turing pattern, IC - information computation pattern


Dynamic behaviors for cell parameter point „5“ and different initial conditions

Stationary patterns, mayhave a computationalmeaning, e.g. detectingedges from a gray levelimage (the initial state)

Note that point „5“ waschosen within the „edgeof chaos“ domain forthe case of D2>0 (twodifussion coefficients)

79


The influence of the coupling currents

The influence of I1

The influence of I2


a) No emergence when the cells are not coupled

Emergent Behaviors for Cell Parameter Point (CPP) 6

b) Emergent spiral waves when D1>0

c) Emergent spiral waves when D2>0



a) Oscillatory (but not emergent) dynamics in the array of uncoupledcells (D1=D2=0) with parameters in the locally active and unstableregime (near the small edge of chaos region)

b) Emergent chaotic patterns when D1>0

c) Emergent static patterns when D2>0



a) Stationary (but not emergent) dynamics in the array of uncoupledcells (D1=D2=0) with parameters in the locally active and stableregime (near the small edge of chaos region)

b) Emergent (dynamic) spiral patterns when D1>0

c) Emergent (static) Turing patterns when both D1, D2>0



a) Emergent (dynamic) periodic patterns when D1>0

b) Emergent (static) information computation (generating atexture) patterns when D2>0


Emergent Behaviors for Cell Parameter Point (CPP) 10,11,12

CPP 10) Emergent (dynamic) spiral patterns

CPP 11) Emergent (static) information computation (cornerdetection) patterns

CPP 12) Emergent (static) information computation (globaledge detection) patterns


A new section, for ε = −2 Emergent Behaviors for Cell Parameter Point (CPP) 13,14,15

CPP 13) Emergent (dynamic) periodic patterns when D2>0

CPP 14) Emergent (static) information computation (cornerdetection) patterns

CPP 15) Emergent (static) information computation(generating a texture) patterns when D2>0


A new section, for ε = −2 Emergent Behaviors for Cell Parameter Point (CPP) 16,17,18

CPP 16) Emergent (static) information computation (noiseremoval from background) patterns.

CPP 17) Emergent (static) information computation(localized edge detection) patterns

CPP 18) Emergent (static) information computation(generating a texture) patterns when D2>0


Two new sections for 1.0=c Emergent Behaviors for Cell Parameter Point (CPP) 16,17,18

In this case an interesting type of emergent behavioris observed. The final pattern has a number of active cells(red) proportional to the voltage of the initial state. Thisbehavior can be regarded as analog to digtal conversion.


Another Example of Locating Emergence: The Brusselator CNN

The Brusselator model [Prigogine, 80] is chosen here for its historical significance: It is the firstmathematical model used to explain the self-organization phenomena observed in chemicalreactions of the reaction-diffusion type. Based on this model, a theory of dissipative structuresoperating far from thermodynamic equilibrium had been developed by Nobel laureate IlyiaPrigogine

As shown next, the local activity theory offers a rigorous and effective tool for sharpeningexisting results on systems operating far from thermodynamic equilibrium in the sense that itcan identify more precisely those regions in the cell parameter space which are capable ofemergent behaviors. In particular it is confirmed that all results previously reported by thePrigogine school using the far from thermodynamic equilibrium theory can be derived directlyfrom the local activity theory.

( ) ( ) ( ) ( )( ) ( )

( ) ( ) ( )( ) ( )

∂∂

∂∂

V x y

ta b V x y V x y V x y D V x y

V x y

tbV x y V x y V x y D V x y

11 2 1

21

21

21 2 1

2

22

2

1( , )

, , , ,

( , ), , , ,

= − + + + ∇

= − + ∇

The mathematical model


The 4 Steps - A Basis for Further Analysis


before


The cell parameter projection profile (CPPP)


Fig. 22

Cell parameter projection profile of the Brusselator


Locating emergent behaviors near and within the “edge of chaos”


Detailed Bifurcation Diagrams


Emergent Static Patterns (the ID identifying the cell parameter point is on the left)


Emergent Static Patterns (the ID identifying the cell parameter point is on the left)


Emergent Dynamic Patterns (the ID identifying the cell parameter point is above)


Emergent Dynamic Patterns (the ID identifying the cell parameter point is above)


Although the “edge of chaos” domain as defined within this chapter is giving a much narrowdomain than the local activity domain, its definition cannot still guarantee the computationalemergence (static or dynamic). The edge of chaos membership of a cell parameter point is anecessary but not sufficient condition for emergence. But this is reasonable since we performall our analysis using the uncoupled cell, therefore ignoring the coupling. On the other hand amathematical analysis of the whole CNN systems (coupled cells) is prohibitive and difficult tocarry.In addition there are some heuristics based on observations coming from the “edge of chaos”analysis that we have performed on several second order Reaction-Diffusion systems. Suchheuristics can be used to choose better inside the edge of chaos domains and they also giveindications on how to choose the diffusion coefficients such that emergence will occur.


Heuristics in determining cell parameter points leading to emergence

It was clearly determined that richer emergent behaviors occur in cells having more equilibrium points at the “edge ofchaos”. Let us recall the FitzHugh-Nagumo model where the most interesting emergent phenomena occur in the edgeof chaos region in a sub-region characterized by three stable equilibrium points and in the nearby of the cell parameterpoints satisfying both conditions C3 and C4 in the set of local activity conditions. Such behaviors as spiral patternswere not observed in systems characterized by a unique equilibrium point. There are several more common aspects,which appear to be independent of the cell types:


Engineering perspectives and potential applications of the edge of chaos analysis

The whole methodology exposed in this chapter is applicable only for second order cells (i.e.defined by 2-state variables). In a practical implementation these corresponds to maximum twolayers of resistive grids connecting the cells (i.e. the implementation of the diffusion localconnectivity). The “reaction” part is in fact the locally active cell, which can be easilyimplemented as a nonlinear electronic (or possibly molecular) circuit.

Certain dynamic phenomena can be identified afterwards using the analytic techniques of localactivity and edge of chaos exposed herein. Such circuits can be used for various tasks, forexample as an intelligent and programmable image sensor where several basic processingfunctions can be easily integrated on the same chip including the sensing device.

Integrated circuit designs to implement a second order Reaction-Diffusion CNN were alreadyreported in [Arena et al., 1998], [Serrano and Vázquez, 1999], where some basic emergentbehaviors were also reproduced.

Another area of interest for applying the edge of chaos based methods is the emerging area ofnanotechnology.


4. EMERGENCE IN DISCRTE-TIMECELLULAR SYSTEMS:

MEASURES OF COMPLEXITY,MUTATIONS AS A TOOL FOR

LOCATING EMERGENCE

103


To identify the emergent behaviors, we have introduced a non-homogeneity measure, called cellulardisorder measure, inspired from the local activity theory. Based on its temporal evolution, we are able topartition the cell parameter space into a class U (unstable-like) region, a class E (edge of chaos-like) region,and a class P (passive-like) region. The similarity with the "unstable", "edge of chaos" and "passive"domains defined precisely and applied to various reaction-diffusion CNN systems (see previous chapters)opens interesting perspectives for extending the theory of local activity to discrete-time cellular systemswith non-linear couplings. The cell is parametric, i.e. described trough a nonlinear equation, instead of atransition table often used in cellular automata literature.

The novelty of our approach consists in a method for precisely partitioning the cell parameter space intosub-domains via the failure boundaries of the piecewise linear CNN (cellular neural network) cells [Dogaru &Chua, 1999a] of a generalized cellular automata. Instead of exploring the rule space via statistically definedparameters (such as in [Langton, 1990]), or by conducting an exhaustive search over the entire set of allpossible local Boolean functions, we explore a deterministically structured parameter space built aroundcertain cell parameter points. These cell parameter points are chosen such that their associated cellimplement "interesting" local Boolean logic functions (or transition tables). The meaning of “interesting”here is that such cells were already proved to generate some form of emergent behavior.In the next we will exemplify the above starting from the „Game of Life“ local Boolean function

Description of the method


The Generalized Cellular Automata Model


The Generalized Cellular Automata Model

106


The GCA mathematicalmodel

3 different realizations for the„Game of Life“ cell. All of themare piecewise-linear making thusvery easy to determine linearfailure boundaries

The framework for the Uncoupled GCA


Failure boundaries and paving stones in the cell's parameter space

Choose a realizationof the interestingfunction (here Gameof Life) and introducea set of parameters.Here 2 parameterswere introduced sincetwo-dimensionalparameter spaces canbe easily representedgraphically.

108


( )( ) ( )( )( )23892146715 115.25.05.1sgn uuuuuuuu-uy +++++++++++= λλ

Failure Boundaries and Local Functions Induced by the G.of.Life Boolean Cell

Parametric cell

„Paving Stone“Corresponds to a local Boolean function


A structured view of the cell param. space and some representative cell parameter points

Fig.2. A detailed view of behavioral domains with 11 representativequalitatively distinct dynamic behaviors described in the textbelow, and identified here by numerical labels attached to theircorresponding "paving stones" within the cell parameter space.Failure boundaries are enhanced for visibility and shown in darkblue. The color code of the behavioral sub-domains is defined inthe legend.

110


How can be that structure induced ?

Define a non-homogeneity measure m(t) of the instantaneous state at time t of a generalizedcellular automata, henceforth called its cellular disorder measure. Based on the temporal evolution of the disorder measure m(t), we assign the correspondingdynamic behaviors to one of three distinct qualitative classes; namely, an "Unstable-like" or "U"class, a "Passive-like" or "P" class, and an "Edge of chaos"-like or "E" class.

Class U is characterized by an exponential increase of the disorder measure, which isreminiscent of the unstable behaviors in nonlinear systems. Class "P" is characterized by a fastdecrease to 0 of the disorder measure reminiscent of damped oscillations. Class E ischaracterized by a slow decrease of the disorder measure to a non-zero value.

By simulating the generalized cellular automata dynamics for each member of the 148-functionfamily, and by calculating the evolution of the cellular disorder measure m(t) for each case, weare able to assign a qualitative label to each "paving stone" as shown in Fig.3.

111


Some structure is induced by defining and computing a measure of emergence for each „paving stone“


Cellular Disorder Measure


Temporal Evolution of Cellular Disorder Measure – Signature of Complexity


Examples of dynamic behaviors - Class P dynamics


Examples of dynamic behaviors - Class E dynamics (subclass 1)

Sub-class 1 corresponds to those paving stones colored with magenta. Its characteristic is a relatively slowconvergence towards a non-homogeneous static (fixed-point) or dynamic (low-period oscillations) patternhaving a small disorder measure. An example belonging to this subclass is shown in Fig. 6(a) for the localBoolean function labeled "32". Observe that compared to the Class P "passive-like" behavior describedabove, 44 time steps are now required, instead of 13 time steps, to reach steady state



•• Sub-class 2, which is reminiscent of the "Class IV" cellular automata proposed in [Wolfram, 1984]or of the "edge of chaos" proposed in [Langton, 1990], corresponds to those paving stones colored in red.Its characteristic is long transients (with a decreasing tendency in m) and the emergence of regularinteracting patterns. Next the behaviors resulting from mutations of the seed function "Life„ will beillustrated: Boolean functions labeled "122", "17", "55", and "45". Observe the rate of decrease in m whichtends to be slower and the average disorder measure at steady state (after a sufficiently large number ofiterations) tends to be larger than those from sub-class 1.



For the cell labeled "17", instead of a random initial condition, a small red cross (i.e., only the 5 cells composing the crossare assigned an initial output equal to +1, all others are assigned an initial output equal to -1) initial condition was chosen.Instead of an indefinite growth (typical for Class U "unstable-like" behaviors), such an initial condition typically evolvesinto a "spaceship"-like organized pattern, which changes its shape and position until the dynamics eventually settles intopatterns having a constant disorder measure.





In this case, "ant" patterns similar to that obtained for cell parameter point 55 are "born" during the first 150 cycles, andwhich evolve into colonies (e.g. at time step 141). However, besides these CNN "ants" traveling from east to west, severalnew and distinct structured patterns (many of them traveling in an opposite direction) are seen to have emerged, evolved,and interacted with the "ant" colony (time steps 141 to 321). Eventually these competing patterns are seen to have won thecompetition with the CNN "ants" so that after t=611 no "ant" pattern can be seen. A 10-step sequence shown after t=611reveals the diversity of "patterns" and their dynamic evolution.


Examples of dynamic behaviors - Class U dynamics (very long transients)

The orange color (in Fig. 2) corresponds to very long transients as observed from cell "114". In fact, as seen in Fig. 2, suchbehaviors are rather rare, where only 7 out of 148 paving stones are assigned to this sub-class. Observe also from Fig. 4that in this case the disorder measure m(t) even decreases initially as in Class E ("edge of chaos"-like) behaviors, but aftera while the dynamics eventually switches to a steady increase in the local maxima of m(t) towards 0.13 . This interestingtype of Class U ("unstable-like") behavior is presented above where large patterns grow slowly from a small initial redcross pattern in a fashion reminiscent of the stretching of a piece of rubber, or of a plant growing from its seed.


Examples of dynamic behaviors - Class U dynamics (typical)

122

The evolution depicted above for cell "12" exhibits symmetrical patterns with increasing complexity anddisorder. Recently we conducted some perception experiments with groups of people that were exposed toa unique family of fancy patterns (qualitatively looking like those shown above) and were told to categorizethem (into "like" and "dislike" categories). These experiments suggest that there exist a resonance betweena certain person and a certain group of symmetrical but unstable patterns so that, in general, it was possibleto clearly identify a person based on his/her reaction to different groups of fancy patterns. More details onthis topic will be discussed in last chapter.


Designing for Emergence in Continuous State Generalized Cellular Automata

In this section our choice is for a continuous state space. Using a simpler cellular model;namely, an uncoupled generalized cellular automata, we show that under a proper choice of thegene parameters autopoietic patterns of organization emerge resembling the behaviordescribed as [McMullin & Varela, 1997]: “..a morphology becomes established which isapparently particularly robust, persisting in each case for approximately 1000 time steps of themodel”. The results presented in this section exploit the analog state space of generalized cellularautomata (GCA) via cell mutations of the piece-wise linear cells designed to implement the"Game of Life" (see Section 5.1). Using a metaphor we may call these experiments "analogmutations in life", although similar results can be obtained using different seed cell functions.The results show an interesting self-organization phenomena; namely, the emergence ofpatterns having the organizational features reminiscent of unicellular organisms. Such patternshave a "growth" stage, when certain membranes emerge, isolating different chaotic modes.This stage is followed by a "maturity" phase, when the membranes evolve without majorchanges and, finally, by an "aging" and “dying” stage when both the membranes and thepattern organization vanish.


The framework for the Continuous State Uncoupled GCA

124


The „seed“ cell

Mutation

125

An example of analog mutation


Growth

Maturity

Aging and death

Emergent dynamic behaviors - reminiscent of simple life forms (simple initial condition – one point in the middle)

Note the emergence ofcertain „self-making“ (orautopoietic) membraneswith a limited life-time


Emergent dynamic behaviors - reminiscent of simple life forms (another initial condition – two dots and one line in the middle)

Growth

Maturity

Aging and death

Note the similarity withthe growth of a complexorgan starting from asimple skeleton


Emergent dynamic behaviors - reminiscent of simple life forms

Effects of mutating the first parameter from 0.9 to 0.91, the restis like in the previous experiment. Now the cell is:

Note the sensitivity toparameters. Although thesame phases areobserved (growth,maturity, death) thepicture changed quitedramatically. Moreexperiments show thatthere is a quite narrowvariation domain for eachparameter such that thebehavior remainsunchanged.


Designing for Emergence in Coupled Generalized Cellular Automata

In addition to the case of uncoupled generalized cellular automata, which is the main focus ofthe previous section, we will consider next an example of a mutation in the "Game of Life" witha coupled CNN cell. This class of generalized cellular automata (GCA) is potentially moreinteresting than “classic” cellular automata. In [Chua, 1998] it was proved that while uncoupledgeneralized cellular automata are equivalent with binary or continuous state cellular automata,the computational properties of coupled GCAs expand beyond that of classic cellular automata.From a practical perspective, a coupled GCA can be implemented via the Cellular NeuralNetworks – Universal Machine [Roska & Chua, 1993], a concept which has already several chiprealizations reported [Roska & Vázquez, 2000a].As shown at the end of the section, emergent computation within a coupled GCA cell can beapplied in image processing applications. In our example, the non-trivial task of reconstructingand detecting patterns from a very noisy environment is considered. As shown, a GCA withproperly tuned parameters is capable of detecting and reconstructing patterns from a highlycorrupted input image where the noise level is 10 times stronger than the uncorrupted signal.


The framework for the Coupled GCA

previously

previously

previous slides


Emergent dynamic behaviors with applications in detection of weak signals embedded in noise


Detailed dynamic evolution


In the absence of a corresponding analytic theory of local activity for the generalized cellularautomata (GCA) with discrete time dynamics, we introduced in this chapter an empirical scalar functionof time m(t), called the cellular disorder measure, for measuring the non-homogeneity of patterns. Acompletely homogeneous pattern corresponds to m(t)=0 . Therefore, a passive-like behavior wasdefined, analogously to the local activity theory, by a rapid decrease of m(t) to the steady state value 0.Consequently, any other form of time evolution of m(t) starting from a non-zero value will correspond toan active GCA behavior, where the word "active" is used in the same sense as in the local activitytheory. Within this "activity" domain, we can further differentiate between an "unstable-like" behavior(corresponding to an increase in m(t) with time) and an "edge of chaos"-like behavior (corresponding toa decrease in m(t) with time).

By mutations of cell parameters (in the sense of crossing failure boundaries), a wide range ofdynamic behaviors was observed, yielding many interesting properties. Although the "Game of Life"function was chosen here as the seed function, any other piecewise-linear cell can be similarlyexplored. When the gene parameters were chosen as an analog mutation of the "Game of Life", the paradigm ofgeneralized cellular automata, particularly when the recurrent coupling is considered among the CNNcells, was found promising for engineering applications. A difficult problem in vision; namely, therecovery of meaningful patterns representing objects hidden in highly perturbed images was shown tohave a simple and effective solution using a heuristically tuned generalized cellular automata.

Concluding Remarks


5. Brain Signatures:A Novel Idea for BiometricAuthentication Based onPerceptual Resonance

Between Emergent Patterns in CellularComputing Systems and Humans


Outline

1. The “resonance” method and its motivation2. The stimuli 3. Experimental results4. Generating Stimuli (Design for Emergence)4. Conclusions

Main goal: Propose a new simple and robust approach for biometric authenticationwhere sequences of patterns generated by recurrent CNNs and Generalized CellularAutomata determine person-specific sequences of responses. Using enough stimuli itis proved that each person could be identified based on this signature.


Basic Ideas of the “Resonance” Method

Similar yet different patterns

Brain signatures

Perceptual resonance

Such patterns are usually the result of a biological process. For example, take a poolof human faces. Psychological experiments reveal that some of the faces areperceived as “beautiful” while others perceived as “not-beautiful”. The same standsfor other such patterns (e.g. flowers, animals and so on).CNNs operated in or near the “edge of chaos” may also be used to generatesequences of such patterns. Moreover, sounds can produce similar effects.

DislikeLike Dislike DislikeLikeANN:

MARY: LikeLikeLike DislikeDislike


Such Stimuli Patterns Can Be Generated:

By nature: Sophisticated hardwareis needed to store and display the stimuli

Using CellularSystems

Simple hardware can generate a widepalette of stimuli.Video output on chip(e.g. TFT) will add a plusof compactness


The “brain signature” method versus existent authentication methods

Face recognition- Complex classification software- Require huge computational resources- May fail if the person uses simple“make-up”techniques or if the person suffers aface injury

The use of passwords- Require memorizing the passwords- People prefer names and words,therefore breaking a passwordit is actually much easier than predictedby the theory

The “resonance” method•Is simple and can be easilyimplemented in portable systems•Can use simple classifiers•Is not sensible to “make-up” and otherevasion methods•Does not require memorizing passwords


Experimental Setup

{ }Nk PPPPH .,,..,, 21=The set of N persons

The set of M stimuli

{ }Mm SSSSS ,..,.., 21=

The space of brain signatures

Jim

Bill

Ann

Ben

{ } MmQqjqkk rR ,...,1;,..,1, ===

{ }Qq ,..,1∈ Presentationsessions


Generating Stimuli using Generalized Cellular Automata (GCA)

For ∞= ,..2,1t

( )Ttytytu nijijij ∆−=−= )1()(

( ) ( )lkij ugenety ,= ,

End

GCA dynamics

InputsOutputs

Nonlinear parametric function(e.g. piecewise-linear)implementing the CNN cell

=∈Nklklu

11,,11,1

1,,1,

1,1,11,1

+++−+

+−

+−−−−

jijiji

jijiji

jijiji

uuu

uuu

uuu

123

456

789

uuu

uuu

uuu≡

C10: [ ] ----.5-----2.1--4.5 98765432 uuuuuuuuusigny 1=

Cell neighborhood

Cell gene


Sets of stimuli used in experiments

Gene: C11 Sample: 62 Gene: C10 Sample: 253

S1 S2

Gene: C10 Sample: 89Gene: C10 Sample: 48

S3 S4

Gene: C12 Sample: 142

Gene: C12 Sample: 202

S5

S6

Gene: C13 Sample: 51 Gene: C3 Sample: 176 Gene: C3 Sample: 216Gene: C12 Sample: 24

[ ] -1.5-0.83-1.25 2864 uuuusigny +++=C3:

S7 S8 S9S10

C10:

C11: [ ] ----.5-----2.1--2.5 98765432 uuuuuuuuusigny 1=

[ ] ----.5-----2.1--4.5 98765432 uuuuuuuuusigny 1=

C12: [ ] ( )9876543212823 8- , 6-29-10 uuuuuuuuusigny 1 +++++++=−+++= −σσσσ

C13:

[ ] ( )9876543212833 8- , 67-10 uuuuuuuuusigny 1 +++++++=−−++= σσσσ

141


Experimental Results

There were threepresentation sessions.

In each session, foreach subject there were 2presentations of the entireset of stimuli (10presented above andother similar 10)

A column in the above graph represents a “brain signature” vectori.e. the reaction (like, dislike or neutral) of the subject (its label is plotted belowits corresponding column) to the sequence of stimuli.

Training set Test set


Testing the resonance hypothesis - the case of 20 stimuli

( ) ( ) 212

max

11

max

12121, kk rr q

Q

q

Q

q

T

qkkMa ⋅= ∑ ∑= =

1. The correlation index between two users k1 and k2 :

2. A dispersion matrix Di calculated for the vector: ( ){ }max,..,12,121

21

Qqqq

T

q =kk rr

Resonance hypothesis: The correlation index between the same user in different presentation sessions is always larger than the (cross) corellationindex between different users in different presentation sessions. In other wordsthe diagonal of the correlation matrix Ma is “above” the rest of the elements

All 20 stimuli

Ma Di

=

29.43 11.40 19.20 8.20 11.93

11.40 67.20 23.20 10.92 17.07

19.20 23.20 33.87 11.18 4.00

8.20 10.92 11.18 11.58 1.07

11.93 17.07 4.00 1.07 4.27

Da

=

10.31 0.25 2.50 1.25 3.75-

0.25 7.00 3.50 3.88 2.00

2.50 3.50 13.50 5.13 2.50

1.25 3.88 5.13 15.38 5.00

3.75- 2.00 2.50 5.00 18.00

Ma

The lowest auto-correlation (7) obtainedfor user “4” is larger than the highest cross-correlation (5.13) between “1” and “2”

4max =Q

143


Testing the resonance hypothesis - the case of 10 stimuli

The first 10 stimuliMa Di

=

9.25 0 1.50- 0.75 2.50-

0 6.00 1.00 0 2.00

1.50- 1.00 9.00 5.50 3.00

0.75 0 5.50 9.25 4.50

2.50- 2.00 3.00 4.50 10.00

Ma

The lowest auto-correlation (6) obtained for user “4” islarger than the highest cross-correlation (5.5) between “1”and “2” but now there is a very narrow band of only 0.5between them. Reducing thenumber of stimuli to 9 results in a violation of the aboveindicating that the signature space becomes too small.


-0.5

-0.4

-0.3

-0.2

-0.1

0

+0.1

+0.2

+0.3

+0.4

+0.5

2λ

1λ-0.5 -0.4 -0.3 -0.2 -0.1 0 +0.1 +0.2 +0.3 +0.4 +0.5

“Class P”

“Class E”

“Class U”

Legend

Long transients,complex behaviors

Convergence tonon-homogeneouspatterns

Very long transients Long transients

Short transients

Short transients which convergeto quasi-static patterns

114

12

17

32

37

45

5

55

61

23

122

( )( ) ( )( )23892146715 115.25.05.1 uuuuuuuu-uw +++++++++++= λλ

)(wsigny =cell

Designing for Emergence - Mutations in The Game of Life(the binary case)

Conway’sGame of Life


Examples of Emergent Snowflake-like Patterns From The “Green” Domain

12

12

61

61

61

61


Designing for Emergence - Mutations in The Game of Life(the continuous case)

)(5.0 98764321 uuuuuuuuu +++++++⋅=++++σ

( ) σσσ u.u.u.-u.u..w 5060302303090 55 +++++=++++++++

uMutation 1


Conclusions

Preliminary results on a limited group of users indicate optimistic results. - Still much research has to be done using larger groups- Other line of investigation includes the choice of stimuli patterns and therelationship that may exists between the expression of the GCA gene and thediscrimination properties of the above system.

Compared to other existent methods, it offers important advantages and thepromise of a very good accuracy at a reasonably low implementation cost andpower consumption. These features makes it a good candidatefor use in portable equipment. CNN-UM with video output !

It is a completely new biometric authentication method allowing one to build easily abiometric systems using already existent CNN-UM chips or dedicated ones. A simple,dedicated CNN chip with an LCD-display on it can be designed and a simple controlscheme would allow to program the gene, the initial condition, and the sequence ofdisplayed patterns


REFERENCES













Documents

Computational Emergence in Cellular Computing Systemsatm.neuro.pub.ro/radu_d/html/bn2007_2008/emergence_reason_2.pdfsystems is evolutionary, the emergence of new functions being sometimes