THE CNN PARADIGM: SHAPES AND COMPLEXITY

August 2, 2005 9:44 01330

Tutorials and Reviews

International Journal of Bifurcation and Chaos, Vol. 15, No. 7 (2005) 2063–2090c© World Scientific Publishing Company

THE CNN PARADIGM: SHAPES AND COMPLEXITY

PAOLO ARENA, MAIDE BUCOLO, STEFANO FAZZINO,LUIGI FORTUNA∗ and MATTIA FRASCA

Dipartimento di Ingegneria Elettrica Elettronica e dei Sistemi,Universita degli Studi di Catania, viale A. Doria 6, 95123 Catania, Italy

∗[email protected]

Received October 8, 2004; Revised November 12, 2004

The paper stresses the universal role that Cellular Nonlinear Networks (CNNs) are assumingtoday. It is shown that the dynamical behavior of 3D CNN-based models allows us to approachnew emerging problems, to open new research frontiers as the generation of new geometricalforms and to establish some links between art, neuroscience and dynamical systems.

Keywords : CNN; shape; complexity; art; neuroscience.

1. Introduction

Shapes are the fundamental attribute of visibleobjects; they give us the perception of structure.They also represent the alphabet of art [Arnheim,1974]. Shapes represent the object of visible beauty.It is well known that the mathematics of shapeand space is geometry. Which is the relationshipbetween dynamical systems and shapes? Couldthe mathematics of nonlinear dynamics help us toestablish an innovative way to go in depth into theworld of shape generation?

The human mind process of creating shapes isof course a dynamical process and often its complex-ity involves more steps in combining emergent con-ditions. The shape generation in the human mindleads often to the emergence status of mental statesand the brain-mind complexity is related also to itsshape organization process.

Some artistic experiences and painters of thelast century are taken as examples of the previ-ous concepts. The painter Joan Miro reached thethird dimension in his “Still Life with Old Shoe”[Lynton, 1994] that represents a bifurcation pointin the production of the artist whose previouspaintings were characterized by subjects spatiallyrepresented in two dimensions. This fact indicatesin the artist’s mind a possible rearrangement of his

shape archetypes. Moreover, in the Still Life withOld Shoe the idea of broken forms typical of theperiod of analytical Cubism appears. The brokenforms, like bursts, are evident in the Salvator Dalıart and in particular in Tete Raphaelesque Eclatee.The dynamical trends of Giorgio De Chirico’s artis-tic life are well known: he founded with Carlo Carrathe Methaphysical Painting. Surrealists honored hisearly paintings, but in the Twenties he switched toa Renaissance-based Classicism. He completed hisartistic experience in NeoMethaphisical Painting.

Both the artist’s evolution trends and the pro-cess of shapes creation are emergent phenomena.Viewing the art in each period, accurate analysisallows us to identify the complexity of beauty andthe perception of the harmony of created shapes,that are the signatures of the dynamical process ofthe artist in his creative efforts. The evolutionarytrends of figurative arts further remark the finger-prints of complexity in the shape history. Moreover,if we try to establish when art arose, no definitiveanswer does exist. Art itself possibly arose beforethe birth of history. The examples of primitiveart like the aboriginal cave paintings in Australia,representing hunt scenes, animals and even moreabstract figures such as the half-human, half-ani-mals figures called “therianthropes”, witness what

2063

August 2, 2005 9:44 01330

2064 P. Arena et al.

primitive men tried to reproduce from nature. Therole of this representation is vague possibly relatedto religion, magic and so on. When did rich abstractshapes appear in visual arts? Many, many doubtsare related to this topic.

Aesthetics have recently presented more ideasin order to evaluate the beauty of art, and pro-vide a criteria in order to understand the role ofart in history. New shapes genesis is essentially dueto an emergent mechanism in the human mind.One of the tasks of this paper is to prove experi-mentally that shapes are also emergent phenomenain chaotic spatially extended systems. Moreover,emergent phenomena that occur in visual cortexcould also be related to the previous items. Infact, many researches on this subject focused onthis topic [Ermentrout et al., 1979; Dalhem et al.,2000]. The hallucination phenomena arising in ill-ness like migraine lead to a generation of new formsknown as phosphenes or scintillating scotoma, for-tifications and vortices. New forms and structuresemerge in the visual cortex without external inputs.Possibly these unpredictable events influenced pos-itively some paints of famous artists like Giorgio DeChirico or Vincent Van Gogh [Podoll et al., 2001;Bogousslavsky, 2003].

Furthermore, a very appealing approach to facethe visual cortex behavior has been introducedin [Zeki, 1999], where the neuroscientist discussesthe understanding of both the single visual cor-tex areas and its global organization, by analyz-ing various artists’ paints. In this last remark, thetwofold link between art and neuroscience appears.In the actual literature these aspects are reinforcedby recent studies on surprising emergent creativetendencies shown in patients with strong mentaldiseases [Giles, 2004]. Shapes are the links in thisimportant route.

In this paper, the universality of the CNNparadigm [Chua, 1998] is proved to be the appro-priate tool to generate emergent shape patterns,to relate shape evolution with spatially extendeddynamical systems and therefore to open a realbridge among circuits, art and neuroscience. Theessential prerogative of the CNN paradigm is totake advantage of the cooperative behavior of sim-ple dynamical nonlinear circuits in order to obtaincomplex global tasks. The reported study is relatedto three-dimensional, in the space, CNN (3D-CNN).Merging cells to achieve E-merging patterns anddynamics and to emulate complex system behav-iors fully expresses the 3D-CNN role in the proposed

study. In fact, from the coupling of the single 3D-CNN units impressive patterns and shapes emerge.It is shown that the evolution of the 3D-CNNdynamics generates harmonious yet unpredictableshapes. It is well known that a partial differentialequation can be matched in a CNN-based algorithm[Chua, 1998] and that RD-CNNs are appropri-ate to reproduce complex phenomena in biology,chemistry, neurodynamics: the generalized 3D-CNNscheme proposed in this work adds further results inorder to highlight the power of the CNN paradigm.

The paper is organized as follows. The defi-nition of the 3D-CNN architecture is discussed inSec. 2. The mathematical details of the 3D-CNNconfigurations are reported in Sec. 3 where a galleryof obtained shapes and patterns is also reported.In Sec. 4 a discussion of the results shown in theprevious section is included. In the conclusion, thetechnological revolution of the CNN paradigm isemphasized. The appendix includes the technicaldetails of the Eˆ3 software tool, developed to per-form the reported experiments.

2. 3D-CNN Model

Let us consider the universal definition of a CNN[Chua, 1998]: A CNN is any spatial arrangement oflocally coupled cells where each cell is a dynamicalsystem which has inputs, and outputs and a stateevolving in accordance with the described dynami-cal laws.

The following general concepts will beassumed:

• the coupling laws are not generally spatiallyinvariant (even if often for practical realizationsthese are assumed invariant);

• the concept of dominant local coupling is assu-med; therefore most of the connections are be-tween cells in a neighborhood of unitary radius,but some nonlocal connections could also beincluded;

• each cell is a dynamical system with assignedstate variables.

Let us consider an isolated cell (from themicroscale CNN point of view). The followingvariables characterize the cell:

• the exogenous, controllable input vector ui,j,k(t)∈ R

mu ;• the exogenous, uncontrollable input vector

Si,j,k(t) ∈ Rms ;

• the state vector xi,j,k(t) ∈ Rn;

https://www.researchgate.net/publication/8478225_Neuroscience_Change_of_mind?el=1_x_8&enrichId=rgreq-22a8c6fb-b1f8-4397-90ad-5ecb986b3f95&enrichSource=Y292ZXJQYWdlOzIyMDI2NDcyNDtBUzoxMDM3Njg1MjM4MDQ2OTFAMTQwMTc1MTc1MzAxMQ==

https://www.researchgate.net/publication/264954922_CNN_A_paradigm_for_complexity?el=1_x_8&enrichId=rgreq-22a8c6fb-b1f8-4397-90ad-5ecb986b3f95&enrichSource=Y292ZXJQYWdlOzIyMDI2NDcyNDtBUzoxMDM3Njg1MjM4MDQ2OTFAMTQwMTc1MTc1MzAxMQ==


August 2, 2005 9:44 01330

The CNN Paradigm 2065

• the output vector yi,j,k(t) ∈ Rmy ;

• the bias vector zi,j,k(t) ∈ Rn that is generally

assumed controllable;

where i, j, k represent the space coordinates.Moreover, let us introduce some useful nota-

tions. Let us indicate with X, U, Y the whole state,input and output sets (referring to all the cells ofthe CNN). Let us define the neighborhood of thecell C(i, j, k) as

Nr(i, j, k)= C(α, β, γ)|max(|α − i|, |β − j|, |γ − k|) ≤ r

and let us indicate with

Xi,j,k =⋃

C(α,β,γ)∈Nr(i,j,k)

xα,β,γ

the state variables of cells in the neighborhoodNr(i, j, k) of the cell C(i, j, k). Analogous definitionscan be given for Ui,j,k and Yi,j,k.

The general CNN model that we introduce isbuilt-up by adding complexity to the simplest CNNand takes into account the following key points:

• the cells are not required to be equal to eachother;

• in a 3D grid the coupling laws are locallydescribed along with the neighbor cells Sα,β,γ ;

• each node, as described below, can be realizedusing n generalized first-order cells constitutinga small multilayer CNN architecture.

More in detail, referring to the considerationsabove, we could give definitions of CNN structuresat several levels, starting from the microscale levelto the mesoscale and macroscale levels.

Definition: Microscale CNN. A CNN obtainedby connecting first-order cells may implement anynonlinear dynamics [Fortuna et al., 2003]. Themicroscale CNN is introduced to account for thisconcept. It can be described by the following equa-tions:

x1 = −x1 + z1 +∑

C(l)∈Nr(1)

A(1; l)yl

+∑

C(l)∈Nr(1)

B(1; l)ul +∑

C(l)∈Nr(1)

C(1; l)xl

x2 = −x2 + z2 +∑

C(l)∈Nr(2)

A(2; l)yl

+∑

C(l)∈Nr(2)

B(2; l)ul +∑

C(l)∈Nr(2)

C(2; l)xl

...

xn = −xn + zn +∑

C(l)∈Nr(n)

A(n; l)yl

+∑

C(l)∈Nr(n)

B(n; l)ul +∑

C(l)∈Nr(n)

C(n; l)xl

y1 = 0.5(|x1 + 1| − |x1 − 1|...

yn = 0.5(|xn + 1| − |xn − 1|(1)

For instance, a CNN made of three first-order cells(n = 3) may implement the dynamics of the Chua’scircuit [Arena et al., 1995] or a Colpitts-like oscilla-tor [Arena et al., 1996]. A schematic representationof a microscale CNN is shown in Fig. 1, where eachfirst-order cell is represented by a small cube andthe possibility of connecting such a CNN with otherCNNs is sketched.

This model can be further generalized asfollows:

x1,ijk = f1;i,j,k(xijk,yijk,uijk)

x2,ijk = f2;i,j,k(xijk,yijk,uijk)...

xn,ijk = fn;i,j,k(xijk,yijk,uijk)

y1,ijk = g1;i,j,k(xijk,yijk,uijk)...

yn,ijk = gn;i,j,k(xijk,yijk,uijk)

(2)

where we make explicit the connections with otherCNNs through the space coordinates i, j, k.

Definition: Mesoscale CNN. By connecting sev-eral microscale CNNs (2) in a 3D grid, a mesoscale

Fig. 1. Schematic representation of a microscale CNN.

https://www.researchgate.net/publication/3322808_How_state_controlled_CNN_cells_generate_the_dynamics_of_the_Colpitts-like_oscillator?el=1_x_8&enrichId=rgreq-22a8c6fb-b1f8-4397-90ad-5ecb986b3f95&enrichSource=Y292ZXJQYWdlOzIyMDI2NDcyNDtBUzoxMDM3Njg1MjM4MDQ2OTFAMTQwMTc1MTc1MzAxMQ==

https://www.researchgate.net/publication/3322567_Chua's_circuit_can_be_generated_by_CNN_cells?el=1_x_8&enrichId=rgreq-22a8c6fb-b1f8-4397-90ad-5ecb986b3f95&enrichSource=Y292ZXJQYWdlOzIyMDI2NDcyNDtBUzoxMDM3Njg1MjM4MDQ2OTFAMTQwMTc1MTc1MzAxMQ==

https://www.researchgate.net/publication/220264565_Modeling_complex_dynamics_via_extended_PWL-based_CNNs?el=1_x_8&enrichId=rgreq-22a8c6fb-b1f8-4397-90ad-5ecb986b3f95&enrichSource=Y292ZXJQYWdlOzIyMDI2NDcyNDtBUzoxMDM3Njg1MjM4MDQ2OTFAMTQwMTc1MTc1MzAxMQ==

August 2, 2005 9:44 01330


CNN can be obtained. A microscale CNN is now acell of a mesoscale CNN. The cell equations for amesoscale CNN are defined as follows:

xijk = fi,j,k(xijk,yijk,uijk) + ai,j,k(Yijk)

+ bi,j,k(Uijk) + ci,j,k(Xijk)

yijk = gi,j,k(xijk,yijk,uijk)

(3)

where ai,j,k(Yijk), bi,j,k(Uijk), ci,j,k(Xijk) representthe coupling terms. It can be noticed that connec-tions among cells are only local. A schematic viewof a mesoscale CNN is shown in Fig. 2.

Definition: Macroscale CNN. We now add to thisstructure the possibility of having long-range con-nections. By including in Eq. (3) these terms, themacroscale model of a CNN architecture assumesthe following form:

xijk = fi,j,k(xijk,yijk,uijk) + ai,j,k(Yijk)

+ bi,j,k(Uijk) + ci,j,k(Xijk)

+ alri,j,k(Y) + blr

i,j,k(U) + clri,j,k(X)

yijk = gi,j,k(xijk,yijk,uijk)

(4)

Fig. 2. Schematic representation of the generalized mesoscale model of 3D-CNN.

Fig. 3. Schematic representation of the generalized macroscale model of 3D-CNN.

August 2, 2005 9:44 01330


where ai,j,k(Yijk), bi,j,k(Uijk), ci,j,k(Xijk) representlocal coupling terms, while alr

i,j,k(Yijk), blri,j,k(Uijk),

clri,j,k(Xijk) take into account long-range coupling

terms.The overall model described by Eqs. (4) is

schematized in Fig. 3, which represents a schematicview of such a CNN, in which local and long-rangeconnections are allowed. Moreover, the space depen-dency of each cell also includes the uncertainties inthe model of each cell. These uncertainties can beeither parametric or structural.

This model can then be further generalized andrepresented by the following state equation:

xijk = fi,j,k(X,Y,U)

yijk = gi,j,k(X,Y,U).(5)

The overall model (macroscale CNN) schema-tized in Fig. 3 is built-up by adding complexity tothe simplest CNN (the microscale CNN) throughthe mesoscale CNN in which connections are onlylocal.

Example: 3D-CNN with first-order cells withstandard nonlinearity. A macroscale 3D-CNNwith first-order cells with standard nonlinearity isdescribed by the following equations:

xijk = −xijk + zijk

+∑


A(i, j, k;α, β, γ)yα,β,γ

+∑


B(i, j, k;α, β, γ)uα,β,γ

+∑


C(i, j, k;α, β, γ)xα,β,γ

+ Alri,j,k;α,β,γ(yα,β,γ) + Blr

i,j,k;α,β,γ(uα,β,γ)

+ C lri,j,k;α,β,γ(xα,β,γ)

yijk = 0.5(|xijk + 1| − |xijk − 1|)(6)

Fig. 4. Eladio Dieste, Church of Atlantida, Atlantida, Uruguay, 1958.

August 2, 2005 9:44 01330


where A(i, j, k;α, β, γ), B(i, j, k;α, β, γ) andC(i, j, k;α, β, γ) are the feedback template, the con-trol template and the state template; Alr

i,j,k;α,β,γ,Blr

i,j,k;α,β,γ and C lri,j,k;α,β,γ describe the map of the

long-range connections (they are N × N × Nmatrices). Equations (6) match equations (4) ifthe following assumptions on the coupling termshold:

alri,j,k(Yijk) =

∑C(α,β,γ)∈Nr(i,j,k)

Alr(i, j, k;α, β, γ)yα,β,γ

ai,j,k(Yijk) =∑


A(i, j, k;α, β, γ)yα,β,γ

blri,j,k(Yijk) =


Blr(i, j, k;α, β, γ)yα,β,γ

bi,j,k(Yijk) =∑


B(i, j, k;α, β, γ)yα,β,γ

clri,j,k(Yijk) =


C lr(i, j, k;α, β, γ)yα,β,γ

ci,j,k(Yijk) =∑


C(i, j, k;α, β, γ)yα,β,γ

Remark. The conceived structure, showing emerg-ing pattern behaviors by using cells, indicating apath from simplicity to achieve complexity, is anappealing research subject in several fields. Letus consider the well-known architect Eladio Dieste[Morales, 1991]; he used the brick as the funda-mental element to build his main structures in civilengineering as shown, for example, in Fig. 4. TheChua’s cell [Chua & Yang, 1988a, 1988b; Fortunaet al., 2003] represents the basic electronic elementto design complex circuits exactly like the EladioDieste’s brick. Moreover as the architect Frei Otto[Otto, 1982] considered emerging forms in nature asbuilding paradigms to be realized in civil structures,in a dual manner, the search for complex patternsand their circuit realization by means of electronicstructures find the link in the CNN architecture.

3. Emergence of Forms in 3D-CNNs

Looking at a picture and in general at an artisticrepresentation, a great number of impressions arereceived, all at the same time. Let us consider apainting: a lot of elements appear, the impressionsthat are received combining the various elementsgive the perception of the specific characteristic of

the painting. The unifying principle of the differentelements is the shape that expresses the whole: theshape is an emergent property of the whole.

In the 3D-CNN paradigm the single contribu-tions from the various cells lead us to the globalcharacteristics of the architecture as regards boththe structure and the emergent dynamical behavior.There is a dichotomy in this view between CNNsand shapes. The idea that the dynamics of a 3D-CNN is strongly evolving is assumed. Moreover,we cannot directly perceive the internal dynamicalevolution of the single cell or of a group of cellsthat leads us to understand the whole emergency ofthe CNN behavior, that could only be synthetizedinto a shape. The internal global dynamics couldbe only perceived by coupling to the 3D-CNN thevision of the shapes. A dichotomy does exist: theshape generation capabilities of CNNs are empha-sized, moreover, the shape as partial abstractionof the emergent property in a distributed nonlin-ear dynamical system does appear. These conceptsare the leading points of the next part of this sec-tion where in detail both the configurations and thevisual representations of the generalized 3D-CNNparadigm are included.

The model used in the following experimentsis related to a very simplified coupling law, wherea simple space-constant diffusion law in Eqs. (2) isassumed as follows:

ci,j,k(Xijk) = D∇2ijkx (7)

where the discretized Laplacian in a 3D space isdefined by the following relationship:

∇2ijkx = xi−1,jk + xi+1,jk + xi,j−1,k

+xi,j+1,k + xij,k−1 + xij,k+1 − 6xijk. (8)

Moreover, all the cells are equal, i.e. fi,j,k(xijk,uijk) = f(xijk,uijk).

With these assumptions Eqs. (2) can thereforebe rewritten as follows:

xijk = f(xijk) + D(xi−1,j,k + xi+1,j,k + xi,j−1,k

+xi,j+1,k + xi,j,k−1 + xi,j,k+1 − 6xijk)

which matches the well-known paradigm ofreaction–diffusion equations:

x = f(x) + D∇2x.

The experiments discussed below have beenperformed considering different dynamical laws foreach cell of the CNN. In most of the cases, the

August 2, 2005 9:44 01330


behavior of each cell is chaotic. The coupling lawis characterized by low coupling coefficient (weakdiffusion). In all the experiments, zero-flux bound-ary conditions have been chosen.

The experiments will be detailed in the fol-lowing. In all of them the evolution of the sys-tem leads to the emergence of self-organized forms.The rich, unpredictable, beautiful dynamics of theglobal forms arising in the various experiments isrelated to the model used for the cell of the 3D sys-tem. The chaotic behavior of each cell, representedby the beauty of the strange attractor, is reflectedin the beauty of the 3D forms like those shown inthe gallery of reported models.

The evolutionary forms reported in the variousgraphs are isosurfaces that have been obtained intime in the 3D space defined by the spatial coordi-nates i, j, k.

As concerns the cell dynamics f(xijk), differ-ent chaotic laws have been simulated. In particular,the Lorenz system, the Rossler system [Strogatz,2000] and the Chua’s circuit [Chua, 1998] have beeninvestigated. Moreover, the chaotic dynamics of sev-eral neuron models have been taken into account inorder to emulate the global behavior of neural net-works in a 3D space.

3.1. Initial conditions

Different initial conditions for the various experi-ments have been chosen and will be discussed indetail in the following. However, the idea underly-ing this choice is common to all the examples andwill be briefly introduced in this section.

Initial conditions x0(i, j, k) have been createdstarting in some topological form. In particular,they can be viewed as the composition of two func-tions σ and γ, describing the topological form takeninto account.

Let us focus on a 3D-CNN made of third-ordernonlinear units (as in most cases investigated in thispaper) and let us indicate by i, j, k the three coor-dinates of the 3D space and i =

√−1.Thus, the two functions σ and γ can be defined

as follows:

γ : R3 ⊇ Ω → C

σ : C → R3

and initial conditions are given by the followingrelation:

x0(i, j, k) = σ γ.

The definition of the function γ makes use ofcomplex curves used for the study of topologicalforms. For instance, a (m : n) torus knot can berepresented by the following equations

Z = γ(i, j, k)

=(

d2 − 11 + d2

+ i2k

1 + d2

)m

−(

2i1 + d2

+ i2j

1 + d2

)n

with d =√

i2 + j2 + k2. Then, initial conditions forthe state variables x, y, z of each nonlinear unit ofthe 3D-CNN are created from Eqs. (9) through therelations:

x0(i, j, k) = Ax Re(Z) + Bx

y0(i, j, k) = Ay Im(Z) + By

z0(i, j, k) = Az Re(Z) + Bz

where Ax, Ay, Az, Bx, By and Bz are real con-stants used to scale the initial conditions to matchthe dynamic range of the nonlinear units constitut-ing the 3D-CNN.

3.2. 3D waves in homogeneous andunhomogeneous media

First of all, let us consider a 3D-CNN, whereeach cell is a second-order nonlinear system, imple-menting a reaction–diffusion. Two examples arediscussed. The first deals with an homogeneousmedium, the second is an example of unhomoge-neous medium.

The equations of the generic second-order cellCijk are the following:

xi,j,k;1 = k(−xi,j;1 + (1 + µ + ε)yi,j;1 − syi,j;2

+ i1 + D1(yi−1,j,k;1 + yi+1,j,k;1

+ yi,j−1,k;1 + yi,j+1,k;1 + yi,j,k−1;1

+ yi,j,k−1;1 − 6yi,j,k;2))

xi,j,k;2 = k(−xi,j;2 + (1 + µ − ε)yi,j;2 + syi,j;1

+ i2 + D2(yi−1,j,k;2 + yi+1,j,k;2

+ yi,j−1,k;2 + yi,j+1,k;2 + yi,j,k−1;2

+ yi,j,k−1;2 − 6yi,j,k;2)).

(9)

The parameters of the CNN cell have been cho-sen according to µ = 0.7, s = 1, i1 = −0.3, i2 = 0.3,


August 2, 2005 9:44 01330


D1 = 0.1, D2 = 0.1. For these values, the 3D-CNNbehaves as a nonlinear medium in which autowavespropagate. The parameter k accounts for the possi-ble unhomogeneity of the medium.

In the first example, all the cells in the 40 ×40 × 40 3D-CNN have the same values of k(k = 1). In the center of the 3D-CNN, there is a“pacemaker” cell whose outputs are fixed to thevalues y20,20,20;1 = 1 and y20,20,20;2 = −1. This cellelicits the generation of autowaves in the neighbor-ing cells. Figure 5 shows the behavior of a 3D-CNNmade of cells represented by Eqs. (9). As it can benoticed, when two wavefronts collide they annihi-late each other.

In the second example, the cells of the 3D-CNN have different values of the parameter k. Thevalue k = 0.6 characterizes “slow” cells, while thevalue k = 1 characterizes “fast” cells as schemat-ically shown in Fig. 6(a). The whole CNN con-sists of 41 × 41 × 41 cells. Simulation results areshown in Fig. 6. Figure 6(a) shows the initial con-figuration: we simulated an initial point of excita-tion (indicated by an arrow) and a “wall” in anunhomogeneous 3D medium. Figures 6(b)–6(d) rep-resent the evolution of the RD-CNN. The presenceof unhomogeneity in the medium clearly leads to the

emergence of spiral waves. This experiment repre-sents a fascinating emergent behavior shown by acomplex system.

3.3. Chua’s circuit

This example deals with the Chua’s circuit [Chua,1998]. In the case of a 3D-CNN made of Chua’s cir-cuits, Eqs. (9) can be rewritten as follows:

xijk = α(yijk − h(xijk)) + D∇2ijkx

yijk = xijk − yijk + zijk (10)

zijk = βyijk

where

h(x) = 0.5((s1 + s2)x + (s0 − s1)(|x − B1| − |B1|)+ (s2 − s0)(|x − B2| − |B2|)) + ε

and the diffusion term only acts on the first statevariable xijk(t). An array of 80 × 80 × 80 chaoticunits has been considered, i.e. 1 ≤ i, j, k ≤ 80 inEqs. (10). The parameters of each single unit havebeen chosen according to α = 9, β = 30, s0 = −1.7,s1 = s2 = −1/7, ε = −1/14, B1 = −1 and B2 = 1,in order to set the Chua’s circuit in the bistabilityregion. The diffusion coefficient has been fixed to

Fig. 5. Behavior of a 3D-CNN generating autowaves in a homogeneous medium.

August 2, 2005 9:44 01330


(a) (b)

(c) (d)

Fig. 6. Behavior of a 3D-CNN generating spiral waves in an unhomogeneous medium.

the value D = 0.1. In order to visualize the behaviorof the whole 3D-CNN, we considered an isosurfacedefined by xijk = 0.1. The emergent behavior leadsto the formation evolving in time in a nonrepetitive

way. Figure 7 shows some frames of the evolutionof a 3D-CNN made of Chua’s circuits, where theformation of shapes and structures evolving in timeis evident.

Fig. 7. Forms obtained by a 3D-CNN made of Chua’s circuits.

August 2, 2005 9:44 01330


Figure 7 refers to the following initialconditions:

z = γ(i, j, k)

=(

d2 − 11 + d2

+ i2k

1 + d2

)7

−(

2i1 + d2

+ i2j

1 + d2

)5

σ(z) = (4Re z − 1.25, Im z, 4Re z + 1.25).

Moreover, the stretching and folding dynamicsappears in the frames shown in Fig. 8.

3.4. Lorenz system

The example reported in Fig. 9 deals with a CNNmade of 60 × 60 × 60 chaotic Lorenz systems des-cribed by the following equations [Strogatz, 2000]:

xijk = σ(yijk − xijk) + D∇2ijkx

yijk = rxijk − yijk + xijkzijk + D∇2ijky

zijk = xijkyijk − bzijk

(11)

where the parameters have been chosen accordingto σ = 10, r = 28, b = 8/3 in order to set the well-known butterfly attractor. The diffusion coefficienthas been fixed to the value D = 0.5.

Initial conditions have been chosen as follows:

z = γ(i, j, k)

=(

d2 − 11 + d2

+ i2k

1 + d2

)6

−(

2i1 + d2

+ i2j

1 + d2

)9

σ(z) = (4Re z − 1.25, Im z, 4Re z + 1.25)

Figure 9 shows the isosurface defined byxijk = 2.

Even in this case the evolution of the systemleads to ever changing regular forms.

3.5. Rossler system

The emergence of organized forms and structureshas been also observed in a 3D-CNN of Rossler units[Strogatz, 2000] as follows:

xijk = −yijk − zijk + D∇2ijkx

yijk = xijk + ayijk (12)

zijk = b + xijkzijk − czijk

where the diffusion term only acts on the first statevariable xijk(t). An array of 60 × 60 × 60 chaoticunits has been considered. The parameters of eachsingle unit have been chosen according to a = 0.2,b = 0.2, c = 5, in order to set the Rossler chaotic

attractor. The diffusion coefficient has been fixed tothe value D = 0.1. In order to visualize the behaviorof the whole 3D-CNN, we considered a level surfacedefined by xijk = 0. Figure 10 shows some framesof the evolution of a 3D-CNN made of Rossler sys-tems, where the formation of forms and structuresevolving in time is evident.


z = γ(i, j, k)

=(

d2 − 11 + d2

+ i2k

1 + d2

)(2i

1 + d2+ i

2j1 + d2

)

+(

2i1 + d2

+ i2j

1 + d2

)5

−(

d2 − 11 + d2

+ i2k

1 + d2

)3

σ(z) = (4Re z − 0.25, Im z, 4Re z + 0.25).

3.6. FitzHugh–Nagumo neuron model

The first neuron model investigated is theFitzHugh–Nagumo (FHN) model [FitzHugh, 1961;Nagumo et al., 1960] of spiking neurons describedby the following equations:

vijk = εvijk(1 − vijk)(vijk − uijk + b

a

)+ D∇2

ijkv

uijk = vijk − uijk(13)

and the diffusion term only acts on the first variable.The parameters have been chosen according to:

a = 0.75, b = 0.01, ε = 50. An array of 50× 50× 50neurons (13) coupled with a diffusion coefficientD = 1 has been taken into account. Some frames ofthe evolution of the isosurface defined by xijk = 0.5are shown in Fig. 11.

3.7. Hindmarsh–Rose neuron model

A 3D-CNN where the basic cell is the Hindmarsh–Rose model [Rose & Hindmarsh, 1989] of burst-ing neurons is discussed here. The dynamics of thismodel is described by the following equations:

xijk = yijk + ax2ijk − x3

ijk − zijk + I + D∇2ijkx

yijk = 1 − bx2ijk − yijk

zijk = r(S(xijk − xc) − zijk)

(14)

where a diffusion term acting on the first variablehas been included.

The parameters have been chosen according to:a = 3, b = 5, r = 0.0021, S = 4, xc = −1.6, and

August 2, 2005 9:44 01330


Fig. 8. Stretching and folding dynamics in 3D-CNN made of Chua’s circuits.

I = 0 and 30 × 30 × 30 neurons (14) have beencoupled with a diffusion coefficient D = 0.75 lead-ing to very interesting results. Some frames of theevolution of the isosurface defined by xijk = 0.5 areshown in Figs. 12 and 13.

3.8. Inferior-Olive neuron model

This neuron model was proposed in [Giaquintaet al., 2000] to mimic the behavior of Inferior-Olive (IO) neurons. They are characterized by sub-threshold oscillations. The dimensionless equations

August 2, 2005 9:44 01330


Fig. 9. Evolution of the isosurface xi,j,k = 2 generated by a 3D-CNN made of Lorenz systems.

August 2, 2005 9:44 01330


Fig. 10. Forms generated by a 3D-CNN made of Rossler systems.

describing this model are the following:

xijk =xijk(xijk − γ)(1 − xijk) − yijk

ε+ D∇2

ijkx

yijk = −Ωzijk + rijk(A − z2ijk − r2

ijk) + D∇2ijky

zijk = Ωrijk + zijk(A − z2ijk − r2

ijk) + D∇2ijkz

(15)

with rijk = (yijk/M) − x.The parameters have been chosen according to:

ε = 0.01, γ = 0.2, M = 0.5, A = 0.0006, Ω = −1.6.The CNN consists of 40×40×40 neurons (15) cou-pled with a diffusion coefficient D = 0.001. Someframes of the evolution of the isosurface defined byxijk = −0.15 are shown in Fig. 14.


z = γ(i, j, k)

=(

d2 − 11 + d2

+ i2k

1 + d2

)7−2 d2

D2

−(

2i1 + d2

+ i2j

1 + d2

)5+3 d2

D2

σ(z) = (2Re z − 0.4d2

D2, Im z + 0.4,

sin((Re z)2 − (Im z)2)

with d =√

i2 + j2 + k2 and D =√

M2 + N2 + P 2.

3.9. Izhikevich neuron model

A recent neuron model was proposed by Izhikevichin order to conjugate accuracy of the modeland computational resources needed to simulatelarge arrays of neurons [Izhikevich, 2003]. Themodel accounts both for different spiking behav-iors (tonic, phasic and chaotic spiking) and forbursting behavior,dependingon the parameters cho-sen. It can be described by the following equations[Izhikevich, 2003]:

vijk = 0.04v2ijk + 5vijk + 140 − uijk + I + D∇2

ijkv

uijk = a(bvijk − uijk)(16)

with the spike-resetting

if v ≥ 30mV, then

v ← cu ← d

v and u are dimensionless variables, and a = 0.2,b = 2, c = −56, d = −16, and I = −99 are the

August 2, 2005 9:44 01330


Fig. 11. Shapes generated by a 3D-CNN made of FHN neurons.

August 2, 2005 9:44 01330


Fig. 12. Frames of the evolution of a 3D-CNN made of HR neurons.

parameters (chosen to set a chaotic spiking activity[Izhikevich, 2004]). The simulation of the 3D-CNNmade of Izhikevich neurons shown in Fig. 15 hasbeen carried out by considering 30 × 30 × 30 units,D = 0.01 and an isosurface defined by vijk = −65.4.

Initial conditions have been chosen asfollows:

(vijk(0), uijk(0))

=

(−56,−112) if rr < 5 and i > 0

(20, 40) if rr < 9 and i < 0

(0, 0) otherwise

with r =√

i2 + j2 + k2.

August 2, 2005 9:44 01330


Fig. 13. Frames of the evolution of a 3D-CNN made of HR neurons.

3.10. Neuron model exhibitinghomoclinic chaos

Another experiment was carried out by using acell model based on a CO2 laser model. This

model shows Shilnikov chaos and can be also takenas representative of a class of neuron dynamicswith chaotic inter-spike intervals. The followingdimensionless equations describe the behavior ofthis model:

August 2, 2005 9:44 01330


Fig. 14. Frames of the evolution of a 3D-CNN made of IO neurons.

August 2, 2005 9:44 01330


Fig. 15. Shapes generated by a 3D-CNN made of Izhikevich neurons.

August 2, 2005 9:44 01330


x1,ijk = k0x1,ijk(x2,ijk − 1 − k1 sin2(x6,ijk))

+ D∇2ijkx1

x2,ijk = −Γ1x2,ijk − 2k0x1,ijkx2,ijk + γx3,ijk

+ x4,ijk + P0 + D∇2ijkx2

x3,ijk = −Γ1x3,ijk + x5,ijk + γx2,ijk + P0

x4,ijk = −Γ2x4,ijk + γx5,ijk + zx2,ijk + zP0

x5,ijk = −Γ2x5,ijk + zx3,ijk + γx4,ijk + zP0

x6,ijk = −βx6,ijk + βB0 − βRx1,ijk

1 + αx1,ijk

(17)

Parameters have been chosen according to[Pisarchik et al., 2001; Ciofini et al., 1999] as fol-lows: R = 220, k0 = 28.5714, k1 = 4.5556,Γ1 = 10.0643, Γ2 = 1.0643, γ = 0.05, z = 10,

β = 0.4286, α = 32.8767, P0 = 0.016, B0 =0.133. For this set of parameters, homoclinic chaosappears.

We considered an array of 30×30×30 units dif-fusively connected with D = 0.01. Figures 16 and 17show some frames of the evolution of a 3D-CNNmade of units with Eq. (17) starting from initialconditions chosen as follows:

xijk(0) =

k1 if rr < 5 and i > 0k2 if rr < 9 and i < 0k3 otherwise

(18)

where r =√

i2 + j2 + k2 and k1, k2 and k3 arevectors of six constants. The isosurface shown inFigs. 16 and 17 is defined by x1,ijk = 403 ∗ 10−3.

Fig. 16. Frames of the evolution of a 3D-CNN made of neurons [Eq. (17)] with homoclinic chaos.

August 2, 2005 9:44 01330


Fig. 17. Frames of the evolution of a 3D-CNN made of neurons [Eq. (17)] with homoclinic chaos.

The movies of the experiments discussed abovecan be downloaded from the webpage www.scg.dees.unict.it/activities/complexity/CNNindex.html.

4. General Discussion: Remarks andConsiderations

First of all, a general remark regarding the previousexperiments must be made: shapes in 3D-CNNs arethe fingerprint of emergent phenomena. This occursfor all the adopted 3D-CNN configurations. More-over, in the considered cases, the dynamical chaoticbehavior of each cell leads to harmonic shapes inthe 3D-CNN configuration. In order to reinforce theprevious remark, let us consider a counterexample.

Let us consider a system in which each cell is arandom generator with a given probability distribu-tion and let us consider the same coupling diffusivelaws and grid dimension of the other experiments.Figure 18 shows the results obtained by simulat-ing such a system. In this case, the level surfaceis irregular and there is no clear form arising. Forregularity, self-organization is not possible.

Let us consider the various shape trends shownin the various sequences of the previous section,the following strong observation is possible: eachshape is not recurrent in time. Moreover, in manyof them, spatial symmetries in each frame are evi-dent. The variety of shapes are related to celldynamics: however, some of them reflect global

August 2, 2005 9:44 01330


Fig. 18. When random generators are coupled together intoa 3D-CNN, there is no self-organization and regular shapesare not formed.

0 50 100 150 200 250 300 350 400 450 5000

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

t

area

Fig. 19. Trend of the shape area generated by a 3D-CNN ofHR neurons.

time-related features like the stretching and foldingphenomenon.

In order to reinforce the previous remark, thesurface area for each shape family is computedat each time. Chaotic time series is obtained. InFig. 19, the time series referring to the shape trendof a 3D-CNN of HR neurons is reported.

Following the introduction of the emerg-ing shape generation phenomena, the complexevolution trend by using 3D-CNN dynamics and the

role of self-organization in the previous phenomena,the problem of discovering recurrent patterns bothin the 3D-CNNs shapes and in modern art isencountered. It is not the aim of this paper toinvestigate on computer based arts or to deal withthe well-known evolutionary art [Bentley & Corne,2001]. This is to remark on the value of the stagger-ing complexity we are dealing with. Only few exam-ples are reported here. Let us consider the Miropaint Still Life with Old Shoe; recurrent patternsare found in the 3D-CNN generated shape when aLorenz system is adopted as cell unit. This is shownin Fig. 20.

The form of the sculpture of Duchamp is recur-rent in many patterns obtained during the 3D-CNNevolution. In particular, they appear in the consid-ered cases when either a grid of Rossler systemsor a grid of Inferior-Olive systems are taken intoaccount. In Fig. 21 the discussed example is shown.

Let us consider now the Robert Delaunay’spainting study. He started as an impressionist underthe suggestion of Cezanne and taking into accountthe Cubism, started an analytical research on theform in relationship to the multiplication of lightplanes. An example of this study is in Joie devivre where he expressed the emergence of lightand nature by using the contrast of colors whoseexpressions are sequences of closed curves. Let uscompare this painting with the 3D-CNN-generatedsurfaces as shown in Fig. 22. In this case the 3D-CNN cell is the HR dynamical system. In Fig. 23the Salvator Dalı’s Tete Raphaelesque Eclatee isshown. It reflects the concepts of broken forms,just introduced in the analytic cubism. In thisartistic expression the knowledge we have of thesubject is a complex sum of all its perceptions.The recurrent fingerprint patterns could be discov-ered in many 3D-CNN shapes like that derived byusing Izhikevich cells or like that obtained by usingLorenz cells as shown in Fig. 24.

Each observed dynamics of frames thatincluded highly organized shapes. Even if each cellis chaotic, even if each cell is characterized bythe geometrical form of the corresponding attrac-tor with a defined shape, the 3D-CNNs complexpatterns generate both an unusual complexity andan astonishing unpredictability. The discovery ofrecurrent patterns between 3D-CNN dynamics andartist paintings indicates the emergent character-istics of both underlying organized complexities.Through examples like those, it is emphasizedthat the dynamical evolution of coupled cells can

August 2, 2005 9:44 01330


Fig. 20. A form generated by the Lorenz 3D-CNN and Still Life with Old Shoe by Miro (black and white reproduction of theoriginal paint [Miro, 1937]).

Fig. 21. Some shapes recur in different 3D-CNNs like these shapes generated by a 3D-CNN either of IO neurons or Chua’scircuits. These forms resemble the artistic shape represented in Priere de toucher, by Duchamp [1947].

August 2, 2005 9:44 01330


Fig. 22. Shapes generated by a 3D-CNN of HR neurons and Joie de vivre, oil on canvas, by Delaunay [1930].

Fig. 23. Tete Raphaelesque Eclatee by Salvador Dalı [Delau-nay, 1930].

produce very beautiful and rich patterns drawn byartists.

Let us observe now the sequence of framesrespectively reported in Figs. 12 and 13. Reg-ular line segments are evident, unpredictableordered fragmentation explodes, a certain typeof intermittency frequently appears. Moreover

irregular shining flashes, like phosphenes appear insome of the frames. Moreover, in both Figs. 15and 24 circular waves appear. These forms areparticularly unstable showing very fast changes inshape, size and time-scale evolution. The variouscircles run giving us an impressive image of vorticesand turbulence.

The dynamical combination of shapes give us aglobal view perception whose effect is much morethan the sum of single shape contributions. Weare dealing with a complex visual pattern gener-ator. The information contained in the whole ismany times greater than the sum of the infor-mation contained in single parts. There existsa parallelism between the previously consideredframes and those referred in the migraine auraor in general, in the complex hallucination phe-nomena. A detailed description of these phenom-ena is widely reported in literature [Sacks, 1993;Kluver, 1967]. Moreover the visual effect of hal-lucinations is considered as a complex emergentdynamical phenomenon [Dalhem & Muller, 2003].In particular, many scientists view hallucinationas the propagation of Reaction–Diffusion waves inneural tissue. Many models have been proposed inthis direction. Cortical organization [Dalhem et al.,2000] in migraine aura is supposed, and mathe-matical models introduced to explain analyticallythe phenomena are widely accepted. Ermentroutand Cowan [Ermentrout et al., 1979] modeled thisphenomenon by using the relationship between

August 2, 2005 9:44 01330


Fig. 24. Shapes generated by a 3D-CNN of HR neurons and Joie de vivre, oil on canvas, by Delaunay [1930].

inhibitory and excitatory neurons. However, theyintroduced a linearized model that, even if theparameter relationship for deriving the instabil-ity condition is established, did not express theemergent mechanism of the hallucinations due tothe nonlinearity effects.

The myriad pattern formation due to themigraine aura is a fascinating phenomenon. Thediversity of migraine auras in various forms under-lines its complexity. Moreover the phenomenaemulated in the reported experiments are gen-erated by using 3D-CNNs in reaction–diffusionconfiguration using as cells integrate-and-fire neu-ron models. It is not the aim of this experi-ment to model using a grid of cortical neurons;moreover, the introduced 3D-CNN strategy allowsus to emulate real self-organizing phenomena inthe visual cortex. The CNN model for hallucina-tions has been also approached in [Chua, 1998].The experiments reported in our paper regard awider set of hallucination phenomena. They havebeen obtained thanks to the 3D-CNN architec-ture and to the introduction of more complex celldynamics with respect to those used in previouspapers.

In view of the appealing field of computa-tional neuroscience and in particular, in the area of

computer simulation algorithms devoted to explorethe nervous system, the reported examples reinforcethe suitability to adopt 3D-CNN circuits for emu-lating brain emergent phenomena.

Fig. 25. Il rimorso di Oreste by Giorgio De Chirico.

August 2, 2005 9:44 01330


CNN exactly translates the meaning of com-plexity in terms of electronic circuits. The resultsof coupling many simple nonlinear circuits giveus a global circuit whose capabilities are muchmore than the predictable performances obtainedby summing the single cell circuit contributions.This allows us to create artificially emergent pat-terns. Moreover, the same emergent behavior hasbeen discovered in the visual cortex phenomena ofmigraine aura and in the hallucination. The rich-ness of forms and their combination reflect a fur-ther impressive example of complexity. A furtherremark: the metaphysical art represents anotherexample of complexity from an aesthetic point ofview. Thinking Complexity is a new point of view,a new methodology. The vision of complexity themeis resumed, as an example, in the famous paintingof Giorgio De Chirico Il rimorso di Oreste shown inFig. 25.

5. Conclusions

In this paper, the use of 3D-CNN generalizedparadigm to generate sequences of emerging shapesand forms is discussed. A wide range of organizedresults from the evolution of 3D-CNN dynamicalsystem has been shown. Different cells dynamicshave been taken into account. Locally active cellsor cells at the edge of chaos have been chosen inorder to assure pattern formation. Complex 3D pat-terns emerging from various experiments have beencritically discussed. Links among circuits, art andneuroscience emerged thanks to the universality ofCNN formalization.

In his impressive book The Structure of Scien-tific Revolutions, Thomas Kuhn founded his theoryon the concept of paradigm [Kuhn, 1962]. With thisterm, Kuhn indicates the “scientific conquests uni-versally accepted which, for a period, give a modelfor problems and solutions for people that maderesearch in a particular field”. When the paradigmchanges, a critical breakpoint occurs in science andtherefore bifurcation conditions occur under whicha new theory replaces an old one. In any case, eachscientific relationship is a set of ideas that lead usto a small or big settling when they are replacedinto an old scientific paradigm.

Kuhn observed that the scientific revolutionscould be small or big, but both have the samestructure, the same characteristics when they occur.In fact, what happens is like the same emergentphenomenon that occurs in the sand pile or in

earthquake events. The cause could be small orbig, but what is remarkable is the emergent behav-ior that arises after the paradigm is changed. Inour opinion, what Kuhn remarks for the scientificrevolution occurred with the invention of CNN inthe field of information technology.

The emergent behavior of a scientific revolutionis related to the criticism of some aspects of classicalor previously accepted paradigms: the positive crit-icism of scientists leads to new paradigms in orderto overcome problems not solved by previous theo-ries. This is the starting point: ideas slowly evolveuntil the emergence occurs and the new paradigmswitches to the revolution! In the history of dis-tributed intelligence paradigms a fundamental limitof perceptron architectures has been highlighted byMinsky and Papert [1988] that established a percep-tron cannot tell whether two labyrinthine patternson the cover are connected or not. The CNN locallyconnected networks can solve such a problem! It hasbeen proved that a locally connected network likeCNN has the properties to recognize local functions[Chua, 1998]. Therefore, in order to overcome aproblem a new successful paradigm has been intro-duced. The critical point of a technological revo-lution has been established and now after sixteenyears, the change of the paradigm in the area ofconnectionism leads to a scientific revolution! Therevolution has been related to the new CNN pro-posed approach. Moreover, the increase of inter-est in the CNN paradigm worked like an attractor.Starting from pattern recognition, complex modelbehaviors, vision, neuromorphic models, robotics,neuroscience problems have been faced in termsof CNN paradigm. The new formalization allowedthe conception of new advanced equipments, anda revolution in the field of information technol-ogy is ongoing. The dynamical richness of CNNbased architecture allowed to reformulate classicalproblems in a new formalization. The last effortproposed in this paper is to investigate on theuniversality of the 3D-CNN in order to discoveremergent 3D shapes. Self-organization that is thecore of the emergent characterized CNN systemallows shorter distances between technological sci-ence, art and neuroscience.

Acknowledgments

This work was supported by the Italian “Minis-tero dell’Istruzione, dell’Universita e della Ricerca”(MIUR) under the Firb project RBNE01CW3M.

August 2, 2005 9:44 01330


References

Arena, P., Baglio, S., Fortuna, L. & Manganaro, G.[1995] “Chua’s circuit can be generated by CNNcells,” IEEE Trans. Circuits Syst.-I 42, 123–125.

Arena, P., Baglio, S., Fortuna, L. & Manganaro, G.[1996] “How state controlled CNNs generate thedynamics of the Colpitts-like oscillator,” IEEE Trans.Circuits Syst.-I 43, 602–605.

Arnheim, R. [1974] Art and Visual Perception: A Psy-chology of the Creative Eye, 2nd Revised Edition(University of California Press).

Bentley, P. J. & Corne, D. W. [2001] Creative Evolution-ary Systems (Morgan Kaufmann).

Bogousslavsky, J. [2003] “The neurology of art — Theexample of Giorgio de Chirico,” Eur. Neurol. 50, 189–190.

Chua, L. O. & Yang, L. [1988a] “Cellular neural net-works: Theory,” IEEE Trans. Circuits Syst.-I 35,1257–1272.

Chua, L. O. & Yang, L. [1988b] “Cellular neural net-works: Applications,” IEEE Trans. Circuits Syst.-I35, 1273–1290.

Chua, L. O. & Roska, T. [1993] “The CNN paradigm,”IEEE Trans. Circuits Syst.-I 40, 147–156.

Chua, L. O. [1998] CNN: A Paradigm for Complexity,World Scientific Series on Nonlinear Science, Series A,Vol. 31.

Ciofini, M., Labate, A. & Meucci, R. [1999] “Stabiliza-tion of unstable fixed points in the dynamics of a laserwith feedback,” Phys. Rev. E60, 398–402.

Dahlem, M. A., Engelmann, R., Lowel, S. & Muller, S. C.[2000] “Does the migraine aura reflect cortical orga-nization?” Europ. J. Neurosci. 12, 767–770.

Dalhem, M. A. & Muller, S. C. [2003] “Migraine auradynamics after reverse retinotopic mapping of weakexcitation waves in the primary visual cortex,” Biol.Cybern. 88, 419–424.

Dalı, S. [1951] Tete Raphaelesque Eclatee, ScottishNational Gallery of Modern Art, Edinburgh.

Delaunay, R. [1930] Joie de Vivre, Pompidou Center,Paris, France.

Duchamp, M. [1947] Priere de Toucher, Modern ArtMuseum of the City of Paris, Paris, France.

Ermentrout, G. B. & Cowan, J. D. [1979] “A mathe-matical theory of visual hallucination patterns,” Biol.Cybern. 34, 137–150.

FitzHugh, R. [1961] “Impulses and physiological statesin theoretical models of nerve membranes,” 1182-Biophys. J. 1, 445–466.

Fortuna, L., Rizzo, A. & Xibilia, M. G. [2003] “Modelingcomplex dynamics via extended PWL-based CNNs,”Int. J. Bifurcation and Chaos 13, 3273–3286.

Giaquinta, A., Argentina, M. & Velarde, M. G. [2000]“A simple generalized excitability model mimickingsalient features of neuron dynamics,” J. Stat. Phys.101, 665–678.

Giles, J. [2004] “Neuroscience: Change of mind,” Nature430, p. 14.

Izhikevich, E. M. [2003] “Simple model of spiking neu-rons,” IEEE Trans. Neural Networks 14, 1569–1572.

Izhikevich, E. M. [2004] “Which model to use for corticalspiking neurons?” IEEE Trans. Neural Networks 15,1063–1070.

Kluver, H. [1967] Mescal and the Mecanism of Halluci-nations (University Press of Chicago, Chicago).

Kuhn, T. [1962] The Structure of Scientific Revolutions(University of Chicago Press).

Lynton, R. [1994] The Story of Modern Art (PhaidonPress).

Minsky, M. & Papert, S. A. [1988] Perceptrons: An Intro-duction to Computational Geometry, Expanded edi-tion (MIT Press, Cambridge, MA).

Miro, J. [1937] Still Life with Old Shoe, The Museum ofModern Arts, NY, USA.

Morales, C. [1991] “Profile: Eladio Dieste,” in MIMAR41: Architecture in Development (Concept MediaLtd., London).

Nagumo, J., Arimoto, S. & Yoshizawa, S. [1960] “Anactive pulse transmission line simulating 1214-nerveaxons,” Proc. IRL 50, 2061–2070.

Orsucci, F. [2002] Changing Mind, Transitions in Natu-ral and Artificial Environments (World Scientific).

Otto, F. [1982] “Frei Otto’s new work,” in MIMAR 4:Architecture in Development (Concept Media Ltd.,Singapore).

Pisarchik, A. N., Meucci, R. & Arecchi, F. T. [2001]“Theoretical and experimental study of discretebehavior of Shilnikov chaos in a CO2 laser,” Europ.Phys. J D13, 385–391.

Podoll, K., Robinson, D. & Nicola, U. [2001] “Themigraine of Giorgio De Chirico. 1. History of illness,”Neurol. Psych. Brain. Res. 9, 139–156.

Rose, R. M. & Hindmarsh, J. L. [1989] “The assemblyof ionic currents in a thalamic neuron. I. The three-dimensional model,” Proc. R. Soc. Lond. B237, 267–288.

Sacks, O. W. [1992] Migraine (Vintage Books).Strogatz, S. H. [2000] Nonlinear Dynamics and Chaos

(Perseus Publishing).Wilkinson, M. & Robinson, D. [1985] “Migraine art,”

Cephalalgia 5, 151–157.Yang, T. & Chua, L. O. [2001] “Testing for local activity

and edge of chaos,” Int. J. Bifurcation and Chaos 11,1495–1591.

Zeki, S. [1999] Inner Vision: An Exploration of Art andthe Brain (Oxford University Press).

Appendix ACharacteristics of Eˆ3

In this section, the main features of Eˆ3 are illus-trated. This software has been designed to simulate

August 2, 2005 9:44 01330


complex systems made of interacting dynamic non-linear units as described in Sec. 2. Therefore Eˆ3

has been designed to include three properties: thedynamics of each simple unit is arbitrary, the cellsof the complex system are either identical or differ-ent, connections are arbitrary.

Eˆ3 provides the user with the possibilityof implementing in a very simple way arbitrarydynamics for each cell of complex systems. Follow-ing this approach, each cell of the complex CNNconsists of an arbitrary n-order system, and maybe defined by the user by writing its equations.This differs from standard CNN in which each celldynamics is defined by templates, which thereforecontain both cell dynamics and connections amongcells.

Moreover, the cells of the 3D-CNN may be dif-ferent from each other. The more general case isthat the equations defining each of the cells consti-tuting the complex systems are different from cellto cell. This is, for example, the case in which onewould model fire propagation in two different adja-cent substratums. Moreover, cells may be nearlyidentical. In this case, cells of the complex systemsmay differ only for the value assumed by their char-acterizing parameters. This case is different fromthe previous one, in fact, in this case one does notneed to write new equations, but the possibility tohave space-variant parameters should be included.

Finally, the connections among the units of theCNN and, in general, of a complex system may be ofseveral types. Several examples of complex systemsmade of locally interacting units have been studiedin literature; all-to-all coupling is also very com-mon in modeling complex systems. While randomnetworks efficiently model phenomena like stockmarkets, and small world connections account formodels as spread of diseases, modelling the struc-ture of the world wide web requires a dynamicallychanging network (scale-free network). A generalsimulator for complex systems should provide thepossibility of implementing all these structures inan easy way and at the same time should allow theuser the possibility of reconnecting arbitrary cellsof the system.

Another important characteristic of Eˆ3 is theuse of parallel computing. A general structure hasbeen designed for Eˆ3. The simulator can be runeither on a single machine or on a network ofpersonal computers. This second case implementsparallel computing. The software has been writtenin C (routines for numerical integration, output

visualization, definition of cell dynamics) andJava (communication between processors, man-agement of data distribution) and is based onopen-source components

¯.

Parallel computing may be realized by usingcoarse grained or fine grained architectures. Acoarse grained architecture refers to the case inwhich the computation is distributed to a smallnumber of high-capability processors, while thearchitecture is fine grained when a high number ofsimple processors is used. In this sense a CNN isthe most significant example of fine grained archi-tecture. The architecture of Eˆ3 is based on a smallnumber of high-capability processors.

The strategy adopted to implement parallelcomputing is the so-called domain decomposition,in which data are distributed among the processorsexecuting the same operations on different portionsof the data. In fact, the problem of emulating a sys-tem made of many units can be simply decomposedinto several domains made of subparts of the wholeset of the cells. A processor plays the role of masterand collects all the results coming from the elabo-ration by the other processors. Moreover, throughthe so-called message passing each processor mayobtain data processed by other units of the parallelarchitecture.

A very efficient way to implement message pass-ing is to create a cluster of workstations in a LANnetwork. The whole software therefore consists oftwo main modules, called server and client (Fig. 26).The server runs in each processor of the network,while the client runs on the master PC, coordinatingthe data coming from the different processors. Thecomplex system to be emulated has to be definedin the client. The first operation executed by theclient is to create the structure and to assign toeach client a portion of the cells to be simulated.Then the integration routine is performed.

To allow the definition of the cell dynamics bythe user it has been chosen to implement a rou-tine — called compile — able to create a .dll file

Fig. 26. Main modules of the software architecture of Eˆ3.

August 2, 2005 9:44 01330


starting from the cell equations written in C lan-guage. The integration routine uses the .dll file: itis created by the client and then distributed to eachserver. This choice has been adopted to create anuniversal tool that does not require long training tobe used. Practically, the only code that the user hasto write refers to the cell equations.

As regards the topology of connections, it hasbeen chosen to implement the more general case,i.e. not restrict to the case of local connections,by allowing the user the possibility of customizingeach connection. The whole set of connections isdescribed in the .net file, that can be modified bythe user. Moreover, several routines to generate themost commonly used topologies (local connectionsin 1D, 2D and 3D spaces, all-to-all coupling, and so

on) have been designed. These routines create the.net file that can be successively edited to introducemodifications in the simulated system.

Eˆ3 provides several built-in procedures foroutput visualization such as 3D level surfaceand 2D density plots. Moreover, all the datacan be stored in universal format for furtherevaluation.

The input files for the client module of Eˆ3 arereassumed in the following:

• the .mpi file containing the parameters definingcommunication between client and server;

• the .net file containing the network topology;• the .out file containing the parameters for the

numerical integration and output parameters.

Documents

THE CNN PARADIGM: SHAPES AND COMPLEXITY