Noname manuscript No.(will be inserted by the editor)

The Cortexionist Architecture: Behavioural Intelligenceof Artificial Creatures

David Panzoli · Sara de Freitas · Yves Duthen · Herve Luga

the date of receipt and acceptance should be inserted later

Abstract Traditionally, producing intelligent behavioursfor artificial creatures involves modelling their cognitiveabilities. This approach raises two problems. On theone hand, defining manually the agent’s knowledge is aheavy and error-prone task that implies the interven-tion of the animator. On the other hand, the relation-ship between cognition and intelligence has not beentheoretically nor experimentally proven so far. The eco-logical approaches provide a solution for these prob-lems, by exploring the links between the creature, itsbody and its environment. Using an artificial life ap-proach, we propose an original model of memory basedon the synthesis of several neuroscience theories. TheCortexionist controller integrates cortex-like structureinto a connectionist architecture in order to enhancethe agent’s adaptation in a dynamic environment, ulti-mately leading to the emergence of intelligent behaviour.Initial experiments presented in this paper prove thevalidity of the model.

Keywords computer animation · autonomous adap-tive agents · cognitive modelling · human memory

1 Introduction

Realistic human-like agents are nowadays able to followgoals, plan actions, manipulate objects [1], show emo-

David Panzoli, Yves Duthen, Herve LugaIRIT-CNRS, UMR 5505, Universite de Toulouse,Toulouse, France.E-mail: {David.Panzoli, Yves.Duthen, Herve.Luga}@irit.fr

Sara de FreitasCoventry University,Serious Games Institute,Coventry, United Kingdom.E-mail: S.deFreitas@coventry.ac.uk

tions [2] and even converse with humans. Despite theseagents being endowed with many abilities, the questionof intelligence, even for simple animal agents, is stillbeing considered. Indeed, intelligence is not necessarilyrelated to the ability to manipulate objects or synthe-sise speech, follow goals or plan actions. Many machinesaround us can do that although they may not be con-sidered intelligent.

From the earliest work of Alan Turing [3], artificialintelligence (AI) has considered that building an intel-ligent system implies the imitation of human mentalprocessing, what is referred to as cognitive modelling.In practice, imitating the way the human mind worksinvolves writing by hand complex algorithms, scripts,or sets of rule, the relevance of which mostly dependson how they are interpreted.

Lately, radically different research works proposednew ways to understand and consider intelligence. Pfeiferand Bongard [4] introduced the concept of embodiment,which foregrounds the major role that the environmentplays in the cognitive abilities of any creature that livesin it. In parallel, Jeff Hawkins introduced the memory-prediction framework [5], a general theory of the neo-cortex, emphasising the role of memory in intelligence.

Our research is positioned within an artificial life(AL) context, and builds upon recent work of the teamat IRIT [6]. Lassabe and colleagues suggest that en-dowing a virtual creature with realistic behaviours im-plies that this creature’s morphology emerges from theenvironment, taking away the need of designing thesebehaviours by hand. Following a similar idea, the workpresented in this paper intends to show in addition thatthe behavioural intelligence of this creature should alsobe strongly connected to the complexity of the environ-ment, as opposed to relying on human expert cognitive

2

modelling.

Section 2 introduces the related work of the disci-pline, reviewing the trends in agent design in animation.Section 3 sets out the contributions of the present work.Section 4 details the foundation of our approach, no-tably through the investigation of the relation betweenintelligence and the cortex. In Section 5, we present theCortexionist model: a reactive connectionist architec-ture endowed with the ability to deal with inner rep-resentations. This section emphasises the way knowl-edge is created, stored, maintained and used througha process we call extended action-selection. Section 6presents experimental data outlining how the model isexpected to reveal intelligent behaviour. Some interest-ing results are also presented and discussed. Section 7proposes an extension to increase the accuracy of theCortexionist model. Finally, Sections 8 and 9 provide aconclusion and a discussion about future work.

2 Related Work

In the field of behavioural simulation, hybrid archi-tectures [1,7–10] have been the most competitive sofar. These behavioural controllers aim to combine re-activeness and cognitive abilities. A first reactive com-ponent provides a direct interplay between perceptionand action in order to ensure the reactiveness of theaction-selection mechanism [11]. In parallel, a delibera-tive component is responsible for modelling the cogni-tive abilities, such as planning, reasoning or communi-cating. Basically, AI algorithms such as A* are appliedto world representations of the environment. These rep-resentations are organised through topological or gridmaps where the objects of the environment are them-selves represented symbolically such as Frames [12] orSmart Objects [13].

Cognitive modelling makes clear that endowing au-tonomous virtual characters or creatures — hereafterreferred to as agents — with a memory of knowledgerepresentations enhances their general behaviour, thusenabling them to interact with the environment, includ-ing with the other agents. However, the question of howthis knowledge is designed raises two problems. First,the animator is expected to design every feature of ev-ery object in the environment, not only their propertiesand interacting possibilities (e.g. the affordances), butalso the relation they maintain with one another. Notonly is this a tedious task, but it is error-prone as well.Secondly, a well-known weakness of symbolic represen-tations is the symbol grounding problem [14]. Briefly,

Harnad states that whatever one can expect with ma-nipulating symbols, “relating the symbols to the agent’sperception is coextensive with the problem of cognitionitself”. This reveals the lack of integration of the knowl-edge in the behaviour, usually stressed by the separa-tion of memory and control into distinct modules in thecontroller’s architecture.

As a consequence, although cognitive modelling in-herits fifty years of AI expertise, it has begun to loseground to the benefit of more ecological approaches [4,5] which place an emphasis upon the predominant roleof adaptation, thus considering cognition as a conse-quence rather than a prerequisite for intelligence. In thisperspective, AL architectures have regained popularity.Although they appear as more limited than hybrid ar-chitectures, owing to their relative simplicity, they giveemphasis to the adaptation of the virtual creature tothe detriment of cognitive abilities, and offer the op-portunity to bring together memory and control into asimple and lightweight system.

3 Contribution

Traditionally, AL approaches focus on action-selection.Since action-selection consists of associating perceptionwith action, AL considers adaptation as the ability todetect whether such an association is not relevant andhence performs changes in the controller in order to im-prove it. In this paper, one original vision of adaptationis investigated. We believe the interest of a memorydoes not reside in the way representations are storedor can be manipulated but rather in the representa-tions themselves, particularly how they can enhancethe adaptation of the agent to the environment. More-over, we postulate that the ability to create patterns ofknowledge from the environment, and take advantageof them to improve action-selection may reveal intel-ligent behaviour, insofar as the environment allows it.Indeed, we assume that intelligent behaviour cannot beexpected from a creature operating in a trivial environ-ment.

The contribution of this work is dual. First, we pro-vide a methodology for integrating a memory into a re-active agent, using an approach that is primarily con-cerned with avoiding the weaknesses of hybrid archi-tectures. This memory, then, must be integrated intothe action-selection (e.g. does not require to be user-defined) and grounded upon perception (e.g. not sym-bolic). We then prove through experiments that mem-ory is a bridge between complexity in the environmentand intelligence of the behaviour.

3

4 Foundations

The term Cortexionist, firstly introduced in [15], is aneologism formed from cortex and connectionist. Thelatter part derives from the connectionist foundation ofthe model. We believe indeed that neural networks pro-vide an adequate way to model the brain mechanisms,since they are a computational abstraction of the struc-tures of the nervous system. However, simulating an en-tire brain is a more complex task. The last ambitiousproject [16] to date intends to simulate the brain of amouse. Using the IBM BlueGene super-calculator, themodel could simulate 8 million neurons —which is halfthe actual size of a mouse brain— every one of eachhaving 6300 synapses, for a duration of 10 seconds, at 10times less the actual speed of the brain. Although thissimulation was not a failure, it turned out to be unableto output any exploitable results. The reason invokedis the lack of current computing power, as comparedto the high computational complexity of the brain, asevery neuron is linked to thousands of others.

Trying to reduce this complexity often means di-viding the model into sub-parts, each of them beingresponsible for the simulation of a sub-system or anidentified function in the brain. The Psikharpax [17]project for instance aims to model the brain of a rat.It is composed of many modules, each responsible for asingle task, such as navigation [18] or action-selection[19]. This ‘divide and conquer’ approach is successfulbut, as Gibbs notices: “scholars seem more interestedin studying parts of people than they are in figuring outhow the parts work together to create a whole, intelli-gent human being” [20].

As a matter of fact, we believe the reason why theprevious attempts to model the brain were unsuccess-ful is not a purely computational limitation, nor is it amisunderstanding of how the brain areas are connectedtogether. We believe it is rather the consequence of ourinability to understand the mechanism of intelligence asa whole process, and to provide the right structures toadequately model this process. It is a fact indeed thatevolution has favoured the appearance of structures inthe brain, and it is likely as well that the appearanceof new abilities is related to these new structures. Thisidea is the very centre of McLean’s triune brain the-ory [21]. McLean hypotheses that the brain is actuallycomposed of three components, which successively ap-peared during evolution (Figure 1).The reptilian brain, or archicortex, appears when thefish left the water to populate the ground as batra-chians. It is the first central nervous system, totallyinsensitive to learning and applying stereotypical andrigid schemes. It brings the first natural instincts (sur-

Fig. 1 McLean’s hypothesis considers that the brain is made ofthree layers, inherited from evolution.

vival, aggression and territorial instinct). Anatomicallyspeaking, the reptilian brain corresponds to the brainstem and basal ganglia.The limbic brain, or paleocortex, appears progressivelywith some mammals, bringing some specialised areasresponsible for emotions, fear or desire, but above all amotivation centre that introduces the notions of successand failure. As a consequence, the limbic brain allowsthe creature to learn, by associating situations withfeelings, and thus fostering the adaptation of the crea-ture to its environment or a social group. The limbicbrain regroups numerous sub-cortical structures, themost important being the hippocampus, the hypotha-lamus and the amygdala.The neocortex is the most recent layer, known to beresponsible for imagination, abstract thinking and con-sciousness. It appeared with the big mammals, increas-ing in importance with the primates, and finally peak-ing with human beings. It is important to note thatin the course of evolution, the neocortex has always ex-panded — from 410 cm2 in Australopithecus to 1400 cm2

with Homo Sapiens — and every growth phase has al-ways reflected a significant evolution of their abilities.

Although McLean’s theory is still debated today,and scientists prefer to refer to brain structures (thebasal ganglia) or systems (the limbic system) insteadof layers, it still provides an insight with how evolutionhas gradually ‘designed’ structures to allow the crea-tures to learn, therefore leading to the emergence ofthe individual over the species. Besides, studying therelations between the cortex and the rest of the brainhas the potential to elucidate the relation between be-haviour and intelligence.

Hawkins’ memory-prediction [5] framework1 providesa simple yet insightful view of how the cortex works. It

1 The memory-prediction theoretical framework, issued to theHierarchical Temporal Memory model, is supported by an imple-mentation based on Bayesian networks [22].

4

has been observed that the cortex was horizontally splitinto 6 layers, C1 to C6 in Figure 2, and vertically intomany columns, called cortical columns. Studying howthese columns were functioning, Hawkins noticed that,despite their apparent complexity, they were dedicatedto a really simple process: associating and delaying neu-ral signals in order to store temporal patterns. Thesepatterns exist at different levels of complexity (a mu-sical piece is perceived as a sequence of musical notes,each note being itself perceived as a sequence of varia-tions, etc).

In parallel to storing these sequences, the cortexconstantly makes predictions about the next percep-tion, at all of the different levels of complexity. Depend-ing on this level, these predictions express the abilityto recognise a known object from some of its features,or to anticipate a situation on the basis of a few clues.Hawkins’ opinion owes much to the ecological approach.Basically he states that human intelligence is based onthe fact the environment is structured, and is catalysedby the ability to detect and represent these structures(in the cortex).

C1

C2C3

C4

C5

C6

Thalamus

Fig. 2 The thalamus plays a major role in the learning of se-quence, by delaying and sending back the data sent by everycortical column. This figure schematises a cortical column.

5 Description of the Model

The Cortexionist model is founded on a simplified hy-pothesis. The human brain benefits from a long evo-lution, during which several structures have appeared,enabling the human beings first to behave, then to learnfrom and adapt to their environment, and finally to be-come intelligent. We think human beings not only owe

this last evolution to the appearance of the cortex, butto the way it interacts with the previous structures.Therefore, the model aims to emulate and investigatethis interaction by transposing a neocortex-like struc-ture (e.g. able to detect, capture and mirror the struc-tures in the environment) onto a computational lim-bic system (e.g. a structure responsible for adaptivebehaviours). From the interaction between these twocomputational structures, we hypothesise that we willbe able to observe the emergence of behavioural intel-ligence.

5.1 Control

The model we propose here is based on a pure ALagent. The AL view is that the agent’s adaptivity isclosely tied to the controller’s ability to be modelled,through learning or evolution. Connectionist controllershave regularly proved suitable for accurately shaping adesired behaviour [23–25].

Our controller is based on the simplest connectionistcontroller, a perceptron with two layers: one input layerfor the perception and one output layer for the actions.The sensors of the agent are bound to neurons from theinput layer. In a similar way, neurons from the outputlayer are bound to the agent’s actuators. Feedforwardneural connections link the input layer to the outputlayer. Finally, these connections are tweaked during atraining stage by a classic back-propagation algorithm[26].

Basically, the action-selection works as follows: astimulus from the environment activates neurons on theinput layer. The resulting signal is spread to the outputlayer where output neurons are either activated or not,depending on the connections. The activation of outputneurons is translated into actions to be performed bythe agent in the environment. At the end of the cycle,the agent faces a new situation and the loop starts overagain.

5.2 Knowledge representation

Beyond the reactive control of the agent’s behaviour,our model gives emphasis to the ability for buildinginner representations.

The use of neural networks is totally appropriateconsidering they are firstly a metaphor of human ner-vous structures, such as the brain. From the connection-ist point of view, memory is distributed and highly as-sociative [27], which means every piece of knowledge is

5

built from the assembly of neurons operating through-out the brain, as associative networks. These networks(see [28] for a survey) are dedicated to reproducing themechanisms underlying the creation, maintenance anddestruction of knowledge. To that end, they rely on as-sociative rules, all of which are derived from the originalHebbian rule [29].

In our model, for the purpose of seamlessly inte-grating the memory inside the controller, such a ruleapplies to existing neurons of the controller. More pre-cisely, it applies to the input layer as knowledge has tobe grounded upon perception.

5.2.1 Creating knowledge

The unsupervised Hebbian rule states that co-activatedneurons tend to strengthen their mutual connections,in such a way that the mere exposure to co-occurringfeatures leads to their association into a piece of knowl-edge.

In our model, perceiving several features with oneanother in the environment results in the activation oftheir related neurons on the input layer, thus leadingto the creation of a pattern of knowledge as shown inFigure 3.

A

B

C

E D

A

B

C

E D

(a) (b)

A

B

C

E D

(c)

Fig. 3 A pattern is created by applying an associative law. Letus consider a set of neurons (a). When a subset of neurons is co-activated (b), their mutual connections are strengthened in orderto form a pattern of knowledge (c). A black arrow on top of aninput neuron means the neuron is activated by its related sensor.

5.2.2 Retrieving and generalising knowledge

The main interest in creating patterns of knowledge isthe possibility of retrieving it subsequently from a few

clues. This is achieved by a property inherent in asso-ciative networks called pattern completion (describedin Figure 4).

A

B

C

E D

A

B

C

E D

(a) (b)

Fig. 4 Pattern completion is quite a simple rule: when a subsetof neurons from a pattern is large enough, the whole pattern isactivated. This can be understood in this example by neurons A,B and E propagating to C.

Interestingly, this property is also the very founda-tion of the ability to transfer or generalise knowledge.Indeed, both these processes rely on the ability to detectthe proximity of two patterns of knowledge. As knowl-edge is distributed into numerous neurons, the similar-ity of a pattern with another logically relates to thenumber of neurons shared by both patterns. Knowingthat, scaling the degree of generalisation in our mem-ory can be attained by amending pattern completion,and by adjusting the weight value of each participatingconnection when building a pattern.

Let us have a closer look at how this may work.We consider binary neurons, with a Heaviside transitionfunction whose threshold s is set to 0.65. This meanseach neuron is activated when the sum Σ of enteringsignals is higher than 0.65. We fix arbitrarily the com-pletion threshold Sc to 50%, so that half (or more) neu-rons of a pattern are necessary to activate the whole.When n neurons are co-activated, the weight value ofeach connection of the new pattern is computed as fol-lows:

w =s

n× Sc. (1)

Figure 5 shows how an entering signal may or maynot be associated with a known pattern of knowledge.

Finally, we need to make sure that overlapping knowl-edge does not prevent pattern completion from work-ing properly. Indeed, dealing with distributed repre-sentations implies that each pattern is likely to shareone neuron or more with others. We hypothesise thatthe rule can apply to overlapping patterns without anymodification, as shown in Figure 6.

6

A

B

C

A

B

C

0,43

0,43

0,43

(a) (b)

C

A

B

0,43

0,43

0,43

C

A

B

0,43

0,43

0,43

(c) (d)

Fig. 5 The weight value of each connection inside a pattern re-lies on the pattern size and the completion threshold. (a-b) Inthis example, applying the above formula during learning setseach weight to w = 0.65

3×0.5= 0.43. In (c), neurons A and C are

activated. The entering signal in neuron B is Σ = 2×0.43 = 0.86.Completion happens and B is thereby activated because 0.86 >0.65. In (d), only neuron A is activated. The entering signal inneurons B and C is 0.43, which is not sufficient compared to theactivation threshold. Completion does not occur.

5.2.3 Unlearning knowledge

Although learning new knowledge is the most impor-tant ability of a memory, unlearning is another fun-damental property in order to keep this connectionistmemory from saturation and to accommodate changesin the environment.

Although, this work does not cover a comprehensivestudy about forgetting in neural networks, it appearsthat the prevailing model considers that knowledge inmemory decays in time, so that any material that is notoften re-learnt is likely to be forgotten eventually. Weadvocate a slightly different idea. We envisage that for-getting something actually means learning somethingdifferent, although it is close enough to overwrite theformer knowledge. This section explains how this maywork in practice.

Once again, the process must be kept simple enoughto be seamlessly integrated into our associative net-work. We postulate that the simple use of inhibitoryconnections can help us solve the issue.

B

A C

DE

B

A C

DE

(a) (b)

B

A C

DE

B

A C

DE

(c) (d)

B

A C

DE

(e)

Fig. 6 Two patterns Pi and Pj are likely to overlap when Pi

does not share enough neurons with Pj to cover it. (a-b) {A,B,C}cannot be completed so that a new pattern {C,D,E} is formedaside. (c-e) Despite two patterns overlapping, the completion ruleapplies normally.

C

B

A

D

C

B

A

D

(a) (b)

C

B

A

D

(c)

Fig. 7 Areas account for the creation of inhibitory connections.When several neurons belong to a common area (a), creating apattern (b) involves the creation of excitatory connections be-tween co-activated neurons but also inhibitory connections withnon-activated neurons of the area (c).

To date, we have used an associative rule to cohereneurons by building excitatory connections. The pur-pose of inhibitory connections is, on the contrary, to

7

dissociate neurons. Since co-activated neurons are asso-ciated, neurons that are not activated at the same timeshould therefore be dissociated. Considering that only afew neurons (relatively to the total number of neurons)are co-activated every millisecond in the brain, it shouldbe saturated with inhibitory connections between non-coactivated neurons. The reason why it is not is the ex-istence of cerebral regions, which are brain areas thatregroup the neurons dealing with the same modalityand partition the modalities one from the other.

Transposed to the model, managing inhibitory con-nections requires the associative layer to be partitionedinto areas that regroup neurons related to the same sin-gle sense. Inside such an area, we can presume that twoneurons are unlikely to be activated at the same time,just as perceiving an object is blue reliably involves it isnot red. Figure 7 shows how these patterns may includeinhibitory connections. Following from this, and giventhe ability to represent things that cannot co-occur,Figure 8 shows how one pattern can replace another.

A

B

C

E D

A

B

C

E D

(a) (b)

A

B

C

E D

(c)

Fig. 8 The ability to forget is illustrated by replacing a patternwith another, close but still different. In (a), {A, B, C, E} repre-sents an already known pattern. C and D neurons inhibit eachother since they belong to the same area. When A, B, E and Dare activated, D prevents the completion of the original pattern.Regardless, the associative rule applies and builds a new pattern{A, B, C, E}.

5.3 Integration and extended action-selection

Following the insight that inner representations of theworld could help enhance the behaviour of the agent,we have turned the input layer of a simple connection-ist controller into a fully workable associative memory,

imitating to a certain extent some of the most funda-mental properties of memory.

Indeed, referring to the Squire and Cohen theory onprocedural and semantic memory [30] indicates we haveobtained a complete model of memory. In the oppositedirection of the traditional multi-store model of mem-ory [31] that differentiates between short-term (STM)and long-term memory (LTM), Squire and Cohen pro-pose a more functional dichotomy between procedu-ral memory, which benefits a supervised and iterativelearning, and semantic memory, whose learning is con-versely unsupervised and non-iterative (e.g. ‘one-shot’).

Making a parallel with this theory, the initial per-ceptron is clearly an instantiation of a procedural mem-ory, whereas the associative network has all the fea-tures of a semantic memory. From now on, we will usethe term ‘procedural’ to refer to the perceptron con-nections, to the behavioural rules these connections arestanding for, and to the learning applied to the per-ceptron. We will however retain the term ‘associative’when referring to the associative network in order toavoid any confusion, owing to the particular meaningof the term ‘semantic’ in computer science.

input layer

output

layer

proprioception sight

smelltouch

sightsmell

touch

proprioception

perception

from sensors

action

to actuators

patterns

perceptron

connections

Fig. 9 The behaviour of the agent, which can be a virtual char-acter or a robot equally, consists of performing actions in theenvironment. Actions are triggered by actuators, following theactivation of output neurons. Output neurons are themselves ac-tivated by input neurons, through the vertical propagation of thesignal along the perceptron connections. Finally, input neuronscan be directly activated by sensors, depending on the featuresperceived inside the environment, or indirectly by other inputneurons, as patterns complete horizontally.

The model resulting from the integration of the as-sociative network into the perceptron, presented in Fig-ure 9, combines action-selection and sensory-driven knowl-edge representations. As a matter of fact, it also allows

8

the investigation of a deeper relation between procedu-ral and semantic memory.

Firstly, this new model works exactly like a standardperceptron, as the introduction of the associative net-work does not interfere with the action-selection (Fig-ure 10, a and b). However, the associative network’sability to create and retrieve patterns of knowledge isresponsible for some neurons on the associative layerbeing activated by pattern completion, e.g. being acti-vated, although they are not actually perceived in theenvironment. As a result of these neurons also beingpart of the perceptron, as input neurons, they may leadto the selection of an action, if they happen to be partof a procedural rule (Figure 10.c).

(a) (b) (c)

Fig. 10 Standard action-selection (a) states that if there is aconnection (e.g. a procedural rule) between the input neuron P0

and the output neuron A0, the activation of P0 entails the acti-vation of A0. Without a connection (b), like between P1 and A0,this activation has no effect. The extended selection illustrates asfollows (c): Considering now the creation of the pattern {P0, P1},the activation of P1 leads, owing to pattern completion, to theactivation of P0, which in turns activates A0.

In short, neurons inside the input layer can eitherbe activated by direct perception or by pattern comple-tion. It means that beyond the procedural rules taughtto the agent during the procedural training, new rulesmay implicitly appear when using knowledge formedduring associative learning. We name this feature theextended action-selection. The next section presents ourset of experiments in which we intend to demonstratethat intelligence may emerge from the extended action-selection.

6 Experiments and Results

Experiments are organised in a 3D virtual environmentwhere an agent (Agent) operates among other crea-tures: creatures of the same kind (Mother, Fellow) andpredators (Predator). As shown in Figure 11.a, Agent isa small green creature with a tentacle shape. Predatoris a purple tentacle with arms. Agent is also surrounded

by other green grown-up tentacles, as Fellow. Mother issimilar to Fellow but has a particular smell that Agentis familiar with.

Every creature except Agent has a user-defined be-haviour. The aim of Predator is to catch and attackAgent, however avoiding conflicts with other grown-upcreatures, that are able to defend themselves. In thisway, Fellows and Mother aim to protect Agent fromPredator. In practice, Mother and Fellow are providedwith a protection range. If they can perceive both Agentand Predator within this range, they move towardsAgent in order to offer some protection.

Agent is equipped with the Cortexionist controller,a representation of which is provided by a specific viewer(Figure 11.b). From several sensors, Agent’s perceptionis transported to dedicated neurons on the input layer:two for colours (‘green’, ‘purple’), two for recognisingbasic shapes (‘tentacle’, ‘arms’) and another that de-tects the smell of Mother (‘smell’). Finally, Agent isalso able to feel whether it is colliding with anothercharacter (‘collision’). Such a perception is far from be-ing based on a detailed imitation of the eye or the nosebut still, it is non-symbolic since Agent is unable todirectly recognise a creature. Agent’s action set con-sists of two high-level actions: ‘move toward the closestcreature’ and ‘run away from the closest creature’.

Finally, Agent is provided with an initial amount ofvital energy. Logically, the energy expenditure dependson the speed of Agent. When running away, Agent spendsa lot of energy. When stopped or at low speed, Agentrecovers some energy.

(a) (b)

Fig. 11 (a) The 3D simulation features a simple environmentpopulated with autonomous creatures. (b) The Cortexionist‘apercu’ provides a visualisation of the controller.

The relevance of the model is proved through acomparison with a traditional connectionist controller.In practice, Agent is firstly confronted with situationswithout the ability to create inner representations, whichcomes down to using a purely reactive controller. Then,in the same conditions, it is given the ability to cre-ate inner representations and possibly reveal a more

9

intelligent behaviour. Regardless of the situation, thecontroller initially contains some reactive behaviours.These behaviours are taught to the agent by means ofa supervised training procedure, which aims to enforcethe following procedural rules.

The first rule makes Agent run away as soon asit collides with another character. This rule is derivedfrom the theory of the personal spaces of Hall [32]. Thesecond rule lets Agent move towards Mother when it issmelled.

Fig. 12 Once adequately trained, the controller holds sev-eral connections between the perceptions and the actions. Redconnections are excitatory, blue connections are inhibitory.In this case, the controller contains the following rules:{collision}→{run away}, {smell}→{¬run away, move toward}.

In practice, these rules appear as excitatory connec-tions between perceptions and actions at the level of thecontroller (in red in Figure 12). Note that since thesetwo rules are learnt in parallel, the second rule impliesthe creation of an inhibitory connection that preventsAgent from running away from Mother (in blue in thesame figure).

Agent’s capacity to survive in the environment ismeasured through two situations.

In the first one, Agent is introduced with Preda-tor. Figure 13 shows the significant steps of the simu-lation. When attacked by Predator, Agent runs away,then stops. After several attacks, the number of whichdepends on the initial amount of vital energy grantedto Agent, it dies.

The second situation is more complex, as Motherand some Fellows are introduced into the environment

(a) (b)

(c) (d)

Fig. 13 A purely reactive controller reveals an incapacity tolearn that the predator is a threat. In (a), Agent is attacked byPredator. (b) The immediate action is to run away from Predator.(c) When Agent is attacked again, (d) it dies.

to help Agent to escape from Predator. Figure 14 de-scribes the most significant steps of the simulation. Inbrief, Agent runs away when it collides with Predator,but also with a Fellow. As a consequence, in the ab-sence of Mother and any kind of protection, the Agentis condemned to die sooner or later under the attacksof Predator.

Although Agent behaves in the exact way it hasbeen trained to, it is rather upsetting to see it unableto learn that Predator is a threat even after being at-tacked several times. The same conclusion is true in thecontext of cooperation as Agent is also unable to relyon a Fellow as long as it has not been explicitly taughtto.

Besides, surviving these situations does not requireelaborate cognitive abilities but rather a better under-standing of the environment. We may ask thereforewhat happens in similar situations if the agent is ableto create and make use of inner representations? Thisis what we are showing below by introducing the Cor-texionist agent in the same situations.

Creating representations is as simple as letting theassociative rule apply while Agent faces different situ-ations.

Figure 15 shows the Cortexionist agent in the firstsituation and describes the significant steps of the sim-

10

(a) (b)

(c) (d)

(e) (f)

Fig. 14 The second situation, featuring the reactive agent, canbe summarised in 6 steps. (a) Agent is introduced in the envi-ronment inside the protective range of its Mother. (b) As soonas Agent —slower than Mother— is distanced, it gets huntedby Predator. (c) Viewing the predator has no consequence, as norule in the behaviour applies in this situation. (d) Predator col-liding with Agent leads to the agent running away, while a nearbyFellow moves towards Agent to offer protection. (e) Fellow col-liding with Agent triggers the same behaviour, leading to Agentrunning away again. (f) Finally, Predator catches Agent on itsway escaping Fellow, or any time later when Agent has run outof energy.

ulation. In brief, Agent first builds a pattern of knowl-edge from the features it perceives from Predator. Whenit is attacked, Agent updates the pattern to take Preda-tor harmfulness into account. Finally, Figure 15.c showsthe resulting pattern in the associative layer of the con-troller.

Figure 16 details the significant step of the Cortex-ionist agent in the second situation. For each step, thecurrent state of the controller is provided. During thefirst 3 steps, Agent builds patterns of knowledge, re-lated to Mother and then to Predator. Note that the

(a) (b)

(c) (d)

Fig. 15 This figure illustrates the behaviour of the Cortexion-ist agent in the first situation. In (a), Agent faces Predator anda new pattern {’tentacle’,’arm’,’purple’} is created, from assem-bling the co-activated neurons related to Predator’s features. In(b), as Agent is attacked, the previous pattern is updated into anew pattern {‘tentacle’,‘arm’,‘purple’,‘collision’}, taking into ac-count this new situation. In (c), while Agent is running away,perceiving Predator, the activation of the ‘tentacle’, ‘arm’ and‘purple’ features leads to the completion of the whole pattern,which includes the ‘collision’ neuron. In turn, the activation ofthis neuron triggers a ‘run away’ behaviour, keeping Agent awayfrom Predator. (d) Shows the final pattern in the controller ofAgent.

(a)

latter overlaps the first, as the ‘tentacle’ neuron is com-mon to both. However, the next steps demonstrate thishas no incidence on Agent’s ability to differentiate thecreatures. Thanks to the Predator pattern, as demon-strated in the first scenario, Agent is able from thispoint to run away from Predator from the mere sens-ing of its morphologic characteristics. When Agent per-ceives a Fellow, the Mother-pattern is activated on thebasis of the perceived characteristics. Recalling this pat-tern causes Agent to move towards Fellow, as it reminds

11

(b)

(c)

(d)

it of Mother. Finally, colliding with Fellow has no moreconsequence, as the ‘run away’ behaviour is innatelyinhibited. Besides, the ‘collision’ neuron alone is alsounable to retrieve the Predator pattern.

Enabling the agent to use an associative rule oninput neurons has no perceptible effect on the proce-dural training, so the reactive behaviours remain un-changed. Yet, using these inner representations duringthe simulation inescapably brings changes in the action-selection. Both cases demonstrate how pattern comple-tion helps retrieving known situations so that the rele-vant actions may be triggered. To the view of the ob-server, such a process may be manifested as a form ofanticipation (in the first situation) or generalisation (inthe second situation).

Different metrics have been investigated to assessthe Cortexionist model’s performance during the simu-

(e)

Fig. 16 This figure shows the significant step of the Cortex-ionist agent in the second situation. For each step, the currentstate of the controller is provided. (a) Agent is introduced in therange of Mother. Two operations occur in the controller. First,the ‘smell’ neuron triggers a ‘move towards’ behaviour. Then, theco-activation of ‘tentacle’, ‘smell’ and ‘green’ are associated into apattern. (b), Agent is now distanced, and faces Predator. A newpattern {‘tentacle’,‘arm’,‘purple’} is created on the associativelayer, partly overlapping the previous one. (c) As Agent is nowbeing attacked by Predator, the pattern related to Predator is up-dated into a larger pattern that integrates the ‘collision’ sensing,just like in the first situation (Figure 15). In parallel, this colli-sion triggers the action to run away. (d) Agent, which is runningaway from Predator, anew perceives a Fellow. On the associativelayer, pattern completion occurs on the Mother-related pattern{‘tentacle’,‘smell’,‘green’} so that perceiving the characteristics‘tentacle’ and ‘green’, associated with the Fellow, activates inturn ‘smell’. The activation of ‘smell’ entails in parallel a ‘movetowards’ action, making Agent leading to the Fellow. (e) Agentcollides with the Fellow, activating the ‘collision’ neuron. How-ever, the ‘smell’ neuron prevents the activation of ‘run away’, sothat Agent stays under the protection of Fellow.

lations. This section presents the most relevant metricsregarding the result we want to stress.

The following curve (Figure 17) compares the meanvital energies of both the reactive and the Cortexionistagents in the first experiment. The data were gatheredfrom several simulations where Predator’s speed wasset to random values around ±40% of a mean value.Statistically, the Cortexionist agent wastes less energythan the reactive agent, as it keeps a safe distance toPredator.

The activity diagrams (Figure 18) compare the be-haviours of the reactive and the Cortexionist agents.They present the effective activity, in terms of firing,of every neurons of the controllers, in such a way thatneurons that participate the action-selection are moreeasily identified. The activity diagrams show very ob-viously that in the same situations (e.g. perceiving thesame features) the two agents exhibit different behaviours.The behaviour has then been modified internally andautomatically, towards a better adaptation of Agent in

12

Time (s)

Mean energy

Fig. 17 This curve shows that Cortexionist agent wastes signif-icantly less energy than the reactive agent, owing to its ability toanticipate the attacks from the predator.

its environment.!"#$%#&&'()*#+,"-",.(/(0$'1,(%.#+,"2

3 4 53 54 63 647"&'(89:

,'1,#+;<' =)';% -'%, >=;%>%' ?%#9 >%=@"&",. 9*'12;"% 9'(%#>>%=+A'%

mother's smelltentacle shape

green colorpurple color

arm shape

run away from closestmove toward closest

collision

(a) the reactive controller

!"#$%#&&'()*#+,"-",.(/(0$'1,(#223+"#,"4

5 6 75 76 85 869"&'(:2;

,'1,#+<=' 3)'<% -'%, >3<%>%' ?%#2 >%3@"&",. 2*'14<"% 2'(%#>>%3+A'%(b) the Cortexionist controller

Fig. 18 Activity diagrams highlight the fact that different be-haviours may be selected on the basis of similar perceptions. Thedifference between the reactive controller and the Cortexionistcontroller is particularly obvious from the black arrow.

During the experimentations, some cases of failureoccurred, revealing the model’s inability to cope withambiguous perception. In the case presented in Fig-ure 19, as soon as introduced in the environment, Agentboth perceives Mother and Predator. As a result, allthe features that belong to Predator and to Mother areregrouped into a single and useless pattern in the con-troller (Figure 19.b), making Agent mistakenly movetowards Predator.

7 Discussion

The Cortexionist architecture has been presented as asimplified yet realistic model of the human memory.Firstly, procedural memory and semantic memory arerepresented, as well as their respective ways of learn-ing. Then, and above all, this architecture differs from

(a) (b)

Fig. 19 In this case, the Cortexionist model does not work as itis unable to deal with ambiguous perception. As Agent is facingboth Predator and Mother (a), a useless pattern is created in thecontroller (b).

previous attempts to model the human memory by notfollowing the traditional multi-store design [31,33], tothe benefit of a more functional approach. Indeed, al-though the multi-store models are still very popularwhen designing computational models — for the conve-nient compatibility between the STM/LTM structureand the way information is processed by a computer—Craik and Lockhart argued the lack of neuro-scientificfoundations of this theory, making any discussion aboutthe storage of information in the brain futile.

In their ‘levels of processing’ theory [34], they ratherinvestigate the way stimuli are processed by the brain.Through experimentations, they find out that a col-lection of random letters is quickly forgotten whereasit can last a whole week if they form a word. Theirconclusion is that the more processing a memory iteminvolves, the longer it is likely to be held in memory.Furthermore, the stimulus seems to acquire more se-mantic meaning according to the time granted to thisprocessing. These results have led Craik and Lockhartto represent human memory as a multi-layered struc-ture, where the deepest layers might present the great-est complexity or abstraction and the best durability.

We believe that taking this theory into account con-stitutes the next step towards making Cortexionist amore realistic model of human memory. Early studieshave led to the creation of Cortexionist 2.0, which addsmore layers, of increasing complexity, in order to pro-duce more complex behaviours (Figure 20). Basically,all the layers are working on the same principle as thecurrent associative layer, but they rely on one anotherto recursively build more complex patterns from theassociation of patterns from the lower layer. The ‘co-activation frame’ is also stretched following the depthof a layer in such a way that more complex patterns canbe created from lower complexity patterns that werenot directly activated at the exact same time, but in a

13

relatively close time-frame. This way, temporal patterncan be created and then used to anticipate previouslylearnt situations.

...

AnticipatedBehaviour

ExtendedBehaviour

ReactiveBehaviour

Fig. 20 The Cortexionist 2.0 model is a prototype where severallayers are stacked in order to allow the creation of knowledge ofincreasing complexity. The complexity is also tied to a time di-mension, so that the extended action-selection also takes on atemporal aspect, allowing the controller to build temporal pat-terns an then anticipate already known situations.

8 Conclusion

To conclude, we have proposed in this paper an orig-inal controller for autonomous creatures where a sim-plified, however realistic model of memory is integratedinto an adaptive reactive architecture. The main nov-elty of this work lies in our approach which is radicallydifferent from traditional computational architectures.First, the knowledge representations are grounded onthe agent’s very perception, defending the idea that thepower of representation is the representation itself, notthe way it can be manipulated. Besides, these represen-tations do not require any additional work from the an-imator. Then, this non-symbolic memory is seamlesslyintegrated into the controller, so that the knowledgedirectly participates in the action-selection. Complexbehaviours are then reflected by the creature’s abilityto take advantage of a better understanding of the en-vironment —namely, the structures and the rules thatgovern it— instead of relying on the animator’s skillsin writing scripts.

Experiments comparing the behaviour of the agentwith and without the ability to create knowledge demon-strate how a better understanding of the environmententails a better adaptation, and therefore intelligent be-haviours. The condition is then the complexity of the

environment, which is guaranteed in our study by in-troducing several autonomous and dynamical creatures.

9 Future work

Considering the simple nature of the experiments pre-sented in the paper, future work may be focused upontesting the controller’s ability to be scaled up to largerenvironments, by using more creatures, a more complexenvironment, and in addition by introducing ambiguouscreatures — for example a creature that looks harmlessbut is actually not.

The experiments consisted in comparing the Cortex-ionist network to a very simple perceptron. However,more recent neural networks, such as recurrent neuralnetworks [35] or NEAT [36], introduce recurrent con-nections in order to endow action-selection with a no-tion of context. Future work plans more comparisonswith these networks, in order to prove the greater effec-tiveness of an associative layer over recurrent connec-tions.

Acknowledgements

All videos from which the pictures of this paper areextracted can be found at the following URL:http://www.irit.fr/~David.Panzoli/

References

1. Stephane Sanchez, Olivier Balet, Herve Luga, and YvesDuthen. Autonomous Virtual Actors. In 2nd InternationalConference on Technologies for Interactive Digital Story-telling and Entertainment, pages 68–78, Darmstadt, Ger-many, June 2004. Springer Verlag LNCS.

2. Catherine Pelachaud. Multimodal expressive embodied con-versational agents. In MULTIMEDIA ’05: Proceedings of the13th Annual ACM International Conference on Multimedia,pages 683–689, New York, NY, USA, 2005. ACM.

3. Alan M. Turing. Computing machinery and intelligence.Mind, 59:433–460, 1950.

4. Rolf Pfeifer and Josh C. Bongard. How the Body Shapesthe Way We Think: A New View of Intelligence. The MITPress, November 2006.

5. Jeff Hawkins and Sandra Blakeslee. On Intelligence. TimesBooks, 2004.

6. Nicolas Lassabe, Herve Luga, and Yves Duthen. A NewStep for Evolving Creatures. In IEEE-ALife’07, Honolulu,Hawaii, pages 243–251. IEEE, 2007.

7. Wei Shao and Demetri Terzopoulos. Autonomous pedestri-ans. Graph. Models, 69(5-6):246–274, 2007.

8. Toni Conde and Daniel Thalmann. An integrated perceptionfor autonomous virtual agents: Active and predictive percep-tion: Research articles. Comput. Animat. Virtual Worlds,17(3-4):457–468, 2006.

14

9. John Funge, Xiaoyuan Tu, and Demetri Terzopoulos. Cogni-tive modeling: Knowledge, reasoning and planning for intelli-gent characters. In the 26th Annual Conference on ComputerGraphics and Interactive Techniques, pages 29–38. ACMPress/Addison-Wesley Publishing Co., 1999.

10. Stephane Donikian. HPTS: A behaviour modelling lan-guage for autonomous agents. Proceedings of the Fifth In-ternational Conference on Autonomous Agents, Montreal,Canada, 2001.

11. Rodney A. Brooks. A robust layered control system for amobile robot. IEEE Journal of Robotics and Automation,pages 14–23, 1986.

12. Marvin Minsky. A framework for representing knowledge.The Psychology of Computer Vision, pages 211–277, 1975.

13. Marcelo Kallmann and Daniel Thalmann. A behavioral in-terface to simulate agent–object interactions in real-time.In Proceedings of Computer Animation 99, pages 138–146,Geneva, 1999. IEEE Computer Society Press.

14. Stevan Harnad. The symbol grounding problem. Physica,42:335–346, 1990.

15. David Panzoli, Herve. Luga, and Yves Duthen. Beyond reac-tive systems: An associative memory for sensory-driven intel-ligent behavior. In Proceedings of the International Confer-ence on CyberWorlds, pages 63–70, Bradford, UK, Septem-ber 2009.

16. James Frye, Rajagopal Ananthanarayanan, and Dharmen-dra S. Modha. Towards real-time, mouse-scale cortical simu-lations. CoSyNe: Computational and Systems Neuroscience,2007.

17. Jean-Arcady Meyer, Agnes Guillot, Patrick Pirim, and AlainBerthoz. Psikharpax: An autonomous and adaptive artificialrat. In Proceedings of ISR2004, the 35th International Sym-posium on Robotics, Paris, 2004.

18. David Filliat and Jean-Arcady Meyer. Active perceptionand map learning for robot navigation. From Animals toAnimats 6, Proceedings of the Sixth International Confer-ence on Simulation of Adaptive Behavior, The MIT Press,Cambridge:246–255, 2000.

19. Benoıt Girard, Vincent Cuzin, Agnes Guillot, Kevin Gurney,and Tony J. Prescott. A basal ganglia inspired model ofaction selection evaluated in a robotic survival task. Journalof Integrative Neuroscience, 2(22):179–200, 2003.

20. Raymond W. Gibbs. A review of ’how the body shapes theway we think: A new view of intelligence’ from Rolf Pfeiferand Josh Bongard. Pragmatics & Cognition, 15(3):610–614,2007.

21. Paul McLean. The Triune Brain in Evolution. Springer,january 1990.

22. Dileep George and Jeff Hawkins. A hierarchical Bayesianmodel of invariant pattern recognition in the visual cortex. InProceedings of the International Joint Conference on NeuralNetworks (IJCNN’05), 2005.

23. Dave Cliff and Geoffrey F. Miller. Co-evolution of pursuitand evasion II: Simulation methods and results. In USAMIT Press, Cambridge, editor, From Animals to Animats4: Proceedings of the Fourth International Conference onSimulation of Adaptive Behavior, pages 506–514, 1996.

24. Demetri Terzopoulos and Xiaoyuan Tu. Artificial fishes:Physics, locomotion, perception, behavior. In SIG-GRAPH’94 Computer Graphics, pages 42–48. ACM SIG-GRAPH, ACM Press, july 1994.

25. Bruce M. Blumberg, Peter M. Todd, and Pattie Maes. Nobad dogs: Ethological lessons for learning in hamsterdam.In Proceedings of the 4th International Conference on theSimulation of Adaptive Behavior, 1996.

26. David E. Rumelhart, Geoffrey E. Hinton, and Ronald J.Williams. Learning representations by back-propagating er-rors. Nature, 323:533–536, 1986.

27. John R. Anderson. The Architecture of Cognition. HarvardUniversity Press, Cambridge, 1983.

28. Michael A. Arbib, editor. The Handbook of Brain Theoryand Neural Networks. MIT Press, Cambridge, MA, USA,2002.

29. Donald Hebb. The Organisation Of Behavior. Wiley, 1949.30. Neal J. Cohen and Larry R. Squire. Preserved learning and

retention of pattern analysing skill in amnesia: Dissociationof knowing how and knowing that. Science, 210:207:209,1980.

31. Richard C. Atkinson and Richard M. Shiffrin. Human mem-ory: A proposed system and its control processes. In K.W.Spence, editor, The Psychology of Learning and Motivation:Advances in Research and Theory Vol. 2, pages 89–195. NewYork Academic Press, 1968.

32. Edward T. Hall. The Hidden Dimension. Anchor Books,1966.

33. Alan Baddeley and Graham Hitch. Working memory. Recentadvances in learning and motivation. New-York. AcademicPress, 8, 1974.

34. Fergus I. M. Craik and Robert S. Lockhart. Levels of pro-cessing: A framework for memory research. Journal of VerbalLearning and Verbal Behavior, 11:671–684, 1972.

35. Sepp Hochreiter and Juergen Schmidhuber. Long short-termmemory. Neural Computation, 9:1735–1780, 1997.

36. Kenneth O. Stanley and Risto Miikkulainen. Evolving neu-ral networks through augmenting topologies. EvolutionaryComputation, 10(2):99–127, 2002.