A Cell-based Developmental Modelto Generate Robot Morphologies

Sylvain Cussat-BlancDEMO Lab, Volen National Center for Complex

System - Brandeis University MS018415 South street, Waltham, MA 02454, USA

cussat@brandeis.edu

Jordan PollackDEMO Lab, Volen National Center for Complex

System - Brandeis University MS018415 South street, Waltham, MA 02454, USA

pollack@brandeis.edu

ABSTRACTThis paper presents a new method to generate the bodyplans of modular robots. In this work, we use a develop-mental model where cells are controlled by a gene regulatorynetwork. Instead of using morphogens as in many existingworks, we evolve a more flexible “hormonal system” thatcontrols the inputs of the regulatory network. By evolvingthe regulatory network and the hormonal system in parallelwith a blind watchmaker, we have generated various virtualrobots with interesting inherent properties such as regular-ity and symmetry. The prototypes of the robotic blocks thatwill be used to actually build the real machines are also pre-sented in this paper.

Categories and Subject DescriptorsI.2.9 [Artificial Intelligence]: Robotics

KeywordsModular robots, Cell-based developmental model, Gene reg-ulatory network, Hormonal system, Body plans

1. INTRODUCTIONModular robots built over the last several years have got-

ten more and more complex. Usually based on cubic blocks,they embed complex mechatronics to make them more accu-rate, more efficient and more diversified [25]. In this domain,the “Molecube” may be seen as an example. First based ona cube able to turn around its central axis to demonstratelimited self-replication, Lipson and his team went on to de-velop new modules to build more complex machines [26].For example, we can cite the gripper modules able to pick upobjects, the wheel cubes, the camera cubes, etc. To controlthese blocks, a neural network is then evolved. However, theuser has to imagine the morphology of the robots by himself.It means that, in case of modification of the machine’s pur-pose, a human has to redesign the morphology of the robotbefore evolving its controller. In 2007, Christensen proposed

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.GECCO’12, July 7-11, 2012, Philadelphia, Pennsylvania, USA.Copyright 2012 ACM 978-1-4503-1177-9/12/07 ...$10.00.

a robotic approach where the behavior of the machine, alsocomposed of rotating cubic blocks, is strongly linked to itsmorphology [5]. Indeed, instead of having a fixed morphol-ogy, the robot is self-reconfiguring in order to accomplish itsfunction. This method generates a very interesting property:generated robots are capable of self-healing, as multicellularorganisms are.

On the other side, many works have been done in sim-ulation to generate virtual modular robots. The first ma-jor work evolving realistic virtual robots is Karl Sims in1994 [24]. It took another six years before evolved robotstook a step from virtual to reality [18]. Sims used a graph-based genotype-phenotype map to generate a morphologycontrolled by a neural network, while Lipson & Pollack di-rectly evolved the morphology. Both systems are coevolvedso that the controller is as much as possible adapted to themorphology. L-Systems have also been used for the samepurpose. Komosinski has made one of the most advancedwork in this domain. Here again, a neural network is coe-volved with the morphology generator (a L-System) to pro-duce artificial robots. More recently, artificial Gene Reg-ulatory Networks (GRN’s) appear to be able, at the firsthand, to generate complex morphologies when they controla developmental model [11, 3] and, at the second hand, tocontrol different kinds of agents [16, 19]. Schramm was evenable to grow a digital aquatic worm that moves by oscillationgenerated by the same regulatory network [22].

However, few of these works have been designed to ac-tually build a real robot. With this idea in mind, Hornbyproposed a model where a L-System is evolved to controla “turtle” factory which could produce a virtual robot con-trolled by a neural network [14]. Here, the best robots haveactually been built by hand and were capable of moving likethe virtual robots.

The main motivation of this paper is to use a cell-baseddevelopmental system driven by a gene regulatory networkto generate real robot morphologies. To present this newmethod to generate robots’ body plans, the paper is orga-nized as follows. The next section presents the prototypesof real robotic blocks that will be used to build the robots.Section 3 presents the developmental model, the gene reg-ulatory network and the hormonal system. In section 4,the blind watchmaker interactive evolutionary method is de-scribed. Section 5 presents a set of results obtained with thismodel and proposes a discussion about the complexity of themorphologies and the emergent properties obtained such asregularity and symmetry. The paper concludes with possibleextensions of this work.

537

Noop block Hinge block Motor block Dovetail latch

Actuator Servo motor

Figure 1: The robots are composed of three differentblocks (from left to right): a noop block, a hinge blockand a motor block. Blocks are tied together by theuse of dovetail latches inserted between the blocks.

2. THE ROBOTIC BLOCKSIn this work, we took inspiration from living organisms to

design our machine. An organism is usually comprised ofbillions of functional units: the cells. Cells can have variousfunctions (neural, muscular, etc.). Thus, our machine usesmultiple blocks that have different functions. The blocksmust be easy to assemble to be able to conceive an automaticassembly process. When a machine is built, its morphologywill not evolve anymore (as many grown living beings donot modify their morphology). With this constraint, theblocks do not need a dynamic linking and unlinking system.Finally, each block must be cheap to produce to keep therobot’s overall cost low.

With all these requirements, we have designed each blockas a cube, scaled to 2 inches (about 5cm) per side. As pre-sented in figure 1, three different units are used in the work:(1) A noop block is used to build the structure of the robots(such as bones are the structure of the body). This blockdoes not have any function, except the capacity to be linkedto other blocks. (2) A hinge block can rotate around a cen-tral axis within the range [−π/3;π/3]. This block does notmove by itself, but can be actuated by a neighboring motorblock through two actuators, as tendons are actuating a realarticulation. (3) A motor block actuates the hinge block.The actuators protubing from the hinge blocks are tied tothe motor so that the motor can actuate the hinge.

This choice to separate the hinge and the motor comesfrom a biological observation. In Nature, most articula-tions and muscles are separated. They are linked togetherwith tendons. With this method, muscles can have vari-ous shapes and sizes, according to local constraints (space,needed strength, etc.). We decided to use this idea for ourbuilding blocks for two main reasons. First, motors are usu-ally the heaviest parts of the robots. Modifying the positionof the motors in the robot can modify its center of mass,which can be relevant. Secondly, we can also imagine thepossibility to increase the strength of a joint by having mul-tiple motors actuating the same hinge (as multiple muscularcells actuate the same articulation).

Because, our robotic system does not reconfigure itselfsince each block’s position is given by the evolved body planof the robot (its morphology), we can keep the linking sys-tem as simple as possible. As presented in figure 1, it consistsin a dovetail latch slipped between two blocks.

These blocks are currently at the stage of prototypes butare close to a final version. A 3-D printer is used to produce

the plastic structure of the blocks and the dovetail latches.Servo motors (Futaba S3102) are embedded in the motorblocks and are linked to the hinge block via the actuators.These actuators are still the main issue of realization: theyneed to be strong, light and inelastic. Three solutions havebeen tested: using nylon strings, steel strings or having theactuators embedded to the hinge, as presented in figure 1.Because the two first solutions provide very unprecise move-ment due to the complexity of the strings’ length calibra-tion, the last solution should be the one adopted for thefinal robots. It requires a more elaborated conception of thehinge blocks but strongly simplify the assembly process ofthe blocks and increase the precision of the movement of theglobal articulation.

To produce the plans of the robots, we propose a novelapproach based on a developmental model in which cells arecontrolled by a gene regulatory network coupled to a novelhormonal system. In comparison to other methods such asGraphtals or L-Systems, this approach has the advantage tobe biologically plausible. Moreover, some properties such asregularity and symmetry emerge from the system, withoutany prior encoding. The next section presents this model.

3. THE DEVELOPMENTAL MODEL

3.1 The cells and their environmentThe developmental model is based on cells that act in a

3-D discrete environment. Each cell has a unique identi-fier given at its birth, a division plan and a position in theenvironment. Two main types of cells are available: (1) un-differentiated cells or stem cells that can divide as much asthey want to and (2) differentiated cells coming from a spe-cialization of a stem cell to one of the three following typesof cells: B-cells, H-cells or M-cells. When a cell is differen-tiated, it cannot divide anymore. The use of stem cells todrive the organism’s growth has already been successfullyused to developed complex shapes by Fontana in [12].

At each time step of the simulation, the stem cells canperform one of the following actions:

• Symmetric division: when a stem cell performs thisaction, a new stem cell is positioned in the positionof the mother cell while the mother cell migrates indirection to its division plan. Whereas the mother cellkeeps the same identifier, the new cell receives a newunique one. Its initial division plan is oriented as itsmother’s one.

• Asymmetric division: a stem cell can divide to createa new differentiated cell. In this case, the mother cellfirst produces an exact copy (same internal state) ofitself in the direction indicated by its division plan.While the newly created cell stays a stem cell, themother cell differentiates to a B-cell. Thus, this cellcannot divide anymore.

• Differentiation: a stem cell can become a B-cell. Be-cause a differentiated cell cannot divide, this actionstops the proliferation of a stem cell.

• Rotation: a cell can reorient its division plan. Whenit choses this action, it can turn in one of the fourfollowing directions: right, left, top or bottom. Whileturning, the cell stays at the exact same position.

538

• Special division: a cell can divide asymmetrically twiceduring one time step. In this case, it first produces aH-cell at the initial position, a M-cell one voxel awayfrom the initial position in the direction of its divisionplan and a copy of itself two voxels away.

Naturally, two cells cannot be in the same voxel of theenvironment. To satisfy this requirement, all inconsistentactions are disabled at a time step of the growth. For exam-ple, if a cell wants to divide, the position in direction of thedivision plan is first checked. If the position is not free (inother words, already occupied by another cell), the divisionis disabled. The cell will have to choose another action. Atleast two actions are guaranteed to be always available: ro-tation and differentiation. Indeed, both actions only affectthe cell itself: no new voxels are necessary.

After the growth of the organism from a single cell to aconnected set of cells, each cell is translated to a roboticblock: B-cells and stem cells are interpreted as noop blocks,M-cells produce motor blocks, and, finally, H-cells produceshinge blocks.

To evaluate the morphologies generated, a virtual robot isbuilt and dropped down in a 3-D physics environment. Thissimulator is used to visualize the produced robots before ac-tually building them. We use the well-known Bullet physicsengine coupled with OpenGL for the graphic visualization.

To select an action, cells are using a gene regulatory net-work. The next section details our model based on a simpli-fication of Banzhaf’s GRN [1].

3.2 The gene regulatory network

3.2.1 Background on GRN’sIn Nature, each cell of a living organism has a gene regula-

tory network (GRN) that controls its behavior. The possibleactions are coded by genes in its DNA and their expressionsare controlled by the regulatory network [8]. The GRN usesexternal proteic signals from the cell’s environment to ac-tivate or inhibit the transcription of genes. Cells can col-lect external signals thanks to protein sensors localized onthe cell membrane. The expression of genes determines thecell’s behavior.

Artificial regulatory networks are based on the same prin-ciple. Torsten Reil was one of the first to propose a biolog-ically plausible simulation of a GRN [20]. He defined thegenome as a bit string. In his model, each gene started withthe particular sequence of 4 bits, named the “promoter”.The dynamics of this regulatory network is very close toBanzhaf’s, detailed below.

Banzhaf proposed an artificial regulation network modelstrongly inspired by real gene regulation [1]. The aim ofthis model was to study the behaviors generated by a bi-ologically plausible regulatory network. He has generatedmany random bit strings and observed various behaviorsfrom oscillators to faster or slower transitory patterns.

Regulatory networks have been proven successful to growcell based artificial creatures with more or less complex mor-phologies [11, 10, 15]. More recently, regulatory networkshave also been used as behavior generators to solve the polebalancing problem [19] or to control a foraging virtual robot[16]. Schramm also used a GRN to control the morphologydevelopment and the behavior of swimming worms [22].

However, no real robotic application has been yet pro-posed for this system. Our work proposes to use a simplifi-

cation of Banzhaf’s regulatory network to control the cellsof our developmental model.

3.2.2 Presentation of Banzhaf’s regulatory networkBanzhaf’s GRN may be currently one of the best models.

It has been designed to be as close as possible to a real generegulatory network. As DNA is composed of a sequence ofnucleotides, Banzhaf’s network is encoded within a sequenceof bits. As a real gene starts with the particular sequenceof nucleotides, a gene in Banzhaf’s network starts with aparticular sequence of 8 bits named the “promoter”. A geneis then encoded next to this sequence by 5 32-bit integers,named the “sites”. This mechanism allows the generationof a variable number of genes in a fixed size chromosome.However, as in Nature, it also generates a certain amount ofnoncoding DNA, the probability to have a promoter beingvery low (2−8). This noncoding DNA (98% of human DNA)is thought to be used in Nature to protect the genome frommutation by lowering the probability that a mutation willaffect a coding nucleotide.

Banzhaf’s model has not been designed to be evolved norto control any kind of agent. However, Nicolau used anevolution strategy to evolve the GRN to control a pole-balancing cart [19]. Even if the cart behavior has been shownconsistent within its environment, the evolution of the GRNhas been an issue. In our opinion, the difficulty of the evolu-tion is due to: (1) the noncoding DNA and (2) the dynamicsof the network. The next section details these two pointsand proposes a possible adaptation of the model to be moreefficient as a computational model.

3.2.3 Adaptation of Banzhaf’s model to an efficientcomputational model

Encoding of the network

As presented above, the noncoding DNA, which may havepositive benefits in natural systems, impedes the evolutionof artificial GRNs.

To get rid of the noncoding DNA, we propose a new en-coding of the network. In our model, each protein is encodedas an indivisible component composed of four attributes:

• The protein identifier coded as an integer between0 and p. The upper value p of the domain can bechanged in order to control the precision of the GRN.In Banzhaf’s work, p is equal to 32, which is the sizeof a site. In our work, we have decided to increase theprecision by setting p to 64.

• The enhancer identifier coded as an integer between 0and p. The enhancer identifier is used to calculate theenhancing matching factor between two proteins.

• The inhibitor identifier coded as an integer between 0and p. The inhibitor identifier is used to calculate theinhibiting matching factor between two proteins.

• The type determines if the protein is an input protein(whose concentration is given by the environment ofthe regulatory network and which regulates other pro-teins but is not regulated), an output protein (whoseconcentration is used as output of the network and whois regulated but does not regulate other proteins) or aregulatory protein (internal protein that regulates andis regulated by other proteins).

539

The GRN is defined as a set of these proteins. A GRNcan have a variable number of proteins. To be evolved, theGRN is encoded in a genome defined as a variable lengthchromosome of indivisible proteins. Each protein is encodedwithin four integers: three between 0 and p for the threedifferent identifiers and one between 0 and 2 for the typeof the protein. If an evolutionary algorithm has to evolvethis chromosome, the modification operators have to be re-defined. First, the crossover consists in exchanging subpartsof two different networks. Because proteins are indivisible,the crossover points have to be chosen between two proteins.It ensures the integrity of each sub-network. The local con-nectivity is thus kept. Only new links between the differentsub-networks are created. The mutation can be applied inthree equiprobable ways: mutating an existing protein byrandomly changing one of its four integers, adding a newprotein randomly generated or removing one protein ran-domly chosen in the network.

Dynamics of the network

The dynamics of the network is, in our opinion, a cru-cial point that determines the reactivity of the network. InBanzhaf’s network, the enhancing dynamics of each proteinare given by the following equation:

ei =1

N

N∑j

cjeβu+

j −u+max

where ei is the enhancing value for a protein i, N is thenumber of proteins in the network, cj is the concentrationof protein j, u+

j is the enhancing matching factor betweenprotein i and protein j (calculated by counting the num-ber of different bit of the proteins signatures) and u+

max isthe maximum enhancing matching factor observed. β is acontrol parameter described later. The modification of theconcentration of a protein i is then given by the differentialequation dci/dt = δ(ei−hi)ci−Φ, where ei is the enhancingsignal previously presented, hi is the inhibiting signal (cal-culated exactly as the enhancing signal by using inhibitingmatching factors instead of the enhancing matching factors),ci is the current concentration of the protein i and Φ is afunction that keeps of the sum of all protein concentrationsequal to 1.

As presented in this last equation, the evolution of theconcentration of a protein i is directly related to its currentconcentration. In other words, if a protein’s concentrationappears to be null, this protein will never be produced any-more. This is problematic because, in the case of a compu-tational model, it means that an output completely disabledat a particular time of the simulation will never be activatedanymore. With this observation, we propose to change thedifferential equation by the following one:

dcidt

=δ(ei − hi)

Φ

With this new equation, the current concentration of theprotein i does not intervene anymore. The scaling functionis also divided instead of subtracted in order to have a moreproportional scaling factor.

With this new encoding and these new dynamics, the reg-ulatory network should be easier to evolve and more reactiveto its environment. Next section presents how this regula-tory network is embedded within the cells of our develop-mental system.

GRN

Protein 1id, en, in:

[0,64]type=output ...

Protein 9id, en, in:

[0,64]type=output...

Protein 10id, en, in:

[0,64]type: {input, regulatory}

Protein nid, en, in:

[0,64]type: {input, regulatory}

Figure 2: The chromosome that encodes the GRNis subdivided into two parts: the first nine proteinsare output proteins and the remainder of the chro-mosomes encodes input and regulatory proteins.

3.2.4 Encapsulation into the cellsCells have 5 different actions: symmetric division, asym-

metric division, differentiation, division plan reorientation(rotation) and special division. However, if a cell choses torotate, it also has to decide in which direction it wants to(left, right, top or bottom). To encode this in the GRN,we decided to subdivide the outputs into two groups of out-put proteins: the first group contains 5 output proteins thatmatch with the 5 possible actions and the second group con-tains 4 output proteins that encodes the 4 possible rotationdirections when the rotation is selected in the first group ofproteins. Thus, 9 output proteins are necessary to encodethe possible actions of the cells.

As illustrated by figure 2, these 9 output proteins are en-coded at the beginning of the genome. Their three identi-fier (protein, enhancer and inhibitor) can be mutated butthe type cannot. The remaining elements of the genomeare composed of 25% of input proteins and 75% of regula-tory proteins (empirically chosen1). It can be mutated andcrossed over as previously presented.

At each step of the simulation, the cells execute the reg-ulatory network for 25 steps (found to be within reasonablerange during prelimanery tests) and select the action corre-sponding to the maximum protein concentration of the first5 proteins: if the first protein has the maximum concen-tration, the cell divides symmetrically, if the second proteinhas the maximum concentration, the cell divides asymmetri-cally and so on. When the rotation action is chosen, it givesaccess to the second group of output proteins. The cell cal-culates the maximum value of the last 4 output proteins’concentrations to determine in which direction to turn.

The coefficients β and δ presented in the dynamics modelare also encoded in the genome. They modify the dynamicsof the network. β affects the importance of the matching fac-tor and δ affects the level of production of the protein in thedifferential equation. The lower both values, the smootherthe regulation. At the opposite, the higher the values, themore sudden the regulation. Both values are encoded asa second independent chromosome that only contains bothvalues. They are coded with double-precision floats in [0.5; 2](empirically chosen1).

Finally, we must provide inputs to the regulatory networkin order to be able to select an action at every time step ofthe growth. In most developmental models, cells can pro-duce morphogens such as in [17, 15]. They are informativemolecules that diffuse in the environment and are the in-

1Empirically chosen values have been determined by a seriesof pre-tests to find a reasonable range.

540

Cell age Cell identifie

r

Input proteinconcentration

Cell age Cell identifie

r

Input proteinconcentration

Figure 3: Examples of generated inputs with Beziersurfaces. They show the diversity of signal we canobtain by using this generative method.

puts of the GRN. In few other models, the morphogens arehand-coded in the environment [4, 7]. This method is lessrealistic than the first one but strongly reduces the com-plexity of the simulation. Because of its complexity, thefirst method generally leads to simple shapes such as 2-Dcolored rectangles, 2-D salamanders or 3-D tubes. The sec-ond method make possible to generate more complex 3-Dcolored structures such as cubes, diamonds or spheres. Inthis work, we choose an intermediate solution. Cells have ahormonal system that produces the inputs of the regulatorynetwork. The next section details this system.

3.3 The hormonal systemThe genotype of our developmental model also describes a

novel hormonal system. This hormonal system generates theinputs to the gene regulatory network based on the timestepand index number of the cell. In order to be efficient, eachinput protein of the GRN must have a different hormonalproduction profile that provides the concentration of the in-put protein all along the simulation. Moreover, this profilemust also be different for each stem cell in the organism.Indeed, if all the cells would have the same profile, it wouldnot allow any variation of their behaviors: they would allperform the same sequence of actions. In summary, theconcentration ci of an input protein i is related to two pa-rameters: the current time step t of the simulation and theidentifier of the cell in which the concentration is calculated.

Many models of artifical hormones exist in the literature.From those applied to robotics, we can cite AHHS used tocontrol the behavior of Symbrion and Replicator reconfig-urable robots [13]. Here, an hormone is diffusing throughtthe robot and is used for local communication purpose.

Thus, the hormonal profile of an input protein can becoded by a 3-D surface. The first two dimensions of thesurface correspond to the simulation time in abscissa andthe identifier of the cells in ordinate. The third dimensiongives the concentration of the corresponding input protein’sconcentration according to the two first dimensions. Theglobal hormonal system is thus defined as a set of p 3-Dsurfaces. Each surface is associate with the identifier of theinput protein (which varies between 0 and p, p being theprecision of the GRN).

To generate these surfaces, we used Bezier’s algorithm [2].Originally used to design car bodyworks, it allows the gener-ation of a surface by the mean of a given set of checkpoints.In this work, we use a Bezier surface of order (n = 10,m =10). This order corresponds to the number of checkpoints.For a (10, 10) order surface, (10 + 1) ∗ (10 + 1) = 121 check-point ki,j are necessary. With these checkpoints, the z co-ordinate of every point of coordinate (x, y, z) of the surface

is given by the following equation:

z =

n=10∑i=0

m=10∑j=0

Bni (x)Bmj (y)ki,j

where Bni (x) is the Bernstein polynomial:

Bni (x) =n!

i!(n− i)!xi(1 − x)n−i

Figure 3 presents some examples of randomly generatedsurfaces using this method. They show the diversity of pos-sible landscape generated with only 121 checkpoints.

Each possible input protein must have a Bezier surfaceassigned. To do so, we have decided to allocate a surfaceto each possible protein identifier. When the identifier ofan input protein is changed by mutation to another value, anew surface is thus assigned. With this method, p surfacesare necessary for a GRN of precision p (a protein identifieris coded with an integer between 0 and p. Here p = 64).To encode this system into the genome, the checkpoints areencoded in the third chromosome that consists in a commonset of 121 ∗ p double-precision floats. Each value can varyin the range [−1.0; 1.0]. This chromosome can be classi-cally crossed over and mutated. In order not to overflow theregulatory network with the input proteins, the input con-centrations given by Bezier surfaces are normalized between0 and 0.05.

3.4 Summary of the modelTo summarize, cells are acting in a 3-D discrete environ-

ment. Stem cells can divide symmetrically, asymmetricallyor twice in the same time step to produce an articulation.They can also reorient their division plan and differentiate toB-cells. To control their developmental behavior, cells havea gene regulatory network coupled to a hormonal system.We have proposed an adaptation of Banzhaf’s well-knownGRN to make it more efficient for computational purposes.

Figure 4: 9 robots evolve at the same time in thephysics simulator. A 6-second phase oscillator gen-erates the robots’ movements. The user can selectthe robot that he wants to evolve. The evolutiononly consists in mutating 10% of the genome. Spac-ing between robots has been reduced for this par-ticular picture.

541

Robot R1 Robot R2 Robot R3 Robot R4 Robot R5Generation 9 Generation 11 Generation 14 Generation 14 Generation 30

Figure 5: Examples of robots obtained in a single run of the blind watchmaker. Generated robots arecomplex and diversified, especially in few generations with few individuals evaluated at each generation (9robots only).

The hormonal system uses Bezier surfaces to generate theinputs to the GRN. Each input protein is produced by ahormone, a 3-D surface indexed by the simulation time andthe cell identifier.

In order to evolve the organisms (and thus the robots’body plan), both systems are encoded in a genome com-posed of three independent chromosomes: the first chromo-some encodes a set of indivisible proteins used to build thearchitecture of the GRN, the second chromosome encodesthe dynamics parameters and the third chromosome encodesthe checkpoints necessary to generate the hormonal system’sBezier surfaces.

In order to evaluate the complexity of possible morpholo-gies generated by this model, we have decided to use a BlindWatchMaker instead of a classical evolutionary algorithm.The aim is to study the capacity of the GRN to inherentlyproduce properties that other models usually include suchas symmetry or regularity. We also want to verify that oursystem is able to generate a wide variety of morphologieskeeping in mind its possible evolution with an evolutionaryalgorithm.

4. EVOLUTION OF THE ROBOTS:THE BLIND WATCHMAKER

The Blind WatchMaker is an interactive evolutionary methodfirst proposed in 1986 by Richard Dawkins [9]. He originallyused this method to sustain the theory of natural evolu-tion using a pedagogical model called biomorphs, fractal-likecreatures generated with a small set of genes. This methodgave birth more recently to the field of interactive evolu-tion. Many applications are nowadays based on this princi-ple to solve various problems. It has for example been used

with genetic programming to generate realistic camouflage[21], or with HyperNeat to generate 2-D pictures [23] or 3-Dshapes [6].

In this work, we first generate 9 random genomes. Thedevelopmental model is then executed to generate the cor-responding virtual robots. When the robots are created,they are dropped down in the physics simulator in 9 differ-ent points of the environment so that they cannot interactwith one another. Figure 4 illustrates nine random robotsevolving in the physics simulator. In this particular exper-iment, we have limited the number of developmental stepsto 20 and the number of stem cells to 10 in order to controlthe scale of morphologies. Later, these limits will be neededbased on the actual availability of robotic cells, but mightbe based on simulating biologically plausible mechanisms.

During the first 5 seconds, the robots do not move in orderto stabilize their structure on the floor. Then, to make therobots move in the environment, an oscillator controls themovement of every hinge of the robots: they rotate clockwisefor 3 seconds (the rotation angle of a hinge cannot be above−π/3) and then counterclockwise for 3 seconds (within aπ/3 limit). A third of the oscillators are inverted: whenit should turn clockwise, an inverted hinge actually turnscounterclockwise and vice versa. Even though this methodgenerates simple movements, it is sufficient to visualize thepotential of the generated morphologies.

The user can use window controls to navigate from a robotto another to observe its movement and its properties. Theuser can also select the robots he wants to evolve. Theapplication then generates 9 new robots by mutating 10%of the selected robot’s genome. We have decided not to usethe crossover operator to enhance the diversity of generated

542

robots. For the same reason, the mutation rate has beendeliberately chosen high.

With this method, we have generated a pool of diversi-fied robots. The next section presents some of them anddiscusses properties of robots obtained.

5. RESULTS AND DISCUSSIONFigure 5 presents 5 robots obtained during the same run

of the interactive evolutionary algorithm. These examplesshow the diversity of morphologies obtained.

The first observation we can make about these robots isthe capacity of the regulatory network to generate robots ofdifferent sizes. The GRN is able to generate as well smallrobots (such as robot R4) or bigger ones (such as R1 or R2).This is mainly possible by using the differentiation actionthat stops the proliferation of the stem cells.

Moreover, interesting properties emerge during the evolu-tion. After only 9 generations, the robot R1 already presentssome regularity in its morphology. We call a regularity a setof cells with the same organization (relative positioning anddifferentiation of the cells) that appears at different placesin the organism. For example, the robot R1 is composed of 3similar “legs” and two identical structure build with double-row structure of cells (see the second picture of R1 on figure5). The robot R2 also has some regularities. It is composedof 2 long “legs” of the same size. The same observation canbe done on robots R3 and R5 (2 small legs on one side forthe robot R5).

In some cases, these regularities can be very interesting:on the robot R4, it produces a double row of H-cells and M-cells, which generates a double joint in the robot. This struc-ture multiplies the strength of the resulting joint. Figure 6presents the growth sequence of the robot R4. The double-row cell structure appears at the beginning of the growth,from step 1 to 3. The initial cell first divides. Then bothcells perform the same actions for 2 steps, before changingtheir behaviors, certainly because of a change in the hor-mone profile of some input proteins at this particular pointof the simulation. While the first cell produces the left partof the robot, the second cell quickly stops its developmentafter few more (parasitic) reorientations of its division plan.

As proven by the blind watchmaker approach, some of therobots can be symmetric such as robots R2 and R3 on figure5. This property is particularly interesting because it is usu-ally expected in robot’s morphologies. A symmetric machinewill have more chances to be stable. The symmetry seemsto be achieved using a similar method which achieves regu-larity – but with a rotation before two equivalent sequencesof actions are performed.

The use of a developmental model to generate robot bodyplans seems to be a promising approach. Some desirableproperties such as regularity and symmetry is easily ob-tained with this system. As presented above, we have ob-tained these properties in few generations with a very smallpopulation. This is not necessarily the case in works basedon HyperNEAT or L-Systems which requires many moregenerations of evolution. Moreover, the hormonal systemprovides for much more creativity than a classical morphogensproduction regulated by the GRN. For now, the profiles ofthe hormone are encoded in the genome, but we could eas-ily imagine using hormones and morphogens simultaneously.Morphogens produced by the cells could, for example, con-trol the production profile of the hormones.

Step Cell 1 Cell 2 Cell 3 Cell 4 Organism

0 Birth

1 SyD2 Birth

2 TurnB TurnB

3 SpD SpD

4 SyD3 TurnB Birth

5 TurnB TurnB SyD4 Birth

6 TurnB Diff Diff TurnB

7 TurnB - - Diff

8 SpD - - -

9 AsD - - -

10 Diff - - -

Figure 6: Actions performed by the stem cells only.Each line corresponds to a growth step. Signi-fication of the acronyms: SyDx=Symmetric divi-sion to create a new cell x; AsD=Asymmetric Di-vision, SpD=Special division, Diff=Differentiationand TurnB=Turn to Bottom

6. CONCLUSION AND PERSPECTIVESThis paper presents an innovative system to generate robot

body plans. It is based on a developmental model in whichcells are controlled by a gene regulatory network. We havebased our GRN model on Banzhaf’s by modifying the en-coding of the GRN into a chromosome without any non-coding DNA and by modifying its dynamics. The resultingGRN is easier to evolve and is more adapted to a compu-tational purpose. Instead of using morphogens as in mostdevelopmental models, we have decided to provide input tothe GRN using a hormonal system which is differentiatedby the cell identifier, the input protein, and the simulationtime. These hormone profiles are generated by the use ofBezier surfaces controlled by a finite set of 121 checkpointswhich are evolved with the GRN.

We have produced various virtual robots by evolving inthe same time the GRN and the hormonal system with ablind watchmaker. Resulting robots have various sizes, fromsmall to very large, and two interesting properties emergefrom the systems: many creatures have regularities in theremorphologies, such as “legs” or double-joints, and some ofthem are symmetric. This is possible because of the oscilla-tions, which is a well-known property of GRN system.

We anticipate much future work on this model. First, wehave to produce more robotic blocks and actually build thevirtual robots into reality. The prototypes of the roboticblocks already exist but the building of a whole robot willlikely require some modifications and re-engineering of therobotic blocks (actuators, motors, dovetail latches, etc.) as

543

well as cause constraints to be added to the physics simulator(motor strength, block friction, etc.).

The joints’ controllers are also something we have to workon. They only consist of oscillators moving the joints clock-wise and then counterclockwise. Two main solutions arecurrently considered to improve the controller. First, wecan imagine that the same GRN also controls the joints whenthe growth of the robot is finished. This method has alreadybeen applied with success by Schramm in [22]. The secondsolution could be to plug a neural network or an equivalentmethod to the joints. Here, two solutions are possible: at thefirst hand, the neural network and the developmental modelcan be evolved at the same time or, at the second hand, itcan be evolved after the evolution of the morphology.

In order to generate efficient robots, their genomes willhave to be evolved by an evolutionary algorithm. The fitnessfunction continues to be the main issue, as the field needs tomove beyond linear movement in a simulated environment.To have other kinds of morphologies, we are considering in-cluding complexity, symmetry and regularity measures tothe fitness function as well as utilizing multi-objective tech-niques for the co-evolution of body and behavior.

7. REFERENCES[1] W. Banzhaf. Artificial regulatory networks and genetic

programming. Genetic Programming Theory andPractice, pages 43–62, 2003.

[2] P. Bezier. Numerical control; mathematics andapplications. John Wiley & Sons, 1972.

[3] J. Bongard and R. Pfeifer. Evolving complete agentsusing artificial ontogeny. Morpho-functional machines:The new species (designing embodied intelligence),pages 237–258, 2003.

[4] A. Chavoya and Y. Duthen. A cell pattern generationmodel based on an extended artificial regulatorynetwork. Biosystems, 94(1-2):95–101, 2008.

[5] D. Christensen. Experiments on Fault-TolerantSelf-Reconfiguration and Emergent Self-Repair. InIEEE Symposium on Artificial Life (ALIFE’07), pages355–361, 2007.

[6] J. Clune, B. E. Beckmann, P. K. McKinley, andC. Ofria. Investigating whether hyperneat producesmodular neural networks. In Proceedings of the 12thannual conference on Genetic and evolutionarycomputation, GECCO ’10, pages 635–642. ACM, 2010.

[7] S. Cussat-Blanc, N. Bredeche, H. Luga, Y. Duthen,and M. Schoenauer. Artificial gene regulatorynetworks and spatial computation: A case study. InProceedings of the European Conference on ArtificialLife (ECAL’11). MIT Press, Cambridge, MA, 2011.

[8] E. H. Davidson. The regulatory genome: generegulatory networks in development and evolution.Academic Press, 2006.

[9] R. Dawkins. The blind watchmaker. LongmanScientific & Technical, 1986.

[10] R. Doursat. Organically grown architectures: Creatingdecentralized, autonomous systems by embryomorphicengineering. Organic Computing, pages 167–200, 2008.

[11] P. Eggenberger. Evolving morphologies of simulated3d organisms based on differential gene expression. InProceedings of the Fourth European Conference onArtificial Life, pages 205–213. MIT Press, 1997.

[12] A. Fontana. Devo co-evolution of shape andmetabolism for an artificial organ. Artificial Life XII,H. Fellermann et al., Eds. Boston: MIT Press, pages16–23, 2010.

[13] H. Hamann, J. Stradner, T. Schmickl, andK. Crailsheim. A hormone-based controller forevolutionary multi-modular robotics: From singlemodules to gait learning. In IEEE Congress onEvolutionary Computation, pages 1–8, 2010.

[14] G. Hornby, H. Lipson, and J. Pollack. Generativerepresentations for the automated design of modularphysical robots. Robotics and Automation, IEEETransactions on, 19(4):703–719, 2003.

[15] M. Joachimczak and B. Wrobel. Evo-devo in silico: amodel of a gene network regulating multicellulardevelopment in 3d space with artificial physics. InProceedings of the 11th International Conference onArtificial Life, pages 297–304. MIT Press, 2008.

[16] M. Joachimczak and B. Wrobel. Evolving GeneRegulatory Networks for Real Time Control ofForaging Behaviours. In Proceedings of the 12thInternational Conference on Artificial Life, 2010.

[17] J. Knabe, M. Schilstra, and C. Nehaniv. Evolutionand morphogenesis of differentiated multicellularorganisms: autonomously generated diffusiongradients for positional information. Artificial Life XI,11:321, 2008.

[18] H. Lipson and J. Pollack. Automatic design andmanufacture of robotic lifeforms. Nature, 406:974–978,2000.

[19] M. Nicolau, M. Schoenauer, and W. Banzhaf.Evolving genes to balance a pole. GeneticProgramming, pages 196–207, 2010.

[20] T. Reil. Dynamics of gene expression in an artificialgenome - implications for biological and artificialontogeny. Advances in Artificial Life, 1999.

[21] C. Reynolds. Interactive evolution of camouflage.Artificial Life, 17(2):123–136, 2011.

[22] L. Schramm, Y. Jin, and B. Sendhoff. Emergedcoupling of motor control and morphologicaldevelopment in evolution of multi-cellular animats.Advances in Artificial Life: Darwin Meets vonNeumann, pages 27–34, 2011.

[23] J. Secretan, N. Beato, D. B. D Ambrosio,A. Rodriguez, A. Campbell, and K. O. Stanley.Picbreeder: evolving pictures collaboratively online. InConference on Human Factors in Computing Systems,CHI ’08, pages 1759–1768. ACM, 2008.

[24] K. Sims. Evolving 3d morphology and behavior bycompetition. Artificial Life IV, pages 28–39, 1994.

[25] M. Yim, W. Shen, B. Salemi, D. Rus, M. Moll,H. Lipson, E. Klavins, and G. Chirikjian. Modularself-reconfigurable robot systems [grand challenges ofrobotics]. IEEE Robotics & Automation Magazine,14(1):43–52, 2007.

[26] V. Zykov, W. Phelps, N. Lassabe, and H. Lipson.Molecubes Extended: Diversifying Capabilities ofOpen-Source Modular Robotics. In InternationalConference on Intelligent RObots and Systems,Self-Reconfigurable Robots Workshop (IROS 08), 2008.

544