Bioinspired ComputingLecture 15

Evolutionary Simulation Models & Bio-inspired robotics

Netta Cohen

2

Previous lectures

• Evolutionary & co-evolutionary algorithms

• Neural nets (RNNs, DNNs, GasNets)

• Evolution as training for artificial neural nets

Today

• Hardware & software models

• Evolutionary simulations• Some applications to

robotics• … and to biological

modelling

3

What is a Model?

Model building is a tool to discover, to probe and to test the implications of a particular way of thinking, of a theory, or of a hypothesis about the world.

Scientific models are not imitations of reality. If they were, accurate models would be as hard to understand as the systems they are modeling.

Instead of fully capturing a real system, a good model will focus on one or a few key features, whose behaviour it is designed to replicate.

4

How do we model?

Scientific process:

theory

model

prediction

observation

Models come in many flavours:

• Mathematical: E=Mc2

• Philosophical: I think therefore I am• Physical: Galileo’s helicopter• Political: Utopia • Platonic: forms

5

RoboticsRobots are embodied and situated; they interact with the environment; they sometimes interact with each other.

Their primary functions rely on sensory and motor behaviours.

They are usually designed to perform a very specific task.

Physical robot requirements often include: cost effectiveness and robustness.

One current trend and effort is to move forward to more complex robots

that perform multiple, complex and adaptive tasks.

The design involved in such problems is increasingly hard the theory is usually lacking.

6

Example: Cricket Behaviour

• Male crickets sing rapid repeated bursts of pure tone.• Females use these to approach a single singer

despite rough terrain/obstacles/other singers.• Female take a zig-zag path towards male of choice.

Existing theory incorporates different tasks sequentially & modularly:

• Recognition: songs are recognised & extracted from general noise

• Selection: songs are compared• Approach: the chosen singer is approached

7

How do crickets do it? Robots as models of bio-computation

“A mobile robot architecture must include sensing,

planning and locomotion which are tied together by

a model or map of the world …” (Kreigman et al, 1987)

But evidence suggests this may not be how crickets work…• Very few neurons appear to be involved – how can 10

neurons build a “model or map of the world”?• Calculating a singer’s location appears dauntingly difficult.

8Adapted from Webb (1995)Adapted from Webb & Hallam (1996)

Left

Right

Delay

firing threshold?

firing threshold?

Left

Right

external sound

auditory response

latency comparison

turn?A single, simple, embodied & situated mechanism performs recognition, choice & approach behaviours simultaneously.

Webb’s Cricket

Webb’s robot mimics the physiology of the real cricket:• Its ears are on its elbows and are linked by a tracheal tube• Sounds arrive twice at each ear, causing a phase difference• For sounds of a particular wavelength, these phase

differences indicate the direction of the source.

Barbara Webb built a simple mobile robot to explore the true mechanism.

9

Simulation ModelsA simulation model is “executable”. Simulations unfold over time according to a set of instructions (a protocol/algorithm) that capture selected aspects of the dynamics of a particular model system.

By building a simulation model according to our current theories, and observing how it unfolds under different circumstances we can improve our understanding of our theories and develop better ones.

Hopefully, we can make specific predictions. With those in hand, we can return to the real world and test them by experimenting on the real-world systems.

10

Evolutionary Simulation ModelsEvolutionary simulation modelling is a cross-disciplinary tool for complex (often multi-agent & adaptive) systems.

• Game theory: evolution of sex, collective behaviour, etc.• Linguistics: learning, evolution.• Geography: urban sprawl, traffic networks, etc.

By their very nature, simulations extend beyond equilibrium or steady state solutions and allow us to characterise the dynamics of the system at hand.

Examples:

11

Evolutionary Simulation ModelsThe strength (and weakness) of evolutionary simulation stems from a functional definition of fitness.

Auke Ijspeert used a GA to evolve recurrent neural network models of salamander locomotion. The neural net was interfaced with a simulated salamander anatomy (muscles, joints).

The fitness function was behavioural (rewarding forward motion). Simulations of the salamander were assessed for its ability to walk/swim.

Two types of simulation:1) Evolution (of the RNN)2) Locomotion (of the body controlled by the RNN)

12

The salamander model

The strength:

We need not have knowledge of the biology to define a fitness function that gives rise to efficient and robust locomotion. A fitness function that rewards fast forward motion might suffice.

The weakness:

If we wanted to model a real salamander, we are in for a disappointment. The neural network that evolved bares little resemblance to the biological one.

13

http://birg2.epfl.ch/movies/SIMS/anim_small_salam.gif

What does it do?

• Walk• Swim • Switch between walking & swimming across a border• Switch to swimming if it falls into the water• Follow targets, turn, modulate speed, and more...

The salamander can:

14

How simple is it?

In fact, designing or evolving such simulation robots based only on high-level descriptions is a daunting challenge.

A high level description of the solution hides the crucial role of low-level components and solutions. In fact, the salamander motor behaviour was designed according to very similar principles as biological neural nets for motor behaviour which we neglected to mention until now.

15

Bio-inspired robotics

By deriving first principles from biological robots (humans or often simpler animal behaviour) we may be able to overcome many of the hurdles.

Applications span a wide range of disciplines, including: Industry (e.g. car production), leisure (e.g. games industry), medicine, and research.

16

Biological motor behaviour

Biological motor behaviour has evolved to offer refined and flexible solutions to a variety of challenges. Motor behaviour involves any muscle activity - heart contractions, talking, chewing, digestion, walking, swimming, flying, even scratching, etc.

17

Biological motor behaviour (cont.)

Central Pattern Generating Neural Networks (CPGs):

Small, relatively simple neural systems withwell-defined units, well-defined circuitry, and well-defined function

Brain control

Central PatternGenerators

Muscles

refle

xes

con

tro

lm

od

ula

tion

fee

db

ack

Such central pattern generators are believed to be responsible for practically all known muscle behaviour.

18

Where do we start?In “simple” motor systems (insects, molluscs, crustacea), central pattern generators have identical architectures in all animals of the same species.

They are typically distributed throughout the body and form a distributed coordinated network of activity.

They also receive high level instructions from the brain and feedback from the low-level muscles.

Ijspeert’s salamander model, while ‘high level’ in its fitness function, was based on a simulation of CPGs and muscles.

http://birg2.epfl.ch/oldbirg/SIMS/sal_wsn.htmhttp://birg2.epfl.ch/movies/SIMS/anim_swim_trot_opt.gif

19

Rhythm generation in CPG circuits

Figure with permission: A. Ayali

Understanding CPG circuits:

Models of biological neural circuits generating self-sustained out-of-phase (or anti-phase) oscillations.

20

A Hexapod Robot

As a different example of switching behaviour consider Randall Beer’s hexapod robot controller…

The hexapod’s six legs are free to swing laterally and to be raised or lowered.

It is stable when the polygon formed by the lowered feet (dashed line) contains its centre of mass (cross).

The hexapod’s task is to fast forward motion.

Six-legged insects achieve this task is a number of ways:• Metachronal Wave: Pairs of legs swing while others stand

• Tripod gait: 3 legs achieve stability, the other 3 swing

This and all subsequent figures from Beer (1995).

21

The Neural Architecture

Each of the six individual legs is controlled by a fully-recurrent network of five continuous-time neurons.

swing backward

raise foot

forward swing

angle sensor

Single Leg Controller

Each neuron inhibits its counterpart in the adjacent leg controllers.

Full Layout

Each neuron receives the leg’s current angle as an input.

Motion is governed by the output of three “motor” neurons.

22

Finding An Effective Controller

The problem:

To discover weights that provide the controller with dynamics that, when coupled with the dynamics of the hexapod body, cause the hexapod to move forward.

Details of the GA: During evolution, the controller generally had access to the leg angle input, but sometimes this input was missing. This prevented the controller from relying on the input alone.

(Real insects can tolerate the removal of limbs, readjusting their gait appropriately.)

23

Finding An Effective Controller

The successful evolved controllers all generated tripod gaits.

A: with sensorsR1 R2 R3 L1 L2 L3V

A R1 R2 R3 L1 L2 L3

B

V

B: without

Notice B’s slower, more erratic gait.

Adapted from Beer (1995)

24

How Does It Use Its Sensors?The same controller improves its performance by exploiting sensory information when it is available:

The leg-angle input (continually oscillating between forward and backward values) is used to entrain the controller.

As the leg swings forward sensory feedback promotes the stance phase. As the leg swings back, feedback promotes the swing phase – feedback fine tunes the cyclic behaviour.

This solution allows the hexapod to respond adaptively to changes in the length of its legs. Although longer legs are slower, the net will automatically slow down to compensate.

25

Additional examples?

The same principles can be used to design complex adaptive walking behaviour for more human-like robot activity. Rybak et al. Have used CPG principles together to design and build physical walking robots. Again, the adaptive mechanisms of the robots allow them to save the balance problem without an explicit specification by the designer.

26http://lslwww.epfl.ch/birg/salamander.shtml

Salamander simulations revisited:

• Small simplified CPG-like networks were assumed for each vertebrate of the spinal cord

• Each CPG was evolved to generate anti-phase oscillations

• A series of connections were evolved down the spinal cord

• Swimming behaviour was evolved

• Sensory inputs were added on

• Walking behaviour was then superimposed

• Additional features (switching, turning, vision, etch.)

The salamander simulation was performed in small steps

27

Some take home messagesEvolutionary simulation models are important scientific as well as engineering tools.

Bio-inspired concepts and design can help develop robust robots and simulation robots may also help us understand the biological systems.

The choice of fitness function, the choice of assessments, and the amount of detail in a model all depend on the specific motivation underlying the research.

28

Suggested reading

• Ijspeert AJ (2001) “A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander” Biological Cybernetics, 84, 331-348.

• Randy Beer (1995), A dynamical systems perspective on agent-environment interaction, Artificial Intelligence, 72,173-215.

• Visually guided walking: http://bach.ece.jhu.edu/~etienne/labweb/projects/index.html andhttp://www.rybak-et-al.net/legloc.html and tutorial: http://www.iguana-robotics.com/presentations/cpgchip/ppframe.htm

• http://www.med.unifi.it/didonline/anno-I/informatica/analogcomputers.html

• http://www.comp.leeds.ac.uk/johnb/celegans/