1
Robot Control using Genetic Algorithms
Summary
• Introduction– Robot Control– Khepera Simulator
• Genetic Model for Path Planning– Chromosome Representation– Evaluation Function– Case Studies
• Conclusions
2
The Robot Controller Problem
Given a robot and a description of an environment,provide commands (motor speeds) to the robot, in
order to achieve a path between two specifiedlocations, which is collision-free and satisfies
certain optimisation criteria.
(xi, yi)
(xf, yf)
Optimisation Criteria
• Robot should:– attempt near-optimal paths– avoid obstacles– perform straight motion
• Controller should be independent of:– the robot’s environment– target location
3
The Khepera Simulator
• Freeware mobile robot simulator (designed by OlivierMichel, University of Nice Sophia-Antipolis)
• User designed worlds• Control algorithms can be written in C/C++• Robot’s position and angle reading• Colourful Graphical Interface• 8 sensors (S0-S7): [0, 1023]• 2 motors (M1, M2): [-10, +10]
S0
S1S2 S3
S4
S5
S7 S6
M1 M2
Front identification
Simulator Readings:sensors, position and angle
S0
S1S2 S3
S4
S5
S7 S6
M1 M2
Front identification
S0-S7: [0, 1023]
1000
1000
X
Y
Robot’s World
S0
S1S2 S3
S4
S5
S7 S6
M1 M2
Front identification
angle of the robot with the world
α : [-π, π]
x
y
0
obstacle not obstacle very
detected closed
4
Control Mode
• To evolve the robot’s attitudes as itinteracts with the external environment
• Each robot action determines:– how well the controller performs with
respect a given task;– the next input stimuli to the controller.
• The controller should learn as the robotinteracts with the environment
Controller Model
Genetic Algorithm
evolves robot’s attitudes
Sensors
Position
Robot’s Angle
Goal Location
Motor 2 Motor 1
KheperaSimulator
5
Proposed Model
IF Obstacle detectedTHEN
Avoid collision, forget target
ELSE Straight to the target according to thetarget direction
END
Sensors Reading Simplification
Sleft Sright
Sback
S0
S1S2 S3 S4
S5
S7 S6
Sleft = ( S0 + S1 + S2 ) / 3
Sright= ( S3 + S4 + S5 ) / 3
Sback = ( S6 + S7 ) / 2
6
Determining the Target Direction
(x,y)α = π/2
arrival position: (xf ,yf)
β
β = tan-1[(yf - y)/(xf - x)]
β - απ/2
π
π/4
0
− π/4
− π/2
3π/4
− 3π/4
Goal point toright side
Goal point infront
Goal point toleft side
Goal pointbehind
Direction =
Model
IF ((Sleft > L) or (Sright > L) or (Sback > L))THEN
Obstacle detected, avoid collision, forget targetProximity-sensor = highest value (Sleft, Sright, Sback)
ELSEObstacle not detected (collision-free), straight to the targetTarget direction = β - α
ENDL=collision threshold=900
7
Genetic Algorithm Modelling
• Chromosome Representation• Evaluation Function• Genetic Operators• Techniques• Parameters
Chromosome Representation
M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2
Target Direction(collision free) Obstacle detected
Left Right BackFront Left Right Back
Attitudes (Genes)
1 2 3 4 5 6 7
Which speed should be imposed to each motor ineach situation the robot is?
8
Evaluation Function
( )10;10;10
**7
1
≤≤≤≤≤≤
= ∑=
AiDiVi
AiDiViFi
• Three objectives:– (V) speed– (D) straight motion– (A) action: reach a target and avoid obstacles
• Calculated for each gene [1,7] at each step
Speed
max)*2/(|)2||1(| MMMVi +=• Normalised sum of the absolute value of the
motors speeds;• Whatever the robot does, it does quickly.
9
Straight Motion
=++=
=]7,6,5,1[,2))max)*2()21((1(
]4,3,2[,1iMMMi
Di
• It favours high positive speeds to bothmotors
• When the robot is not oriented to the target,D=1 avoids contradictory learning
Action
• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness
=−=≤∆=∆=≤∆=∆
=]7,6,5[,max1
0,)0(])4,3,2[(,max0,)0()1(,max
iifSSAAielsegiandiifggiAAielsediandiifddi
AAi
t
t
t
t
≠=
= ∑=
0,)(
0,0
1
TPiTPiAAi
TPiAi TPi
tt
TPi = total of steps executed by attitude i
AAi=action’s fitness at stept of attitude i
10
Action
• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness
=−=≤∆=∆=≤∆=∆
=]7,6,5[,max1
0,)0(])4,3,2[(,max0,)0()1(,max
iifSSAAielsegiandiifggiAAielsediandiifddi
AAi
t
t
t
t
≠=
= ∑=
0,)(
0,0
1
TPiTPiAAi
TPiAi TPi
tt
TPi = total of steps executed by attitude i
AAi=action’s fitness at stept of attitude i
Rates the distance variation to the target between
two consecutive steps, and the maximum distance in one step,
for collision free/front
Action
• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness
=−=≤∆=∆=≤∆=∆
=]7,6,5[,max1
0,)0(])4,3,2[(,max0,)0()1(,max
iifSSAAielsegiandiifggiAAielsediandiifddi
AAi
t
t
t
t
≠=
= ∑=
0,)(
0,0
1
TPiTPiAAi
TPiAi TPi
tt
TPi = total of steps executed by attitude i
AAi=action’s fitness at stept of attitude i
Rates the angle variation between two consecutive steps, and
the maximum angle in one step, for collision freeleft, right, back
11
Action
• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness
=−=≤∆=∆=≤∆=∆
=]7,6,5[,max1
0,)0(])4,3,2[(,max0,)0()1(,max
iifSSAAielsegiandiifggiAAielsediandiifddi
AAi
t
t
t
t
≠=
= ∑=
0,)(
0,0
1
TPiTPiAAi
TPiAi TPi
tt
TPi = total of steps executed by attitude i
AAi=action’s fitness at stept of attitude iIncreases as the distance
to the proximity-sensor increasesin the step
Improving the Target DirectionModel
Â2 - Â1 3π/4 π/4
Target at LEFT
Target Behind Target in FRONT
Target at RIGHT
-3π/4 - π/4
First Target Direction Model
4 possible target directions
0
π/4
π/2
3π/4
π/2
-3π/4 -π/4
-π/2
8 possible target directions
12
Chromosome Representations
Cada e s t ado co r re sponde a uma ún ica at i tude (par de velocidadesM1 e M2) , e c ada a t i t ude co r r e sponde a um gene do c romossoma . Target Direction (collision free) Obstacle detected
M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 Front Left Right Back Left Right Back
Chromosome with 7 attitude Genes
Target Direction (collision free) Obstacle detected
M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 Front Front Left Left Right Right Back Back Left Right Back left right front back front back left right
Chromosome with 11 attitude Genes
Genetic Algorithm
• Real number chromosome• Population Size =100• Generations = 50• Crossover Rate = 80 %• Mutation Rate = 4%• Roulette Wheel Reproduction• Elitism• Linear scaling of fitness• 300 Evaluation Steps for each chromosome• Average of 25 Experiments
13
Genetic Algorithm Performance7 Genes Chromosome
Best Chromosomesin 1 experiment
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
Number of Generations
FITN
ES
S
Genetic Algorithm Performance7 Genes Chromosome
Average of Best Chromosomes in 25 experiments
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
Number of Generations
FITN
ESS
14
Genetic Algorithm Performance11 Genes Chromosome
Best Chromosomesin 1 experiment
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
Number of Generations
FITN
ES
S
Genetic Algorithm Performance11 Genes Chromosome
Average of Best Chromosomesin 25 experiments
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
Number of Generations
FIT
NE
SS
15
Paths Achieved in World 1Case Study 1SITUAÇÃO 1
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
Paths Achieved in World 1Case Study 2SITUAÇÃO 4
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
16
Speed Comparison
0
1000
2000
3000
4000
5000
6000
1 2 3 4 5
Case Studies
Nu
mb
er
of
Ste
ps
11 Genes Chromosome
7 Genes Chromosome
Paths Achieved in World 2Case Study 1SITUAÇÃO 1
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
17
Paths Achieved in World 2Case Study 2SITUAÇÃO 2
CROMOSSOMA DE 7 GENES CROMOSSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
0
500
1000
1500
2000
2500
3000
3500
4000
4500
1 2 3
Case Studies
Num
ber
of S
teps
Speed Comparison
11 Genes Chromosome
7 Genes Chromosome
18
84,62%
40,00%
214,29%
Speed Comparison (%)
Case Study 1
Case Study 2
Case Study 3
Paths Achieved in World 3Case Study 1SITUAÇÃO 1
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
19
Paths Achieved in World 3Case Study 2SITUAÇÃO 2
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
Paths Achieved in World 3Case Study 3SITUAÇÃO 3
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
20
Paths Achieved in World 3Case Study 4SITUAÇÃO 4
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
Speed Comparison
Speed Comparison – World 37 Genes Cromosome 11 Genes Cromosome
Case Study 1 target not reached 729Case Study 2 1124 618Case Study 3 target not reached target not reachedCase Study 4 target not reached 1655
21
Conclusions
• A simple GA could gradually evolve the robot control
• The robot achieved near optimal path towards the goal,avoiding obstacles
• Retraining is not necessary when the environment changes
• Controller improved performance with the 11 genes model
• The robot has no memory about previous unsuccessful pathsand may get lost
• Other tasks can be included in the model (e.g. energy supply)
• Chromosome codification is limited for few robot’s situations