Genetic Algorithms in Robotics - uni-hamburg.de

Universitat Hamburg

MIN-FakultatFachbereich Informatik

GA’s in Robotics

Genetic Algorithms in Robotics

Julius Mayer

Universitat HamburgFakultat fur Mathematik, Informatik und NaturwissenschaftenFachbereich Informatik

Technische Aspekte Multimodaler Systeme

October 31, 2016

J. Mayer 1

Universitat Hamburg

MIN-FakultatFachbereich Informatik

GA’s in Robotics

Outline

1. IntroductionMotivationClassification

2. AlgorithmOverviewPhases

3. ApplicationGA’s in RoboticsNeuroevolution

4. DiscussionEvaluationConclusion

J. Mayer 2

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Introduction - Motivation GA’s in Robotics

Motivation

Problems that are hard to solve with classical optimizationbecause of

I too many parameters (intractability),I large search space,I non-differentiable objective functions,I varying numbers of variables within the optimization process,I a lacking mathematical function specification.

[1]

J. Mayer 3

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Introduction - Classification GA’s in Robotics

Classification

Biologic: Theory of evolution

I Adaptation of population to theenvironment

I Gradual, hereditary change inthe individuals of a species

I Information storage and transferthrough genomes

I Optimizing population fitness byreproduction [3]

J. Mayer 4

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Introduction - Classification GA’s in Robotics

Classification

Biologic: Theory of evolution

I Adaptation of population to theenvironment

I Gradual, hereditary change inthe individuals of a species

I Information storage and transferthrough genomes

I Optimizing population fitness byreproduction

[5]

J. Mayer 5

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Introduction - Classification GA’s in Robotics

Classification

Algorithmic: Stochastic optimization

I Can ’solve’ NP-hard problems

I Solution approximation

I Computational complexity asprohibiting factor

I e.g. ant colony & particle swarmoptimization

[7]

J. Mayer 6

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Algorithm - Overview GA’s in Robotics

Algorithm

1. Initialization

2. Evaluation

3. Selection

4. Mutation

5. Terminate

J. Mayer 7

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Algorithm - Phases GA’s in Robotics

Phase 1. Initialization

Initialize random population

I Choose appropriaterepresentation

I Individuals as encodedproblem solutions

I e.g. real values or binarystrings

1,67 2,34 1,04 5,83 2,10

101010011011010010101 [12]

J. Mayer 8

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Algorithm - Phases GA’s in Robotics

Phase 2. Evaluation

I Choose objective function

I Decode & evaluatechromosome

I Find minima in objectivefunction

I Survival of the fittest

[9]

J. Mayer 9

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Algorithm - Phases GA’s in Robotics

Phase 3. Selection

I Probabilistic selection

I Roulette wheel selection

I Tournament Selection

I Balance exploration &exploitation dilemma

J. Mayer 10

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Algorithm - Phases GA’s in Robotics

Phase 4. Mutation

I Assign mutation rateI Mutate

I BitflipsI Normal distribution

mutation

I RecombinationI CrossoverI Permutation

I Mutation drives change

[12]

J. Mayer 11

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Algorithm - Phases GA’s in Robotics

Phase 4. Mutation

I Assign mutation rateI Mutate

I BitflipsI Normal distribution

mutation

I RecombinationI CrossoverI Permutation

I Mutation drives change

[12]

J. Mayer 12

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Algorithm - Phases GA’s in Robotics

Phase 5. Termination

I If solution is sufficiently closeI Performance is satisfyingI Change is stagnant

J. Mayer 13

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Application - GA’s in Robotics GA’s in Robotics

GA’s in Robotics

I Evolutionary robotics

I Path planing

I Multivariate parameteroptimization

I Evolving artificial networkarchitectures & optimizingconnection weights (learning)

[6]

J. Mayer 14

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Application - Neuroevolution GA’s in Robotics

Neuroevolution of augmenting topologies (NEAT)

I Uses crossover

I Uses speciation to protectstructures

I Increment growth fromminimal structures

[10]

J. Mayer 15

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Application - Neuroevolution GA’s in Robotics

NEAT: Mutation

I Connection weights

I Network structure

I Add new connection(random weight)

I Add new node (splitconnection)

[10]

J. Mayer 16

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Application - Neuroevolution GA’s in Robotics

NEAT: Crossover

I Genes are lined up accordingto innovation number

I Matching Genes are randomlychosen

I Disjoint / excess are takenfrom fitter parent

[10]

J. Mayer 17

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Application - Neuroevolution GA’s in Robotics

NEAT for RoboCup Soccer

Keepaway (robot soccersubtask)I Large state space (unable to

explore exhaustively)I Only partial state

information for each agentI Continuous action spaceI Multiple teammates need to

learn simultaneously[4]

J. Mayer 18

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Application - Neuroevolution GA’s in Robotics

NEAT vs. Temporal Difference Methods

[11]

J. Mayer 19

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Application - Neuroevolution GA’s in Robotics

NEAT vs. Temporal Difference Methods

[11]

J. Mayer 20

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Discussion - Evaluation GA’s in Robotics

Evaluation

Challenges

I Get GA’s parameters right

I Solution encodingI Exploration vs. exploitation

I Premature convergence(local minima)

I slow convergenceI big search space

I Find representative fitnessfunction

Drawbacks

I Approximation instead ofexact solution

I Computational complexity asprohibiting factor

I No guaranteed convergenceto global optimum

I Non-exhaustive coverage ofthe complete solution space

J. Mayer 21

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Discussion - Conclusion GA’s in Robotics

Conclusion

I Inspired by biological evolution

I Stochastic optimization

I Population containing encodedproblem solutions (parameters)

I Different applications in robotics

I e.g. evolving ANN (NEAT) forRoboCup Soccer

I No guarantee for convergence toglobal optimum

[2]

J. Mayer 22

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Discussion - Conclusion GA’s in Robotics

References

[1] http://rednuht.org/genetic walkers/, 2016.

[2] https://cdn.meme.am/instances/41036988.jpg.2016.

[3] https://i.ytimg.com/vi/3ZJNyScv8to/maxresdefault.jpg, 2016.

[4] https://i.ytimg.com/vi/HHlN0TDgllE/maxresdefault.jpg.2016.

[5] http://www.hanskottke.de/wordpress/wp-content/uploads/2012/09/evolutions hip hop.jpg.2016.

[6] http://www.orocos.org.2016.

[7] http://www.turingfinance.com/wp-content/uploads/2014/10/Self-Organizing-Feature-Map-3.png.2016.

[8] John H Holland.Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, andartificial intelligence.U Michigan Press, 1975.

[9] Melanie Mitchell.Complexity: A Guided Tour.In Oxford, volume 1, chapter Cellular A, pages 145–159. Oxford University Press, Inc, New York, 2009.

J. Mayer 23

Universitat Hamburg

MIN-FakultatFachbereich Informatik

Discussion - Conclusion GA’s in Robotics

References (cont.)

[10] Kenneth O Stanley and Risto Miikkulainen.Evolving neural networks through augmenting topologies.Evolutionary computation, 10(2):99–127, 2002.

[11] Matthew E Taylor.Comparing Evolutionary and Temporal Difference Methods in a Reinforcement Learning Domain.pages 1321–1328.

[12] Mattias Wahde.Biologically Inspired Optimization Methods: An introduction.WIT Press, Boston, MA, 2008.

J. Mayer 24