Metaheuristics and Cognitive Models for Autonomous Robot Navigation€¦ · · 2017-11-10Models for Autonomous Robot Navigation Raj Korpan ... highly dependent on initialization

Metaheuristics and Cognitive Models for Autonomous

Robot Navigation

Raj Korpan

Department of Computer Science

The Graduate Center, CUNY

Second Exam Presentation

April 25, 2017

1 / 31

Autonomous robot navigation (ARN)

• Mobile robots move through an environment from one location to another without human intervention

• Two heuristic-based approaches

• Metaheuristics for path planning

• Cognitive models of human navigation

• These approaches

• Learn about the search space or the environment

• Balance the tradeoff between exploration and exploitation

2 / 31

Outline

• Background

• Challenges in ARN

• Heuristics and metaheuristics

• Cognitive science

• Conclusions

3 / 31

Outline

• Background




• Conclusions

4 / 31

Background

• Problem 𝑃 = 𝑆, 𝐼, 𝐴, 𝐺

• 𝑆: set of states

• 𝐼 ⊆ 𝑆: set of initial states

• 𝐴: set of possible actions 𝑎 ∈ 𝐴

• 𝐺 𝑠 is a Boolean goal test

• Problem domain = set of related problems that share some characteristic

• Path = a finite ordered sequence of interleaved states and actions 𝑠1, 𝑎1, 𝑠2, 𝑎2, … , 𝑠𝑟−1, 𝑎𝑟−1, 𝑠𝑟

• Solution = a path where 𝑠1 ∈ 𝐼 and 𝐺 𝑠𝑟 = 𝑇𝑟𝑢𝑒

• Optimal solution has minimum path cost

• Satisfactory solution is good enough

5 / 31

Path planning

• Plan = path proved to be a solution before execution

• Path planning = search for a plan that minimizes travel time, travel distance, or resource consumption

• Path quality = value of a path based on domain-specific criteria

Initial state

Goal state

Solution

Satisfactory solutions

Optimal solution?

6 / 31

Outline

• Background




• Conclusions

7 / 31

Challenges in ARN (1)

• Path planning = how do I get to where I need to go?

• Localization = where am I in the environment? How do I detect my pose 𝑥, 𝑦, 𝜃 despite sensor error?

• Mapping = what does the environment look like?

• Obstacle avoidance = how do I get around obstructions in the environment?

• Motion control = how do I account for error from my actuators?

NP-Hard [Canny, 1988]

Simultaneous localization and mapping (SLAM) [Durrant, 2006]

Reactive and online

Handled by hardware

8 / 31

• Current state-of-the-art for implemented ARN systems on commercial robots = SLAM with A* search

• Lack of standard testbed and performance metrics no conclusive evidence that any approach is best

• Since sub-optimal solutions must suffice many other approaches have been investigated

Challenges in ARN (2)

• Real world vs. simulation

• Environmental issues = observability, multiagency, dynamism, continuity, and modality

[Bartel, 2007] [Zhou, 2014]

9 / 31

Outline

• Background




• Conclusions

10 / 31

Heuristics and metaheuristics

• Heuristic = efficient strategy that can often solve problems, typically specific to a problem domain [Pearl, 1984]

• Metaheuristic = a broadly applicable and non-domain specific technique that uses a heuristic strategy to obtain satisfactory solutions [Glover, 2006]

• Hybrid metaheuristic = 2 metaheuristics or metaheuristic + another method [Blum, 2011]

• No guarantee that an optimal solution will be found in finite time satisfactory solutions suffice in path planning for ARN

• Candidate = path found during search

• Neighbor of a candidate = path with one change to the candidate’s action sequence

11 / 31

changed added removed

Local Global

Hill climbing (HC)

Steepest-ascent HC

Random search

HC with random restarts

Genetic algorithm

Genetic programming

Evolutionary programming

Differential evolution

Biogeography-based optimization

Particle swarm optimization

Bat algorithm

Cuckoo search algorithm

Gravitational search

Harmony search

Ant colony optimization

Artificial bee colony

Firefly algorithm

Memetic algorithm

Single solution

Population

Locality of search

Number of candidates considered

Simulated annealing

Tabu search

Heuristic-based methods

Taxonomy of search methods

Both

Metaheuristics

Evolutionary algorithms

Swarm algorithms

Other metaheuristics 12 / 31

[Talbi, 2009]

Local Global

Hill climbing (HC)

Steepest-ascent HC

Random search


Genetic algorithm

Genetic programming





Bat algorithm



Harmony search



Firefly algorithm

Memetic algorithm

Single solution

Population

Locality of search


Simulated annealing

Tabu search



Both

Metaheuristics


Swarm algorithms


[Talbi, 2009]


• Random search

• Pure exploration

• Hill climbing

• Pure exploitation of learned knowledge

• Hill climbing with random restarts

• Random exploration with periods of exploitation

• Hill climbing with steepest ascent

• Exploitation of learned knowledge with limited exploration around candidate

• Fast but highly susceptible to premature convergence to local optima

14 / 31

Local Global

Hill climbing (HC)

Steepest-ascent HC

Random search


Genetic algorithm

Genetic programming





Bat algorithm



Harmony search



Firefly algorithm

Memetic algorithm

Single solution

Population

Locality of search


Simulated annealing

Tabu search



Both

Metaheuristics


Swarm algorithms


[Talbi, 2009]

Single-solution metaheuristics

• Simulated annealing improves hill climbing [Kirkpatrick, 1983; Černỳ, 1985]

• Occasionally selects candidates with lower quality

• Changes the balance of exploration and exploitation by reducing the probability of selecting a lower quality candidates over time

• Tabu search also selects lower quality candidates [Glover, 1989; 1990]

• Prevents return to recently visited candidates

• Intentionally limits exploitation to increase exploration

• Advantages: efficient and can control memory requirements

• Disadvantages: highly dependent on initialization and parameter settings, may be slow

• Applications to path planning are rare and simulated in simplistic, unrealistic environments

16 / 31

Local Global

Hill climbing (HC)

Steepest-ascent HC

Random search


Genetic algorithm

Genetic programming





Bat algorithm



Harmony search



Firefly algorithm

Memetic algorithm

Single solution

Population

Locality of search


Simulated annealing

Tabu search



Both

Metaheuristics


Swarm algorithms


[Talbi, 2009]

Evolutionary algorithms (1)

• Inspired by Darwinian principles

• Genetic algorithm [Manikas, 2007]

• Evolves a population of candidates with reproduction, crossover, mutation, and selection

• Fitness function evaluates quality of candidates

• Genetic programming = candidates are computer programs [Koza, 1992]

• Evolutionary programming [Fogel, 1999]

• Candidates are value assignments to parameters of a single program

• The next generation is composed of the best candidates selected from among the parents and children

• Differential evolution [Storn, 1997]

• Candidates are vectors of real numbers that are function parameters

• Recombination incorporates a third population member during reproduction

18 / 31

Evolutionary algorithms (2)

• Applications to path planning• Single robot and unmanned aerial vehicle (UAV)• Multi-robot and multi-UAV• Multi-objective• Addition of domain knowledge

• Advantages: incorporates exploitation/hill-climbing through selection, and exploration/randomization through mutation

• Disadvantages:• Parameters must be hand tuned• Can be computationally and memory intensive

19 / 31

Local Global

Hill climbing (HC)

Steepest-ascent HC

Random search


Genetic algorithm

Genetic programming





Bat algorithm



Harmony search



Firefly algorithm

Memetic algorithm

Single solution

Population

Locality of search


Simulated annealing

Tabu search



Both

Metaheuristics


Swarm algorithms


[Talbi, 2009]

Swarm algorithms

• Simulates crowd behavior of organisms

• Ant colony optimization [Dorigo, 2006]

• Indirect communication when foraging

• Individual ants use local search

• Pheromones indicate candidate desirability

• Artificial bee colony [Karaboga, 2007]

• Three groups: employed bees, onlookers, and scouts

• Combines local search and global search

• Particle swarm optimization [Kennedy, 1995]

• Birds in search of food as particles that move through search space

• Direction of each particle’s local search is influenced by other particles

• Other approaches: firefly algorithm, bat algorithm, cuckoo search

21 / 31

Swarm algorithms comparison

22 / 31

Algorithm Individuals Type of search Communication mechanism

Ant colonyoptimization

ants hill climbing pheromone strength


bees hill climbing and random search

dance intensity


particles hill climbing velocity of global best position

Firefly algorithm

fireflies hill climbing flashing brightness and distance

Bat algorithm bats hill climbing and random search

echolocation frequency, loudness, and rate of emissions

Cuckoo search cuckoos random search none

Path planning with swarm algorithms

• Applications• Single robot, UAV, and underwater• Multi-robot and multi-UAV• Multi-objective• Addition of domain knowledge• Dynamic obstacles

• Advantages: communication mechanisms allow local information to influence entire population

• Disadvantages:• Parameters must be hand tuned• Can be computationally and memory intensive

23 / 31

Hybrid metaheuristics

• Categorized by

• Type of hybrid: dual (2 metaheuristics) or metaheuristic + another method

• Approach to ARN: blend components into a new system or use them separately for different tasks

• Applications to path planning

• Mostly in static, simulated environments

• Dual hybrids typically evolutionary algorithm + swarm algorithm

• Other methods include probabilistic roadmap, chaotic search, fuzzy logic, and artificial potential field

• Advantages: tries to resolve shortcomings of individual methods by combining multiple methods

• Disadvantages:

• Difficult to compare approaches

• Parameters must be hand tuned

• Increased computational complexity

24 / 31

Outline

• Background




• Conclusions

25 / 31

Cognitive science

• Multidisciplinary field that studies the mind and intelligence [Friedenberg, 2011]

• Spatial cognition = subfield of cognitive science that studies navigation and wayfinding

• Decision making = agent selects among a set of alternatives

• Reasoning = agent draws a conclusion from information to solve a problem or make a decision

• People exhibit goal-directed behavior

• People use heuristics to make decisions, especially when faced with limited time, knowledge, or computational power [Gigerenzer, 1999]

• People use case-based reasoning

• People employ hierarchical organization of spatial memory [Wiener, 2003]

26 / 31

Human navigation

27 / 31

Strategies for human navigation

What? Spatial knowledge: visual cues (landmarks), frame of reference (egocentric vs allocentric), spatial orientation, path integration, route knowledge, survey knowledge

Internal representation: cognitive map [Golledge, 1999]

Other knowledge: external information, previous knowledge and experience

Who? Demographics: age and gender Spatial abilities: sense of direction

Where? Simulated: virtual environments Real world: indoor and outdoor environments

When? Before navigation: path planning, search and selection

During navigation: following a plan, or decision making without a plan

Why? Adapted to overall goal or current task [Holscher, 2011]

Adapted to overall environment or current state

How? Sense: proprioception, vision and hearing

Decision and action: brain activity and muscle activation

How much? Cognitive economy

Cognitive models

• Cognitive models simulate observed human behavior with a computational system or algorithm

• Application of cognitive models to ARN• Build a hierarchical representation of the environment similar to a

cognitive map [Kuipers, 2000]

• Represent spatial knowledge with a graph• Shift between navigation strategies using a heuristic• Bayesian approach + artificial neural network to learn topological

maps and landmarks [Thrun, 1998]

• Learn abstract representations of the environment and use multiple heuristics to make decisions [Epstein, 2015]

• Advantages: exploit human knowledge and strategies

• Disadvantages:• No one model has simulated all observed human wayfinding behavior

or employed all their strategies• Difficult to account for all individual differences among people

28 / 31

Outline

• Background




• Conclusions

29 / 31

Conclusions

• Most approaches have been evaluated in simplified simulated environments that underestimate the hardware challenges of physical robots and reduce computational complexity

• But people successfully navigate in the real world already so there is room for improvement inspired by human behavior

• Ultimately, autonomous robot navigation systems must

• operate in the real world

• contend with observability, multiagency, dynamism, continuity, and modality issues

• Future work should address these challenges in a real-world environment

• Potential future work could create more robust ARN systems by a synergy between metaheuristics and cognitive models

30 / 31

Thank You

31 / 31

Documents

Metaheuristics and Cognitive Models for Autonomous Robot Navigation€¦ · · 2017-11-10Models for Autonomous Robot Navigation Raj Korpan ... highly dependent on initialization