Evolving a Team of Self-organizing UAVs to Address Spatial Coverage ProblemsIstván Fehérvári, Wilfried Elmenreich and Evsen YanmazMobile Systems Group/Lakeside LabsInstitute for Networked and Embedded SystemsAlpen-Adria Universität Klagenfurt
2Wilfried Elmenreich
Overview
• Unmanned areal vehicles
• The coverage problem
• Discrete simulation model
• Evolutionary approach
• Results
• Conlusion and outlook
3Wilfried Elmenreich
UAVs
4Wilfried Elmenreich
4
Battery-powered UAVs
• Quadcopter platform with onboard sensors and electronic for flight stabilization
• Attached cameras for sensing the environment
• GPS receiver for autonomous waypoint flights
• Limitations on payloads, flight time, weather conditions
www.microdrones.de www.asctec.de
5Wilfried Elmenreich
5
Small UAV Network for Emergency Assistance
Mission Control
UAV Network
6Wilfried Elmenreich
6
Coverage/Detection Problem
• Wireless sensor networks– Civil
• Environmental monitoring: wildfires, volcanoes, glaciers, storms, agriculture fields
• Communication assistance: serve as data fusion centers; serve as mobile base stations (relay)
• Video surveillance: traffic
monitoring, convoy protection
– Military, security• Battlefield assistance; target
detection and tracking; search and destroy; border monitoring
• Coverage problem in robotics– Snow removal, lawn
mowing, car-body painting, floor cleaning, etc.
• Cellular decomposition
• Complete versus randomized (probabilistic)
• Known layout of the environment versus sensor-based coverage
• Time-to-complete: shortest path, minimum energy, minimum number of turns, etc
7Wilfried Elmenreich
Planning and Monitoring Flight Paths
• Example for flight paths of two UAVs (green/red color)
8Wilfried Elmenreich
Simplified Discrete Simulation Model
• finite two-dimensional lattice
• each cell can contain at most one agent or obstacle
• agent can move to one of four directly neighboring cells (vN neighborhood)
• Goal: have each cell being visited at least once after minimum time
9Wilfried Elmenreich
Keep it Simple!
• No a priori knowledge about map size/position of obstacles
• No explicit communication among UAVs• No position estimation mechanism
• No map building
• Looking for a self-organizing online algorithm
• Until now, planned offline algorithms have been used (e.g. using a TSP solver)
10Wilfried Elmenreich
Wanted: the right UAV behavior model
• Controls the UAV as autonomous agent• Processes inputs (from sensors) and produces
output (to actuators)
Control System„Agent‘s Brain“
11Wilfried Elmenreich
EvolutionaryApproach
12Wilfried Elmenreich
Evolving the Control System
• Simulation of target system as testing playground
• Define goal via fitness function (e.g., maximize throughput in a network)
• Run evolutionary algorithm to derive behavior fulfilling the given goal
• Representation must be evolvable• Mutation
• Recombination
• difficult with an algorithm represented in C or Java code…
System modelGoals (fitness function)Simulation of problem
Explore solutions
Evaluate & Iterate
Analyze results
13Wilfried Elmenreich
Artificial Neural Networks
• Each neuron sums up theweighted outputs of the other connected neurons
• The output of the neuronis the result of an activation function (e.g. step, sigmoid function) applied to this sum
• Neural networks are distinguished by their connection structure– Feed forward connections (layered)
– Recursive (Ouput neurons feed back to input layer)
– Fully meshed
14Wilfried Elmenreich
Neural Networks are Evolvable
3.2 -1.2
3.2
3.2
-0.1
-4.2
0.20.0
3.5 -1.2
3.2
3.2
-0.1
-4.2
0.20.0
Mutation
0.0 -1.2
1.2
3.2
-0.1
1.2
1.2
0.0
3.2 2.2
3.2
3.2
-0.1
-4.2
0.20.5
Recombination
3.2 -1.2
3.2
3.2
-0.1
-4.2
0.2
0.0
15Wilfried Elmenreich
A Framework for Evolutionary Design
• FREVO (Framework for Evolutionary Design)
• Modular Java tool allowing fast simulation and evolution
• „Frevo“ means also a hot, „boiling“ dance around here
16Wilfried Elmenreich
Framework for Evolutionary Design
• FREVO defines flexible components for – Controller representation
– Problem specification
– Optimizer
17Wilfried Elmenreich
Modeling the Coverage Problem in FREVO
• Basically, we need a simulation of the problem• Interface for input/output connections to the
agents– Inputs for detecting obstacles
– Inputs for detecting other drones– Navigation output
• Feedback from a simulation run -> fitness value– spatial coverage (number of cells visited at least once
divided by the total number of unobstructed cells)
– completion time (number of simulation steps needed
18Wilfried Elmenreich
ResultsPhoto: wikipedia.org
19Wilfried Elmenreich
Algorithms under Test
• Non-cooperative evolved algorithms– UAVs are not aware of other UAVs– Basically evolving a „better“ random walk
• Cooperative evolved algorithm– Extra input to recognize meeting other UAVs
• Belief-based algorithm– Handcrafted solution based on random direction– Avoids cluttering of UAVs at border/meeting situations
• Reference Algorithms– Random walk– Random direction
20Wilfried Elmenreich
20
Coverage Snapshot for 10 UAVs
21Wilfried Elmenreich
Performance Comparison
22Wilfried Elmenreich
Conclusion and Outlook
Photo: wikipedia.org
23Wilfried Elmenreich
Conclusion and Outlook
• Three promising algorithms• Cooperation feature upon meeting of two UAVs
does not significantly improve results– for realistic density of drones
• Implementation in real system feasible– small computational effort
• Future work– Compare with a priori planning algorithms
– Challenges: dynamic environment, continuous space model
24Wilfried Elmenreich
Visit us!
• (open source):
www.frevotool.tk• Project MESON (Design Methods for Self-
Organizing Systems):meson.lakeside-labs.com
• Project cDrones:www.cdrones.com
• Lakeside Labs Clusterwww.lakeside-labs.com
25Wilfried Elmenreich
Thank you very much for your attention!
A short summary of the talk and the slides will be available at http://demesos.blogspot.com