View
3
Download
0
Category
Preview:
Citation preview
Analyzing Swarms A Stochastic Processes Approach to Studying Robotic Swarm Behavior
Kristina LermanUSC Information Sciences Institute
Tad HoggHP Labs
Thanks to Aram Galstyan Alcherio Martinoli Maja Matarić Chris Jones Brian Gerkey Bernardo Huberman
ScheduleScheduleMotivation How to design swarmsAnalysis based on stochastic processesndash basic theoryndash applications amp comparing to experiments
BREAK
Generalizationsndash adaptive robotsndash reconfigurable and microscopic robots
Summary and future directions
June 8 2005 Analyzing Swarms Tutorial 2118
What is a Robot SwarmWhat is a Robot SwarmLarge number of simple robotsndash limited hardware capabilities
power CPU communications
ndash limited view of environmentlocal neighborhood noisy sensors
ndash simple control methods
Tasks involve overall behavior of groupndash generally average not extreme behaviorndash less interest in specific individual robots
June 8 2005 Analyzing Swarms Tutorial 3118
How to Design a SwarmHow to Design a SwarmWhat hardware capabilitiesWhat local control program
Does resulting behavior solve my taskndash reliably (eg even with some failures)ndash within reasonable time and resource use
What if task changes during operation
June 8 2005 Analyzing Swarms Tutorial 4118
Local Control amp Collective BehaviorLocal Control amp Collective BehaviorLarge design space for swarmsHow to find good designs
key issuerelate designed robot capabilities to resulting swarm behavior
June 8 2005 Analyzing Swarms Tutorial 5118
Poor Design for One Poor Design for One Ok for ManyOk for Many
Example search an areaControl gives a robot 1 chance to find targetndash bad for one robotndash good with swarm of 1000 robots
swarm likely to find target if behaviors more-or-less independent
June 8 2005 Analyzing Swarms Tutorial 6118
Good Design for One Good Design for One Bad for ManyBad for Many
Example search an areaControl gives a robot 100 chance to find targetndash good for one robotndash bad for swarm of 1000 robots
all may pile up at targetmay interfere with each other
June 8 2005 Analyzing Swarms Tutorial 7118
Swarm Sensitivity to Robot BehaviorsSwarm Sensitivity to Robot BehaviorsRandom variation averages out
But correlated changes can have large effectsndash eg save power by increasing time before communicating
interesting eventsndash gradually increasing delay can lead to abrupt system change
from steady-state to oscillatory or chaotic system
ndash such abrupt changes not evident with just a few robotsndash analogous to phase transitions in physical systems
ndash for design identify parameter values giving abrupt changes
June 8 2005 Analyzing Swarms Tutorial 8118
Coordination within SwarmsCoordination within SwarmsRobots mostly act independentlyndash occasional coordination with nearby robotsndash behavior based on their local environment
maintaining no history of past interactions
Complicationsndash robots modify behavior based on historyndash robots interact continuously with neighborsndash robots also have changing global constraints
eg broadcast instructions to whole swarm
June 8 2005 Analyzing Swarms Tutorial 9118
System Design ApproachesSystem Design ApproachesSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 10118
Simulation Simulation Examine behavior of large swarms prior to building robots
Can be computationally intensive to explore design spaceAccuracy depends on knowledge of robots and task environment
June 8 2005 Analyzing Swarms Tutorial 11118
ExperimentExperimentBuild a few robots test in task environment
Expensive especially to explore different hardware capabilitiesMay not have enough robots to see large-scale behaviors
June 8 2005 Analyzing Swarms Tutorial 12118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
ScheduleScheduleMotivation How to design swarmsAnalysis based on stochastic processesndash basic theoryndash applications amp comparing to experiments
BREAK
Generalizationsndash adaptive robotsndash reconfigurable and microscopic robots
Summary and future directions
June 8 2005 Analyzing Swarms Tutorial 2118
What is a Robot SwarmWhat is a Robot SwarmLarge number of simple robotsndash limited hardware capabilities
power CPU communications
ndash limited view of environmentlocal neighborhood noisy sensors
ndash simple control methods
Tasks involve overall behavior of groupndash generally average not extreme behaviorndash less interest in specific individual robots
June 8 2005 Analyzing Swarms Tutorial 3118
How to Design a SwarmHow to Design a SwarmWhat hardware capabilitiesWhat local control program
Does resulting behavior solve my taskndash reliably (eg even with some failures)ndash within reasonable time and resource use
What if task changes during operation
June 8 2005 Analyzing Swarms Tutorial 4118
Local Control amp Collective BehaviorLocal Control amp Collective BehaviorLarge design space for swarmsHow to find good designs
key issuerelate designed robot capabilities to resulting swarm behavior
June 8 2005 Analyzing Swarms Tutorial 5118
Poor Design for One Poor Design for One Ok for ManyOk for Many
Example search an areaControl gives a robot 1 chance to find targetndash bad for one robotndash good with swarm of 1000 robots
swarm likely to find target if behaviors more-or-less independent
June 8 2005 Analyzing Swarms Tutorial 6118
Good Design for One Good Design for One Bad for ManyBad for Many
Example search an areaControl gives a robot 100 chance to find targetndash good for one robotndash bad for swarm of 1000 robots
all may pile up at targetmay interfere with each other
June 8 2005 Analyzing Swarms Tutorial 7118
Swarm Sensitivity to Robot BehaviorsSwarm Sensitivity to Robot BehaviorsRandom variation averages out
But correlated changes can have large effectsndash eg save power by increasing time before communicating
interesting eventsndash gradually increasing delay can lead to abrupt system change
from steady-state to oscillatory or chaotic system
ndash such abrupt changes not evident with just a few robotsndash analogous to phase transitions in physical systems
ndash for design identify parameter values giving abrupt changes
June 8 2005 Analyzing Swarms Tutorial 8118
Coordination within SwarmsCoordination within SwarmsRobots mostly act independentlyndash occasional coordination with nearby robotsndash behavior based on their local environment
maintaining no history of past interactions
Complicationsndash robots modify behavior based on historyndash robots interact continuously with neighborsndash robots also have changing global constraints
eg broadcast instructions to whole swarm
June 8 2005 Analyzing Swarms Tutorial 9118
System Design ApproachesSystem Design ApproachesSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 10118
Simulation Simulation Examine behavior of large swarms prior to building robots
Can be computationally intensive to explore design spaceAccuracy depends on knowledge of robots and task environment
June 8 2005 Analyzing Swarms Tutorial 11118
ExperimentExperimentBuild a few robots test in task environment
Expensive especially to explore different hardware capabilitiesMay not have enough robots to see large-scale behaviors
June 8 2005 Analyzing Swarms Tutorial 12118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
What is a Robot SwarmWhat is a Robot SwarmLarge number of simple robotsndash limited hardware capabilities
power CPU communications
ndash limited view of environmentlocal neighborhood noisy sensors
ndash simple control methods
Tasks involve overall behavior of groupndash generally average not extreme behaviorndash less interest in specific individual robots
June 8 2005 Analyzing Swarms Tutorial 3118
How to Design a SwarmHow to Design a SwarmWhat hardware capabilitiesWhat local control program
Does resulting behavior solve my taskndash reliably (eg even with some failures)ndash within reasonable time and resource use
What if task changes during operation
June 8 2005 Analyzing Swarms Tutorial 4118
Local Control amp Collective BehaviorLocal Control amp Collective BehaviorLarge design space for swarmsHow to find good designs
key issuerelate designed robot capabilities to resulting swarm behavior
June 8 2005 Analyzing Swarms Tutorial 5118
Poor Design for One Poor Design for One Ok for ManyOk for Many
Example search an areaControl gives a robot 1 chance to find targetndash bad for one robotndash good with swarm of 1000 robots
swarm likely to find target if behaviors more-or-less independent
June 8 2005 Analyzing Swarms Tutorial 6118
Good Design for One Good Design for One Bad for ManyBad for Many
Example search an areaControl gives a robot 100 chance to find targetndash good for one robotndash bad for swarm of 1000 robots
all may pile up at targetmay interfere with each other
June 8 2005 Analyzing Swarms Tutorial 7118
Swarm Sensitivity to Robot BehaviorsSwarm Sensitivity to Robot BehaviorsRandom variation averages out
But correlated changes can have large effectsndash eg save power by increasing time before communicating
interesting eventsndash gradually increasing delay can lead to abrupt system change
from steady-state to oscillatory or chaotic system
ndash such abrupt changes not evident with just a few robotsndash analogous to phase transitions in physical systems
ndash for design identify parameter values giving abrupt changes
June 8 2005 Analyzing Swarms Tutorial 8118
Coordination within SwarmsCoordination within SwarmsRobots mostly act independentlyndash occasional coordination with nearby robotsndash behavior based on their local environment
maintaining no history of past interactions
Complicationsndash robots modify behavior based on historyndash robots interact continuously with neighborsndash robots also have changing global constraints
eg broadcast instructions to whole swarm
June 8 2005 Analyzing Swarms Tutorial 9118
System Design ApproachesSystem Design ApproachesSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 10118
Simulation Simulation Examine behavior of large swarms prior to building robots
Can be computationally intensive to explore design spaceAccuracy depends on knowledge of robots and task environment
June 8 2005 Analyzing Swarms Tutorial 11118
ExperimentExperimentBuild a few robots test in task environment
Expensive especially to explore different hardware capabilitiesMay not have enough robots to see large-scale behaviors
June 8 2005 Analyzing Swarms Tutorial 12118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
How to Design a SwarmHow to Design a SwarmWhat hardware capabilitiesWhat local control program
Does resulting behavior solve my taskndash reliably (eg even with some failures)ndash within reasonable time and resource use
What if task changes during operation
June 8 2005 Analyzing Swarms Tutorial 4118
Local Control amp Collective BehaviorLocal Control amp Collective BehaviorLarge design space for swarmsHow to find good designs
key issuerelate designed robot capabilities to resulting swarm behavior
June 8 2005 Analyzing Swarms Tutorial 5118
Poor Design for One Poor Design for One Ok for ManyOk for Many
Example search an areaControl gives a robot 1 chance to find targetndash bad for one robotndash good with swarm of 1000 robots
swarm likely to find target if behaviors more-or-less independent
June 8 2005 Analyzing Swarms Tutorial 6118
Good Design for One Good Design for One Bad for ManyBad for Many
Example search an areaControl gives a robot 100 chance to find targetndash good for one robotndash bad for swarm of 1000 robots
all may pile up at targetmay interfere with each other
June 8 2005 Analyzing Swarms Tutorial 7118
Swarm Sensitivity to Robot BehaviorsSwarm Sensitivity to Robot BehaviorsRandom variation averages out
But correlated changes can have large effectsndash eg save power by increasing time before communicating
interesting eventsndash gradually increasing delay can lead to abrupt system change
from steady-state to oscillatory or chaotic system
ndash such abrupt changes not evident with just a few robotsndash analogous to phase transitions in physical systems
ndash for design identify parameter values giving abrupt changes
June 8 2005 Analyzing Swarms Tutorial 8118
Coordination within SwarmsCoordination within SwarmsRobots mostly act independentlyndash occasional coordination with nearby robotsndash behavior based on their local environment
maintaining no history of past interactions
Complicationsndash robots modify behavior based on historyndash robots interact continuously with neighborsndash robots also have changing global constraints
eg broadcast instructions to whole swarm
June 8 2005 Analyzing Swarms Tutorial 9118
System Design ApproachesSystem Design ApproachesSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 10118
Simulation Simulation Examine behavior of large swarms prior to building robots
Can be computationally intensive to explore design spaceAccuracy depends on knowledge of robots and task environment
June 8 2005 Analyzing Swarms Tutorial 11118
ExperimentExperimentBuild a few robots test in task environment
Expensive especially to explore different hardware capabilitiesMay not have enough robots to see large-scale behaviors
June 8 2005 Analyzing Swarms Tutorial 12118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Local Control amp Collective BehaviorLocal Control amp Collective BehaviorLarge design space for swarmsHow to find good designs
key issuerelate designed robot capabilities to resulting swarm behavior
June 8 2005 Analyzing Swarms Tutorial 5118
Poor Design for One Poor Design for One Ok for ManyOk for Many
Example search an areaControl gives a robot 1 chance to find targetndash bad for one robotndash good with swarm of 1000 robots
swarm likely to find target if behaviors more-or-less independent
June 8 2005 Analyzing Swarms Tutorial 6118
Good Design for One Good Design for One Bad for ManyBad for Many
Example search an areaControl gives a robot 100 chance to find targetndash good for one robotndash bad for swarm of 1000 robots
all may pile up at targetmay interfere with each other
June 8 2005 Analyzing Swarms Tutorial 7118
Swarm Sensitivity to Robot BehaviorsSwarm Sensitivity to Robot BehaviorsRandom variation averages out
But correlated changes can have large effectsndash eg save power by increasing time before communicating
interesting eventsndash gradually increasing delay can lead to abrupt system change
from steady-state to oscillatory or chaotic system
ndash such abrupt changes not evident with just a few robotsndash analogous to phase transitions in physical systems
ndash for design identify parameter values giving abrupt changes
June 8 2005 Analyzing Swarms Tutorial 8118
Coordination within SwarmsCoordination within SwarmsRobots mostly act independentlyndash occasional coordination with nearby robotsndash behavior based on their local environment
maintaining no history of past interactions
Complicationsndash robots modify behavior based on historyndash robots interact continuously with neighborsndash robots also have changing global constraints
eg broadcast instructions to whole swarm
June 8 2005 Analyzing Swarms Tutorial 9118
System Design ApproachesSystem Design ApproachesSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 10118
Simulation Simulation Examine behavior of large swarms prior to building robots
Can be computationally intensive to explore design spaceAccuracy depends on knowledge of robots and task environment
June 8 2005 Analyzing Swarms Tutorial 11118
ExperimentExperimentBuild a few robots test in task environment
Expensive especially to explore different hardware capabilitiesMay not have enough robots to see large-scale behaviors
June 8 2005 Analyzing Swarms Tutorial 12118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Poor Design for One Poor Design for One Ok for ManyOk for Many
Example search an areaControl gives a robot 1 chance to find targetndash bad for one robotndash good with swarm of 1000 robots
swarm likely to find target if behaviors more-or-less independent
June 8 2005 Analyzing Swarms Tutorial 6118
Good Design for One Good Design for One Bad for ManyBad for Many
Example search an areaControl gives a robot 100 chance to find targetndash good for one robotndash bad for swarm of 1000 robots
all may pile up at targetmay interfere with each other
June 8 2005 Analyzing Swarms Tutorial 7118
Swarm Sensitivity to Robot BehaviorsSwarm Sensitivity to Robot BehaviorsRandom variation averages out
But correlated changes can have large effectsndash eg save power by increasing time before communicating
interesting eventsndash gradually increasing delay can lead to abrupt system change
from steady-state to oscillatory or chaotic system
ndash such abrupt changes not evident with just a few robotsndash analogous to phase transitions in physical systems
ndash for design identify parameter values giving abrupt changes
June 8 2005 Analyzing Swarms Tutorial 8118
Coordination within SwarmsCoordination within SwarmsRobots mostly act independentlyndash occasional coordination with nearby robotsndash behavior based on their local environment
maintaining no history of past interactions
Complicationsndash robots modify behavior based on historyndash robots interact continuously with neighborsndash robots also have changing global constraints
eg broadcast instructions to whole swarm
June 8 2005 Analyzing Swarms Tutorial 9118
System Design ApproachesSystem Design ApproachesSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 10118
Simulation Simulation Examine behavior of large swarms prior to building robots
Can be computationally intensive to explore design spaceAccuracy depends on knowledge of robots and task environment
June 8 2005 Analyzing Swarms Tutorial 11118
ExperimentExperimentBuild a few robots test in task environment
Expensive especially to explore different hardware capabilitiesMay not have enough robots to see large-scale behaviors
June 8 2005 Analyzing Swarms Tutorial 12118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Good Design for One Good Design for One Bad for ManyBad for Many
Example search an areaControl gives a robot 100 chance to find targetndash good for one robotndash bad for swarm of 1000 robots
all may pile up at targetmay interfere with each other
June 8 2005 Analyzing Swarms Tutorial 7118
Swarm Sensitivity to Robot BehaviorsSwarm Sensitivity to Robot BehaviorsRandom variation averages out
But correlated changes can have large effectsndash eg save power by increasing time before communicating
interesting eventsndash gradually increasing delay can lead to abrupt system change
from steady-state to oscillatory or chaotic system
ndash such abrupt changes not evident with just a few robotsndash analogous to phase transitions in physical systems
ndash for design identify parameter values giving abrupt changes
June 8 2005 Analyzing Swarms Tutorial 8118
Coordination within SwarmsCoordination within SwarmsRobots mostly act independentlyndash occasional coordination with nearby robotsndash behavior based on their local environment
maintaining no history of past interactions
Complicationsndash robots modify behavior based on historyndash robots interact continuously with neighborsndash robots also have changing global constraints
eg broadcast instructions to whole swarm
June 8 2005 Analyzing Swarms Tutorial 9118
System Design ApproachesSystem Design ApproachesSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 10118
Simulation Simulation Examine behavior of large swarms prior to building robots
Can be computationally intensive to explore design spaceAccuracy depends on knowledge of robots and task environment
June 8 2005 Analyzing Swarms Tutorial 11118
ExperimentExperimentBuild a few robots test in task environment
Expensive especially to explore different hardware capabilitiesMay not have enough robots to see large-scale behaviors
June 8 2005 Analyzing Swarms Tutorial 12118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Swarm Sensitivity to Robot BehaviorsSwarm Sensitivity to Robot BehaviorsRandom variation averages out
But correlated changes can have large effectsndash eg save power by increasing time before communicating
interesting eventsndash gradually increasing delay can lead to abrupt system change
from steady-state to oscillatory or chaotic system
ndash such abrupt changes not evident with just a few robotsndash analogous to phase transitions in physical systems
ndash for design identify parameter values giving abrupt changes
June 8 2005 Analyzing Swarms Tutorial 8118
Coordination within SwarmsCoordination within SwarmsRobots mostly act independentlyndash occasional coordination with nearby robotsndash behavior based on their local environment
maintaining no history of past interactions
Complicationsndash robots modify behavior based on historyndash robots interact continuously with neighborsndash robots also have changing global constraints
eg broadcast instructions to whole swarm
June 8 2005 Analyzing Swarms Tutorial 9118
System Design ApproachesSystem Design ApproachesSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 10118
Simulation Simulation Examine behavior of large swarms prior to building robots
Can be computationally intensive to explore design spaceAccuracy depends on knowledge of robots and task environment
June 8 2005 Analyzing Swarms Tutorial 11118
ExperimentExperimentBuild a few robots test in task environment
Expensive especially to explore different hardware capabilitiesMay not have enough robots to see large-scale behaviors
June 8 2005 Analyzing Swarms Tutorial 12118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Coordination within SwarmsCoordination within SwarmsRobots mostly act independentlyndash occasional coordination with nearby robotsndash behavior based on their local environment
maintaining no history of past interactions
Complicationsndash robots modify behavior based on historyndash robots interact continuously with neighborsndash robots also have changing global constraints
eg broadcast instructions to whole swarm
June 8 2005 Analyzing Swarms Tutorial 9118
System Design ApproachesSystem Design ApproachesSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 10118
Simulation Simulation Examine behavior of large swarms prior to building robots
Can be computationally intensive to explore design spaceAccuracy depends on knowledge of robots and task environment
June 8 2005 Analyzing Swarms Tutorial 11118
ExperimentExperimentBuild a few robots test in task environment
Expensive especially to explore different hardware capabilitiesMay not have enough robots to see large-scale behaviors
June 8 2005 Analyzing Swarms Tutorial 12118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
System Design ApproachesSystem Design ApproachesSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 10118
Simulation Simulation Examine behavior of large swarms prior to building robots
Can be computationally intensive to explore design spaceAccuracy depends on knowledge of robots and task environment
June 8 2005 Analyzing Swarms Tutorial 11118
ExperimentExperimentBuild a few robots test in task environment
Expensive especially to explore different hardware capabilitiesMay not have enough robots to see large-scale behaviors
June 8 2005 Analyzing Swarms Tutorial 12118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Simulation Simulation Examine behavior of large swarms prior to building robots
Can be computationally intensive to explore design spaceAccuracy depends on knowledge of robots and task environment
June 8 2005 Analyzing Swarms Tutorial 11118
ExperimentExperimentBuild a few robots test in task environment
Expensive especially to explore different hardware capabilitiesMay not have enough robots to see large-scale behaviors
June 8 2005 Analyzing Swarms Tutorial 12118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
ExperimentExperimentBuild a few robots test in task environment
Expensive especially to explore different hardware capabilitiesMay not have enough robots to see large-scale behaviors
June 8 2005 Analyzing Swarms Tutorial 12118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
DeploymentDeploymentBuild many robots try swarm on full task
Very expensive
June 8 2005 Analyzing Swarms Tutorial 13118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Another Choice AnalysisAnother Choice AnalysisStochastic models to capture key issuendash relating local control to swarm behavior
Readily explore design spacendash eliminate bad designs
Validate designs with simulation amp experimentAn option pick designs with simple analysisndash Ie deliberately satisfy simplifying assumptions
June 8 2005 Analyzing Swarms Tutorial 14118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulation Experiments Deployment
June 8 2005 Analyzing Swarms Tutorial 15118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 16118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Historical PerspectiveHistorical PerspectiveTheory of stochastic processes grew out of thermodynamics
and probability theoryThermodynamics 1700rsquos ndash Early 1800rsquosEmpirical laws derived from large body of experimental data
Kinetic theory of gases Mid-1800rsquosBulk properties of matter (eg thermodynamic laws for gases) arose
from the dynamics of its constituent elements (gas molecules)
Probabilistic methods first used to calculate gas properties
Statistical mechanicsLate 1800rsquos ndash early 1900rsquosStochastic processes theory - Mathematical foundation (along with
quantum theory) of modern physics
June 8 2005 Analyzing Swarms Tutorial 17118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
ldquoThere is no hope to compute [irregular position of a Brownian particle] in detail but hellip certain averaged features vary in a regular way which can be described by simple lawsrdquo
Van Kampen 1992
June 8 2005 Analyzing Swarms Tutorial 18118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Stochastic ProcessStochastic ProcessStochastic Process Y(t) is a random variable (or a
function of a random variable) that changes with time
Defined by joint probability density
p(y1 t1 y2 t2 hellip yn tn)
probability that Y(t) takes values y1 y2 hellip at times t1 t2 hellip
June 8 2005 Analyzing Swarms Tutorial 19118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Some ExamplesSome ExamplesCoin flipping amp random walk (probability theory)Bacterial chemotaxis (biology)Chemical reactions (chemistry)Stock market prices (finance)Spread of disease through a population (epidemiology)Decay of nuclear particles (physics)Network traffic (computer science)
June 8 2005 Analyzing Swarms Tutorial 20118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Robot as a Stochastic ProcessRobot as a Stochastic Process
Robot can be viewed as a stochastic process
An individual robotrsquos behavior subject tondash External forces
cannot be anticipated
ndash Noise fluctuations and random events
ndash Other robots with complex trajectoriesCanrsquot predict which robots will interact
ndash Randomness in individualrsquos behavior rules eg collision avoidance procedures in robot controllers
June 8 2005 Analyzing Swarms Tutorial 21118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Stochastic Approach to Studying SwarmsStochastic Approach to Studying Swarmsunpredictable individuals predictable swarms
Stochastic processes framework provides simple model of behavior of ensemble of individually unpredictable robots
Details of robotsrsquo trajectories who interacts with whom and when are not importantSingle robot characteristics determine collective behavior of the swarm
June 8 2005 Analyzing Swarms Tutorial 22118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
BREAK2 Generalized Markov Processes
Basic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 23118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Ordinary Markov Process DefinitionOrdinary Markov Process DefinitionStochastic process takes values n1 n2hellip at times
t0 t1 hellip is a Markov process if it has a
Markov property value at time ti+1 depends only on its value at time ti and no other times
Dynamics of the process is fully determined byndash Initial state p(nt0)ndash Transition probability p(n ti+1|nrsquo ti)
Or p(n t+∆t|nrsquo t)
June 8 2005 Analyzing Swarms Tutorial 24118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Random WalkRandom Walk
Random walk is a Markov process
Probability distribution of positionndash p(n t)
Transition probabilitiesndash p(n+1 t+∆t|n t)=12ndash p(n-1 t+∆t|n t)=12
n n+1n-1n-2 n+2
June 8 2005 Analyzing Swarms Tutorial 25118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Dynamics of a Random WalkDynamics of a Random Walk
n n+1n-1n-2 n+2
)()(
)1()1()(
)1()1(
)1()1(
tnpwtnpw
tnpwtnpwdt
tndp
nnnn
nnnn
minusrarr+rarr
rarr+rarrminus
minusminus
++minus=
[ ] )()1()1(21)( tnptnptnp
dttndp
minus++minus=
June 8 2005 Analyzing Swarms Tutorial 26118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Stochastic Master EquationStochastic Master Equation
sumsumprime
primeprime
prime minusprime=n
nnn
nn tnpwtnpwdt
tndp )()()(
Describes dynamics of a Markov processTransition rates w
ttnttnpw
tnn ∆
prime∆+=
rarr∆prime
)|(lim0
June 8 2005 Analyzing Swarms Tutorial 27118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
From One to ManyFrom One to ManyCollective state of an ensemble of stochastic
processesEach process ndash Independent and indistinguishablendash Can take exactly one value from n1hellip nn at time t
Collective state = occupation vector N1 hellip Nmndash Ni is number of processes that have a value ni
ndash P(N t) is probability system is in configuration N at time t
June 8 2005 Analyzing Swarms Tutorial 28118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Collective State of Ensemble of Random Collective State of Ensemble of Random WalkersWalkers
n n+1n-1n-2 n+2
ρ(n t) ndash Number (density) of particles at position x
n
t0
t1
June 8 2005 Analyzing Swarms Tutorial 29118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Collective DynamicsCollective DynamicsCollective Master Equationsndash Describes how probability density of the collective configuration
changes in timendash Can be derived from the probability distribution of the
ensemble and the transition rates
Butndash It is often difficult to specify the correct probability
distribution of the collective statendash Instead average over the ensemble to get the Rate
Equation
June 8 2005 Analyzing Swarms Tutorial 30118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Rate EquationRate Equation
sumsumprime
primeprime
primeprime minus=n
nnnn
nnnn NwNw
dtNd
ndash Describes how Nn average number of processes with value nchanges in time
ndash No need to know exact probability distributionsRegardless of underlying probability distribution the mean evolves according to the Rate Equation
ndash Transition rates w are individual transition ratescan depend on the N values (ldquodensity dependentrdquo)
ndash Usually phenomenologicalCan be written down simply by assessing what important characteristics of the problem are
June 8 2005 Analyzing Swarms Tutorial 31118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Analysis of Rate EquationsAnalysis of Rate EquationsSolve dNndtndash subject to initial conditions values of Nn at t=0ndash To obtain Nn vs t
Analyze solutionsndash Does a steady state (s s) exist
dNndt=0ndash If not what is dynamics
oscillation chaosndash If so
How long to converge to s sHow does s s depend on critical parameters
ndash How quickly does system respond to changes
June 8 2005 Analyzing Swarms Tutorial 32118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Roadmap to Modeling Robot SwarmsRoadmap to Modeling Robot SwarmsApply stochastic processes framework to
study swarms of reactive robotsStarting with an individual robotndash Described by its controllerndash Compute transition rates w from physical parameters
Make transition to a multi-robot swarmndash Practical ldquoreciperdquo for constructing the Rate Equation from the
robot controllerndash Solve equations for a given set of parameters
June 8 2005 Analyzing Swarms Tutorial 33118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Reactive RobotsReactive RobotsReactive robot is a minimalist robot in which
perception and action are tightly coupled
Reactive robot makes decision about what action to take based on the action it is currently executing and input from sensors
June 8 2005 Analyzing Swarms Tutorial 34118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Reactive Robot as Markov ProcessReactive Robot as Markov ProcessReactive robot as a Markov Processndash Robotrsquos action at time t+∆t depends only on the action it is
executing at time tndash Inputs from sensors trigger transitions between actions
Individual descriptionndash p(n t) is probability robot is executing action n at time tndash Stochastic Master Equation describes dynamics of p(n t)
Collective descriptionndash Homogeneous group of reactive robots executing the same
controllerndash Rate Equation describes dynamics of the average number of
robots executing action n at time t
June 8 2005 Analyzing Swarms Tutorial 35118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Representation of a Reactive RobotRepresentation of a Reactive RobotReactive robot controller as a finite state automaton (FSA)ndash State = action robot is executingndash Transitions between states triggered by sensory inputs
Example simplified foraging scenario
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 36118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
MultiMulti--robot System Representationrobot System Representation
Homogeneous swarm is represented by the same FSAndash State = (average) number of robots executing an actionndash Transitions between states triggered by sensory inputs
searching
pickuphoming
start
Gripperclosed
PuckdetectedReach
home
June 8 2005 Analyzing Swarms Tutorial 37118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
CoarseCoarse--graininggraining
searchDetectobject
Avoidobstacle
search Avoidobstacle
search
bull Coarse-graining reduces the complexity of the modelbull Helps construct a minimal model that explains experiments
June 8 2005 Analyzing Swarms Tutorial 38118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
A ldquoReciperdquo for Rate EquationsA ldquoReciperdquo for Rate Equationssearching
Ns
pickupNp
homingNh
startws p
wp h
wh s
=dt
dNssps Nw rarrminus hsh Nw rarr+
=dt
dN psps Nw rarr+ php Nw rarrminus psh NNNN ++=
Initial conditions Ns(t=0)=N Nh(0)=0 Np(0)=0June 8 2005 Analyzing Swarms Tutorial 39118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Transition Rates Transition Rates wwjj kkTransition between states is triggered
ndash By a stimulus Obstacle another robot in a particular state location (eg home) communication
ndash By a timerTurn in a random direction for x seconds
Computing transition ratesndash Calculated from theory
Microscopic theoryMust know exact probability distributions
Phenomenological ldquoscattering cross-sectionrdquo approachTriggers are uniformly distributed in spaceRobots encounter triggers randomly
ndash Estimated from data by Calibration Fitting model to the data
June 8 2005 Analyzing Swarms Tutorial 40118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
ldquoldquoScattering CrossScattering Cross--sectionrdquo Approachsectionrdquo ApproachUseful idealization for roughly estimating model
parameters
A is arena area v is robotrsquos speeddi is robotrsquos detection widthMi is number of objects i
v∆t
di
AMvdw ii=
June 8 2005 Analyzing Swarms Tutorial 41118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Calibration ApproachCalibration ApproachIn simulation or experiment measure single robot
interactions To calculate rate robots encounter each otherndash Run experiment or simulation in an empty arena with two robotsndash Keep track of the number of collisions
To calculate rate robots encounter objectsndash Run experiment or simulation with a single robot and objects
scattered around the arenandash Keep track of the rate robot encounters them (eg picks up
pucks)
June 8 2005 Analyzing Swarms Tutorial 42118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
June 8 2005 Analyzing Swarms Tutorial 43118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Robot ForagingRobot ForagingCollect objects scattered in
the arena and assemblethem at a ldquohomerdquo locationndash Can be accomplished by a
single robot or a group of robots
Foraging model validated using PlayerStage grounded simulations
Goldberg amp Matarić
June 8 2005 Analyzing Swarms Tutorial 44118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Single Single vsvs Group of RobotsGroup of RobotsBenefits of group foragingndash Group is robust to individualrsquos failurendash Group can speed up collection by working in parallel
Disadvantages of a groupndash Increased interference due to collision avoidance
Optimal group sizendash beyond some group size interference outweighs the benefits of
the grouprsquos increased robustness and parallelism
June 8 2005 Analyzing Swarms Tutorial 45118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Robot Controller DiagramRobot Controller Diagram
start searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 46118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
searching homing
avoidingavoiding
startstart searching
avoid obstacle
collect
homing
detect object
reverse homing
detect object
avoid obstacle
June 8 2005 Analyzing Swarms Tutorial 47118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of searching robots at time t
Nh(t) = number of homing robots at time t
Nsav(t) Nh
av(t) = number of avoiding robots at time t
Mu(t) = number of undelivered pucks at time t
searching homing
avoidingavoiding
start
Ns Nh
Nsav Nh
av
June 8 2005 Analyzing Swarms Tutorial 48118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Model ParametersModel ParametersParameters ndash connect the model to experimentsαr αrsquor = rate of encountering a robot while searching homing
αp = rate of encountering a puck
τ = avoiding time
τh = homing time
searching homing
avoidingavoiding
start
αr αrsquor
αp
1τ1τ1τh
June 8 2005 Analyzing Swarms Tutorial 49118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Mathematical Model of ForagingMathematical Model of Foraging
=dt
dNsusp MNαminus [ ]0NNN ssr +minusα h
hN
τ1
+ avsN
τ1
+
searching homing
avoidingavoiding
start
=dt
dNhh
h
Nτ1
minususp MNα [ ]0NNN hhr +primeminusα avhN
τ1
+
June 8 2005 Analyzing Swarms Tutorial 50118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Mathematical Model of ForagingMathematical Model of Foraging
( )0NNN hhr +primeα
searching homing
avoidingavoiding
start
=dt
dN avh av
hNτ1
minus
avhhs
avs NNNNN minusminusminus= 0
Initial conditions Ns(t=0)=N0
June 8 2005 Analyzing Swarms Tutorial 51118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Mathematical Model of ForagingMathematical Model of Foraging
Uncollected puckshavhu NNMM minusminus= 0
Number of pucks in the arena changes when robots deliver them home
hh
Ndt
dMτ1
minus=
Initial conditions M(t=0)=M0
June 8 2005 Analyzing Swarms Tutorial 52118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Searching Robots and Pucks Searching Robots and Pucks vsvs TimeTime
robots
pucks
June 8 2005 Analyzing Swarms Tutorial 53118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Pucks Pucks vsvs Time for Different Group SizesTime for Different Group Sizes
N=5
N=15
N=20
June 8 2005 Analyzing Swarms Tutorial 54118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
000
20000
40000
60000
80000
100000
120000
1 2 3 4 5 6 7 8 9 10
number of robots
time
(s)
5 avoid while homing4 avoid3 reverse home2 home1 collect0 search
PlayerStage SimulationsPlayerStage SimulationsAverage time spent in each behavior
June 8 2005 Analyzing Swarms Tutorial 55118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Linking Model to SimulationsLinking Model to SimulationsGet transition rates from experimental parameters
To calculate robot encounter rate αrndash Run experiments in an empty arena with two robotsndash Keep track of the number of collision avoidance maneuvers
During searching αr=008 (includes wall avoidance)During homing αrsquor=005
To calculate puck encounter rate αpndash Run experiments with a single robot and pucks scattered around
the arenandash Keep track of the rate robot picks them up
αp =002
June 8 2005 Analyzing Swarms Tutorial 56118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Average Homing Time Average Homing Time ττhh
0
20
40
60
80
1 2 3 4 5 6 7 8 9 10
number of robots
ti
me
hom
ing
[ ]Nrhh ταττ prime+= 10Empirical results suggest
June 8 2005 Analyzing Swarms Tutorial 57118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
ldquoldquoInterference Strengthrdquo Interference Strengthrdquo ττ
260
300
340
380
420
460
0 2 4 6 8 10
number of robots
time
colli
sion
(s)
avoid whilesearchingavoid whilehoming
Time per collision avoidance maneuver τ=τ0[1+βN]τ0 measures ldquointerference strengthrdquo ndash studied τ0=3s τ0=15s
June 8 2005 Analyzing Swarms Tutorial 58118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Comparison with SimulationsComparison with Simulations
0
200
400
600
800
1000
1200
1400
1600
0 2 4 6 8 10
number of robots
time
(s)
avoid time = 3savoid time = 1smodel (3 s)model (1 s)
Optimal group size is smaller for higher interferenceOptimal group size is smaller for higher interference
June 8 2005 Analyzing Swarms Tutorial 59118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
CaveatsCaveatsParametersndash αr=008 (calibration) vs 004 (best fit)ndash αrsquor=005 vs 008ndash αp= 002 vs 002ndash τh
1=15s+-1s vs 16s
Despite these differences the minimal model shows good quantitative agreement with the dataModel does not take into account reverse homing or collecting behaviors of simulated robots
June 8 2005 Analyzing Swarms Tutorial 60118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
StickStick--Pulling Experiments in RobotsPulling Experiments in Robots
Collaboration in a group of reactive robotsndash Task completed only through collaborationndash Experiments with 2 ndash 6 Khepera robotsndash Minimalist robot controller
(Ijspeert Martinoli amp Billard 2001)
June 8 2005 Analyzing Swarms Tutorial 61118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Experimental ResultsExperimental ResultsKey observationsbull Different dynamics for
different ratio of robots to sticks
bull Optimal gripping time parameter
June 8 2005 Analyzing Swarms Tutorial 62118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Flowchart of the Robot ControllerFlowchart of the Robot Controllerstart look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 63118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
CoarseCoarse--grained Macroscopic Diagramgrained Macroscopic Diagram
search
grip
s u
start look for sticks
object detected
obstacle
gripped
grip amp wait
time out
teammatehelp
release
obstacleavoidance
success
Y
N
Y
N
N
NN
Y
Y
YIjspeert et al
June 8 2005 Analyzing Swarms Tutorial 64118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Model VariablesModel VariablesMacroscopic dynamic variables Ns(t) = number of robots in search state at time t
Ng(t) = number of robots gripping state at time t
M(t) = number of uncollected sticks at time t M(t)=M0-Ng(t)
search
grip
s u
Ns
Ng
June 8 2005 Analyzing Swarms Tutorial 65118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Model ParametersModel Parametersconnect the model to the real systemα = rate of encountering a stick
αRG = rate of encountering a gripping robot
γ = stick release rate (corresponds to experimental 1τ)
search
grip
s uα
αRG γ
June 8 2005 Analyzing Swarms Tutorial 66118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Mathematical Model of CollaborationMathematical Model of Collaborationsuccessful collaboration
find amp grip sticks
unsuccessful collaboration0NNN gs =+
( ) ggsGgss NNNRNMN
dtdN γαα ++minusminus= 0
consttM =)( for static environment
Initial conditions 00 )0(0)0()0( MMNNN gs ===
June 8 2005 Analyzing Swarms Tutorial 67118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Dimensional AnalysisDimensional AnalysisRewrite equations in dimensionless form by making the following transformations
ndash only the parameters β and γ appear in the eqns and determine the behavior of solutions
Collaboration ratendash rate at which robots pull sticks out
βββ G
s
RMN
NNn
==
rarr~
00
0
( )nnR minus= 1~)( ββγβ
June 8 2005 Analyzing Swarms Tutorial 68118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Searching Robots Searching Robots vsvs TimeTimeSe
arch
ing
robo
ts f
ract
ion
time
June 8 2005 Analyzing Swarms Tutorial 69118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
SteadySteady--state Searching Robots state Searching Robots vsvs γγ
β=10
β=15
β=05
June 8 2005 Analyzing Swarms Tutorial 70118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Collaboration Rate Collaboration Rate vsvs ττ
β=10
β=05
Analytic resultscritical β
optimal stick release rate
Gc R+
=1
2β
)1(2
1 Gopt R+minus=βγ
β=15
June 8 2005 Analyzing Swarms Tutorial 71118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Comparison with ExperimentsComparison with Experiments
theory experiment + simulation
4 robots
6 robots
2 robots
4 robots
6 robots
2 robots
Experiments+simulationsIjspeert et al 2001
Minimal 2-state model with γ
June 8 2005 Analyzing Swarms Tutorial 72118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Comparison with ExperimentsComparison with Experiments
Minimal 2-state modelLerman et al 2001
Complete model+simulationsMartinoli et al 2004
June 8 2005 Analyzing Swarms Tutorial 73118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
BreakBreak
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Adaptive RobotsAdaptive RobotsDynamic task allocation in a multi-foraging scenario
Dynamically achieve an appropriate division of laborFraction of Red robots = fraction of Red pucks
even as the distribution of Red pucks changes
Jones 2004
June 8 2005 Analyzing Swarms Tutorial 75118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Limitations of the approach
June 8 2005 Analyzing Swarms Tutorial 76118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Generalized Markov Process DefinitionGeneralized Markov Process DefinitionStochastic process takes values n1 n2hellip at times t0
t1 hellip is a generalized Markov process of order m if Value at time ti+1 depends on its values at times tiand ti-1 ti-2 hellip ti-m
Dynamics of the process is fully determined byndash Initial states p(n1t1) hellip p(nmtm)ndash Transition probability p(ni+1 ti+1|ni tihellip ni-m ti-m)
Continuous time p(n t+∆t|n1 t-∆thellip nm t-m∆t) - p(n t|h)History h=(t-∆thellip nm t-m∆t)
June 8 2005 Analyzing Swarms Tutorial 77118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Rate Equation for Rate Equation for gMPgMPDerive a macroscopic equation for rate of change of
Nn average number of processes with value n at time t
Same as before except transition rates are averaged over histories
sumsumprime
primeprime
primeprime minus=n
nhnnn
nhnnn NtwNtw
dttdN )()()(
June 8 2005 Analyzing Swarms Tutorial 78118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Dynamic Task AllocationDynamic Task AllocationMulti-foraging task
Robots are assigned to forage for Red or Green pucksGoal
Dynamically achieve an appropriate division of laborbull Fraction of Red robots = fraction of Red pucks
June 8 2005 Analyzing Swarms Tutorial 79118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Dynamic Task Allocation MechanismDynamic Task Allocation MechanismRobot makes local observations and adds them to memory (length m)
Each robot estimates the proportion of pucks and robots in the environment (from memory) and switches its foraging state accordingly
Robot can be modeled as a generalized Markov process of order m
June 8 2005 Analyzing Swarms Tutorial 80118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Model VariablesModel VariablesNR(t) number of robots in Red foraging stateNG(t) number of robots in Green foraging stateMR(t) number of Red pucksMG(t) number of Green pucks
SearchRed
SearchGreen
o1 o2 hellip om
NR
start
NG
June 8 2005 Analyzing Swarms Tutorial 81118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Model ParametersModel ParametersεfR G rate robots switch from Red to Green states εfG R rate robots switch from Green to Red statesαR rate robots collect Red pucksαG rate robots collect Green pucksmicro rate at which new pucks are added
SearchRed
SearchGreen
o1 o2 hellip omfR G fG R
start
June 8 2005 Analyzing Swarms Tutorial 82118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Modeling Adaptive Task AllocationModeling Adaptive Task Allocation
=dt
dNR
NNN RG =+
Initial conditions NR(t=0)=N
SearchRed
SearchGreen
o1 o2 hellip om
start
RGR Nhf )(rarrminus εGRG Nhf )(rarrε
June 8 2005 Analyzing Swarms Tutorial 83118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Mathematical Form of Mathematical Form of ffTransition rates depend on observed densities of robots and pucks
nR observed density of Red robotsmR observed density of Red pucks
Mathematical form of transition ratesfR G=(1- mR)g(nR-mR)fG R=mRg(mR-nR)
ndash Guarantees nR=mR in steady state
g(mR-nR)
June 8 2005 Analyzing Swarms Tutorial 84118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Observed DensitiesObserved Densitiest-∆t-2∆hellipt-|h|∆
Robotrsquosmemory
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
hobsR
hobsR
m
n
1
1
obsR
obsR
mn
0
0
obsR
obsR
mn
helliphellip hellip hellip
)()(
∆minus
∆minus
htmhtn
R
R
)2()2(
∆minus
∆minus
tmtn
R
R
)()(
∆minus
∆minus
tmtn
R
RAllrobots
hobsR
hobsR
mn
2
2
obsR
obsR
mn
1
1
obsR
obsR
mn
sum ∆minus=
sum ∆minus=
=
=
h
iRR
h
iRR
itmh
m
itnh
n
1
1
)(1
)(1
)()()1(
RRRRG
RRRGRnmgmf
mngmfminus=
minusminus=
rarr
rarr
June 8 2005 Analyzing Swarms Tutorial 85118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Physical ParametersPhysical Parameters
Simulations performed in 3D physically realistic world with full dynamics50 pucks 20 robots N=20 M=50Robot speed = 02 msDistance between observations = 2m
10s between observations ε=01 s-1
Robotrsquos view area Avis= 131 m2
Arena area A = 315 m2 αM=AvisMA=21
June 8 2005 Analyzing Swarms Tutorial 86118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Linear Linear ggSolutions show oscillations characteristic of delay equationsSolutions eventually relax to puck distributionMagnitude of oscillations and relaxation time depend on history length
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Dynamics of Red Robots Dynamics of Red Robots ndashndash Power Power ggSolutions relax to correct puck distributionOscillation appear for much larger history length values
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Lessons LearnedLessons LearnedStochastic processes theory provides a fruitful
framework for analyzing robot swarms
Created a models of collective swarm dynamics based on theory of stochastic processes ndash No need to know exact trajectoriesndash Allow quantitative analysis of collective behavior
Application to distributed robotic swarmsndash Reactive robotsndash Adaptive robots
Resultsndash Theoretical predictions agree with experimental resultsndash Analytic results not obtainable by other methodsndash Insights into robot design
June 8 2005 Analyzing Swarms Tutorial 89118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
LimitationsLimitationsSimplifying assumptions made by the stochastic
processes approachDilute limit ndash Robotsrsquo actions are independent of one anotherndash Robotsrsquo detection footprints do not overlap significantlyndash Valid for low densities of robots in the arena
Homogeneous robot systemsndash Robots are largely similar to one another (sensing actuation
control program)ndash Collective behavior is well characterized by mean parameter
values not distributionsndash These parameters can be calculated from individual robotrsquos
behavior
June 8 2005 Analyzing Swarms Tutorial 90118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
LimitationsLimitationsRate Equations describe dynamics of average behaviorndash Swarm behavior is well characterized by its average behavior
Fluctuations are not significantBetter description of larger systems
ndash Should be compared to results averaged over many experiments not a single trial
Spatially uniform environmentndash Transition rates well represented by average valuesndash Probability densities independent of position robotrsquos trajectories
June 8 2005 Analyzing Swarms Tutorial 91118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Swarms with Significant Spatial DependenceSwarms with Significant Spatial DependenceReconfigurable robotsndash robots made of many smaller robots (ldquomodulesrdquo)ndash tight physical constraints
analogous to solids not dilute gases
ndash spatial fields defined by modules used for local controleg to adjust shape to move through pipes
Microscopic robotsndash interesting applications involve environments with spatial
gradients
June 8 2005 Analyzing Swarms Tutorial 93118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Microscopic RobotsMicroscopic RobotsRobots the size of bacteriandash eg programmable bacteriandash eg via molecular engineered devices
Caveat canrsquot build yetndash design studies indicate plausible trade-offsndash analysis and simulation are the only options currently
Swarm motivationndash need many to have large effectndash limited hardware capabilities
June 8 2005 Analyzing Swarms Tutorial 94118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Some Design StudiesSome Design Studies
Artificial blood cells (ldquorespirocytesrdquo)ndash hold ~100x O2 as red cells in same volume
R Freitas Artif Cells Blood Substitutes and Immobilization Biotechnology 26411 (1998)
Artificial immune cellsndash fast detection of infectious microbe
Casal et al Proc of Stanford Biomedical Computation Symposium 2003
Microsurgical repair of damaged nervesndash before nerve degenerates
Hogg amp Sretavan Proc of AAAI-2005
June 8 2005 Analyzing Swarms Tutorial 95118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Task Respond to InjuryTask Respond to InjuryMonitor for chemical signalFollow gradient to sourcendash coordinate avoid too many responders
ie donrsquot block vessels
Identify infectious microbendash chemical contact sensing
fast and accurate at small distances
Pass info to attending physicianndash which immune cells canrsquot do
June 8 2005 Analyzing Swarms Tutorial 96118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
a robotrsquos microenvironmentschematic of one robot in ~20 micron blood vessel
cf artist conceptions oftenshow much more open space
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
vessels lt01mm diameter~10 total blood volume
~95 of ~500m2 surface areagt99 of ~5x104 km length
June 8 2005 Analyzing Swarms Tutorial 97118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Scenario Scenario Find Chemical SourceFind Chemical Source
1012 robots in 5-liter blood volumendash use about 10-5 of blood volume
compared to ~40 used by red cells
ndash total mass of all robots ~02 g
Power to move ~10-12 watt at 1mmsndash so if all move at once 1 wattndash vs a person at rest using ~100 watts
June 8 2005 Analyzing Swarms Tutorial 98118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
task nerve repairtask nerve repairApproaches ndash regeneration via appropriate chemicalsndash repair via replacement with graft tissue
Challengendash disconnected axons degrade in 1-2 days
no longer receive proteins from cell body
June 8 2005 Analyzing Swarms Tutorial 99118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Nerve RepairNerve Repair
graft ~1cm
undamagedhost
undamagedhost
MEMS device1mm3surgery area
(10rsquos of microns long and wide)showing a few exposed axons
in fluid lower than body temp reduces tissue injury
in vitro repair demonstrated for single axons with MEMSin vivo must measure and manipulate ~1000 axons in nerve
D Sretavan et al Neurosurgery to appear 2005June 8 2005 Analyzing Swarms Tutorial 100118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
MEMS + Swarm of Microscopic RobotsMEMS + Swarm of Microscopic Robotseg 104 microscopic robotsExamine individual axons for viabilityMove and connect axonsLong-range coordination via MEMS devices
spatial variation due to chemicals and locations of other robots
June 8 2005 Analyzing Swarms Tutorial 101118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Application for Stochastic AnalysisApplication for Stochastic AnalysisBrownian motion adds randomnessextremely large numbers of robots
Stochastic assumptions are reasonablendash but must include spatial variation
June 8 2005 Analyzing Swarms Tutorial 102118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Stochastic Processes in External FieldsStochastic Processes in External FieldsPrevious theory does not describe systems with spatial correlationsndash Swarms of agents interacting through chemical fields
Ant colonies interacting through pheromonesRobots monitoring chemicals released into fluid
Generalizationsndash Probability a stochastic process takes value k at location x at
time tp(k x t)
ndash Density of processes taking value k at location x at time tNk(x t)
ndash External field ρ(x) with which the processes are interacting
June 8 2005 Analyzing Swarms Tutorial 103118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Generalized Rate EquationGeneralized Rate Equation
int sum
int sum
primeprimeminus
primeprimeprime=part
part
)()(
)()()(
txNtxxwxd
txNtxxwxdt
txN
kj
kj
jjjk
k
ρ
ρ
Transition rates wnnrsquo depend not only on values n and nrsquo but also on spatial coordinates and field valuesndash Contains kinematics
June 8 2005 Analyzing Swarms Tutorial 104118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
SimplificationSimplificationMotion (transitions in space) is decoupled from state transitions
))(()())()(( xwxxxxxxWw jkkjkjk ρδρρδ primeminus+primeprime=
Change in position while in state k
Change in state while at position x
June 8 2005 Analyzing Swarms Tutorial 105118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Simplified Rate EquationSimplified Rate Equation
sum
sum
minus
+image=part
part
jkkj
jjjkkk
k
txNw
txNwtxNt
txN
)()(
)()()()(
ρ
ρ
Kinematics and state transitions decoupledndash Motion operator describes kinematics of a process
may depend on state of robot
ndash Transition rates describe state transitions at some locationbased only on value of field
June 8 2005 Analyzing Swarms Tutorial 106118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Motion OperatorMotion Operator
2nabla=image kk D Diffusive motion with diffusion constant D
nablasdotminusnabla=image vDkk2 Diffusion in a fluid
moving with velocity v
Chemotaxis diffusion and drift in direction of the gradient of ρ with velocity VD
)(2 ρDkk VD sdotnablaminusnabla=image
June 8 2005 Analyzing Swarms Tutorial 107118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Applying Stochastic Analysis to Applying Stochastic Analysis to Swarms with Spatial VariationSwarms with Spatial Variation
External field may also be dynamicndash eg time-dependent diffusion
Robots could modify the fieldndash eg releasing chemicals when in certain states
Need more design studies amp simulationsndash compare predictions with stochastic analysis
June 8 2005 Analyzing Swarms Tutorial 108118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Example Chemical Diffusion in FluidExample Chemical Diffusion in Fluid
flow ~1mms
10 micro
m
chemical source
30 microm
fluid flow pushing objects of size comparable to cellschemical diffusion coef ~300microm2s
robot attempts to detect chemical follow gradient to sourcefield can be complex and dynamic
One example of analysis approach ndash Galstyan et al at this conferencendash start with simple scenario
June 8 2005 Analyzing Swarms Tutorial 109118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
LimitationsLimitationsContinuous field approximationndash relevant scale of variation larger than underlying
discrete systemseg size of modules (for reconfigurable robots)eg mean-free path of molecules
ndash no abrupt changes to swarm due to small motionseg reconfigurable robot modules lose power if disconnected
June 8 2005 Analyzing Swarms Tutorial 110118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
OutlineOutline1 Stochastic processes
1 Historical perspective2 As a framework for studying robot swarms
2 Classes of stochastic processes with applications1 Ordinary Markov Processes
Basic theoryApplications Reactive robots
2 Generalized Markov ProcessesBasic theoryApplications Dynamic task allocation
3 Stochastic processes with spatial dependenceApplication Medical nano-robots
3 Summary
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
System Design ApproachesSystem Design ApproachesAnalysis with stochastic modelsSimulationExperimentsDeployment
June 8 2005 Analyzing Swarms Tutorial 112118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Stochastic Models amp SwarmsStochastic Models amp SwarmsSwarms use many robotsndash each robot fairly simple
limited view of the worldsimple control
ndash focus on tasks where overall behavior is keynot behavior of individual robots
each robot has little effect on swarm as a whole
These features stochastic models
Some examples comparing analysis with experimentsndash good match even for fairly small number of robots
June 8 2005 Analyzing Swarms Tutorial 113118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
ExamplesExamplesFinite-state machinesndash foragingndash collaborative stick pulling
Machines with memoryndash dynamic task allocation
Spatial variationndash shape changing (reconfigurable robots) ndash find chemical source (microscopic robots)
June 8 2005 Analyzing Swarms Tutorial 114118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Some Future Uses for Stochastic AnalysisSome Future Uses for Stochastic AnalysisSwarms of different types of robots egndash a few with special hardware
eg long-range communication
ndash a few large robots + many smaller ones
Accuracy of stochastic assumptionsndash eg pick designs for which analysis works well
Dynamic overall controlndash can analysis suggest how user could alter behaviors in response
to changing global task requirementseg with broadcast communication to the swarm
June 8 2005 Analyzing Swarms Tutorial 115118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Benefits of Stochastic ModelsBenefits of Stochastic ModelsEvaluate average behavior resulting from individual robot controlsndash identify thresholds for drastic changes in collective behavior
Explore wide range of design choicesndash for robot hardware capabilitiesndash for control methodsndash for swarms that cannot yet be built
helping guide future developmenteg identifying trade-offs among hardware capabilities control and task performance
June 8 2005 Analyzing Swarms Tutorial 116118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Limitations of Stochastic ModelsLimitations of Stochastic ModelsNo info on extreme individual robot behaviorsndash eg as relevant for safety guarantees
Requires estimates for transition ratesndash especially difficult if robots have complicated control program
Large fluctuationsndash eg through global broadcast instructions
June 8 2005 Analyzing Swarms Tutorial 117118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Further ReadingFurther ReadingLerman Martinoli amp Galstyan in Swarm Robotics Workshop (LNCS 3342) 2005ndash Mathematical models amp comparison with experimentsndash And references therein
Huberman amp Hogg in The Ecology of Computation North-Holland 1988ndash dynamics due to delays and uncertainty (steady-state
oscillations chaos)
June 8 2005 Analyzing Swarms Tutorial 118118
Recommended