Auton RobotDOI 10.1007/s10514-015-9542-0

Physics-inspired motion planning for information-theoretic targetdetection using multiple aerial robots

Nitin Sydney1 · Derek A. Paley2 · Donald Sofge3

Received: 24 March 2015 / Accepted: 16 December 2015© Springer Science+Business Media New York 2015

Abstract This paper presents a motion-planning strategyfor multiple, mobile sensor platforms using visual sensorswith a finite field of view. Visual sensors are used to collectposition measurements of potential targets within the searchdomain. Measurements are assimilated into a multi-targetBayesian likelihood ratio tracker that recursively produces aprobability density function over the possible target posi-tions. Vehicles are dynamically routed using a controllerbased on a concept from artificial physics, where vehiclemotion depends on the target probability at their locationas well as the distance to nearby agents. In this paradigm,the inverse log-likelihood ratio represents temperature, i.e.,high likelihood corresponds to cold temperature and lowlikelihood corresponds to high temperature. Vehicles moveat a temperature-dependent speed along the negative gradi-ent of the temperature surface while interacting locally withother agents via a Lennard-Jones potential in order to emer-gently transition between the three states of matter—solid,liquid, and gas.We show that the gradient-following behaviorcorresponds to locally maximizing the mutual informationbetween the measurements and the target state. The per-formance of the algorithm is experimentally demonstrated

B Derek A. Paleydpaley@umd.edu

Nitin Sydneynsydney@mitre.org

Donald Sofgedon.sofge@nrl.navy.mil

1 The MITRE Corporation, McClean, VA 22102, USA

2 Department of Aerospace Engineering and Institute forSystems Research, University of Maryland, College Park,MD 20742, USA

3 Naval Research Laboratory, Washington, 4555 OverlookAvenue SW, Washington, DC 20375, USA

for visual measurements in a motion capture facility usingquadrotor sensor platforms equipped with downward facingcameras.

Keywords Cooperative control · Target detection · Pathplanning

1 Introduction

Ground target tracking is now readily available using low-cost aerial vehicles equipped with visual sensors. Exampleapplications include search-and-rescue (Almurib et al. 2011)and aerial surveillance (Yang et al. 2009). The use of mobileplatforms for aerial tracking involves several challenges,including bandwidth limitations of the communication net-work and intelligent routing of the vehicles to gain informa-tion. This paper describes a physics-inspiredmotion planningstrategy based on the output of a Bayesian likelihood ratiotracker that collects noisy measurements of potential targets.The proposed motion planner has the advantage that vehi-cle motion is coupled to target detection in such a way thatthe mutual information between the target state and the mea-surements is locally maximized, while also maintaining acomplexity that is linear in the number for agents. The algo-rithm represents amanifestation of theDynamicData-DrivenApplication Systems (DDDAS) (Darema 2004) paradigm,in which sensor measurements are used to guide subsequentcollection of more data.

There are many approaches to multiple target tracking inthe literature. Probabilistic approaches such asBlanding et al.(2007), Willett et al. (2002), Stone et al. (1999), Clark andBell (2005), Huang andWang (2012) use Bayesian inferencecombined with maximum likelihood or multiple-hypothesistrackers to track multiple (moving) targets. Although the

123

Auton Robot

above approaches address how to combine sensor measure-ments to track and detect targets, only a few techniques (Peotet al. 2005; Rafi et al. 2006; Kim and Kim 2008) effec-tively move the mobile platforms to find targets and most ofthese techniques focus on the use of a single sensor platform.Other approaches (Charrow et al. 2015; Isler and Magdon-Ismail 2008) focus on mapping or optimal sensor selec-tion. Recently, several authors, (e.g., Hoffmann and Tomlin2010; Kreucher et al. 2003), have used information-basedapproaches to design optimal sensor placement/control, butit is difficult to scale these approaches to a large number ofvehicles due to the complexity in calculating mutual infor-mation (Charrow et al. 2015; Julian et al. 2012). In particular,Charrow et al. (2015) presents work that approximates themutual information over a finite time horizon for which vehi-cle paths are optimized to find a single moving target. Whilesimilar to our approach, the control used in Charrow et al.(2015) is better suited for tracking single vehicles, as opposedto the multi-target scenario considered here. Most similarto the work presented here are physics-inspired swarm con-trollers (Cuevas et al. 2014; Apker and Potter 2012; Spearset al. 2004), where vehicles switch behaviors based on their“temperature” to improve the speed and quality of informa-tion gathering. However, the behavior in these approaches isexplicitly imposed, leading to a large number of tuning para-meters. Additionally, these strategies tend to be heuristic, andhence may not have performance guarantees.

In this paper, we describe a potential-based motion-planning strategy for multiple mobile sensor platforms tocollaboratively search for multiple mobile targets usingvisual sensors with a finite field of view. The sensors givea noisy scalar measurement of target presence, e.g., the mea-surement signal increases if a target is within the sensorfield of view. Sensor measurement data are assimilated bya Bayesian likelihood ratio tracker (LRT) that uses a recur-sive formulation to produce a probability density functionover the set of possible target positions. In the LRT formula-tion, the probability builds in locations near potential targetsuntil a predetermined threshold is reached and a detectionis called. Multiple targets may be present; the LRT fusesall measurements into a single state space. The inverse ofthe state-space posterior, analogous to temperature, triggersemergent search behaviors corresponding to the states ofmatter: solid, liquid, and gas. The physics-inspired approachresults in emergent behaviors that are intuitively plausibleand therefore more amenable to human interaction with orcontrol of the multi-vehicle team. An inverse log-likelihoodformulation is used so that a higher target likelihood cor-responds to a lower temperatures and measurements with atarget present add likelihood, whereasmeasurements with notarget present subtract likelihood.

The novelty of this target detection strategy lies in the fol-lowing architecture. There are two main components to the

target detection: (1) theLRT for detection, and (2) a potential-based, physics-inspired motion planner that guides vehiclesto areas where the probability of detection is higher. Unlikeother potential-based control strategies, the proposed motionplanner avoids local minima by coupling the planner to theLRT detector. Vehicle motion is guided by the temperaturesurface according to artificial forces. The first force is alongthe negative gradient of the temperature surface; the speedalong the gradient is proportional to the temperature. The sec-ond force is a spring-like interaction between vehicles viaa Lennard-Jones potential (Lennard-Jones 1924), which iscommonly used to model molecular dynamics between ener-getic particles.We show that the gradient-following behaviorcorresponds to locally maximizing the mutual informationbetween the sensor measurements and the target state. Thecombination of these forces leads to three distinct behav-iors corresponding to three states of physical matter: (a) asolid state, in which vehicles coalesce into a lattice structurearound likely target locations; (b) a liquid state, in which thevehicles migrate towards colder regions likely to a harbortarget; and (c) a gas state, in which solitary vehicles movequickly through hotter regions where targets are unlikely tobe found. One key difference between this algorithm andother physics-inspired approaches is that the three states ofmatter are emergent, so that there are fewer tuning para-meters. The presented algorithm, though implemented in acentralized system, is inherently decentralized under an all-to-all communication architecture, and thereby amenable todistributed computation.

There are several papers in the literature, (e.g., Schwa-ger et al. 2011; Gayle et al. 2009), that use potential-basedapproaches similar to the kind used in this work, however,they suffer from several shortcomings. The primary short-coming of potential-based algorithms is that vehicles mayfall into local minima of the potential. In Gayle et al. (2009),the authors decrease the probability of remaining in a localminima by introducing an excitation factor, which is a heuris-tic method for hill climbing. One of the main benefits of thealgorithm presented in this paper is that it avoids local min-ima by coupling vehicle control to target detection via theevolvingLRT surface.Another advantage is that the LRT sur-face generated on-the-fly (and recursively updated) doesn’tincur the excessive computational burden imposedby explicitcalculation of the mutual information, which usually scalesexponentially.

We demonstrate the proposed cooperative detector firstin simulation, and then in a motion-capture experimentaltestbed, utilizing the Laboratory for Autonomous SystemsResearch at the Naval Research Laboratory in Washing-ton, D.C. The experimental testbed is composed of twoAscending Technologies Pelican quadrotors with downwardfacing Point Grey Chameleon cameras and three Spherorobots—rolling targets programmed to perform a random

123

Auton Robot

walk. Although visual sensors are described here, any sensorcharacterized by its probability of detection and probabilityof false alarm may be employed in this algorithm.

The primary contribution of this paper is a physics-basedmotion planner for a multi-vehicle, aerial sensor networkthat uses a Bayesian likelihood ratio tracker incorporatingposition information from onboard cameras to track mul-tiple moving targets. One of the novelties of the DDDASparadigm is that it directly couples the motion planningto the estimation scheme, so gathered information directlyinfluences the subsequent movement of the platforms. As asecondary contribution, we show that the gradient-followingbehavior of the planner locally maximizes the mutual infor-mation between the measurement and the target state. Unlikeother information-based techniques, the algorithmdevelopedin this paper scales linearly with the number of agents asa result of the rapid drop-off of the Lennard-Jones poten-tial. Additionally, the three states of matter are emergent,based on a combination of the speed control, Lennard-Jones potential, and gradient-descent behaviors. Vehicles aredeployed to detect an unknown number of moving targets ina computationally efficient manner that locally maximizesthe probability of detecting targets.

The rest of the paper is organized as follows. Section 2summarizes the likelihood-ratio detection and tracking cal-culations and the model used for the quadrotors’ visual sen-sors. Section 3 presents the physics-based motion-planningalgorithm for a team of mobile sensors. Section 4 showsexperimental results from the quadrotor testbed. Section 5summarizes the paper and ongoing work.

2 Likelihood-ratio detection and tracking

This section reviews the likelihood ratio tracker (Stone et al.1999) and formulates sensor and motion models for the vehi-cles and targets, respectively. The output of the tracker is usedin the physics-inspired motion planner presented in Sect. 3.

2.1 Likelihood-ratio tracking

A likelihood ratio tracker is used to estimate the positionsof possibly multiple targets. This framework is sometimescalled “track-before-detect” (Stone et al. 1999), because itis based on recursive Bayesian estimation. A Bayesian filteris a probabilistic approach for assimilating noisy measure-ments into a probability density function over the target statespace, in this case their two-dimensional position. The filterimplementation comprises the discrete steps of predictingand updating. Let θk = (xk, yk) ∈ Ω denote the target posi-tion (state) at time step k and zk denote the measurement attime k. The predict step involves computing the conditional

probability (Stone et al. 1999)

p(θk |zk−1) =∫

Ω

p(θk |θk−1)p(θk−1|zk−1)dθk−1. (1)

The measurement update is proportional to the product ofthe measurement likelihood p(zk |θk) and the predicted state(Stone et al. 1999):

p(θk |zk) = p(zk |θk)p(θk |zk−1)

p(zk |zk−1), (2)

where

p(zk |zk−1) =∫

Ω

p(zk |θk)p(θk |zk−1)dθk

is the integral of the numerator. Note thatmeasurements frommultiple sensors are assimilated by executing multiple, inde-pendent update steps.

A likelihood-ratio tracker replaces the measurement like-lihood with the measurement likelihood ratio, i.e., the ratioof two likelihood functions. The numerator of the likelihoodratio represents the conditional probability of the measure-ment given that the target is present (θ+

k ), whereas thedenominator represents the conditional probability of themeasurement given that the target is not present (θ−

k ). Thus,the log-likelihood ratio is

logL(zk |θk) = logp(zk |θ+

k )

p(zk |θ−k )

= log(p(zk |θ+k )) − log(p(zk |θ−

k )). (3)

Let P = log(p). The update step in the log-likelihood ratiotracker becomes

P(θk |zk) = logL(zk |θk)p(θk |zk−1)

p(zk |zk−1)

= P(zk |θ+k ) − P(zk |θ−

k )

+P(θk |zk−1) − P(zk |zk−1). (4)

The first term in (4) represents the new, positive informa-tion (likelihood the target is present), whereas the secondterm represents the new, negative information (likelihood thetarget is not present). The third term represents the prior infor-mation and the fourth term is a normalization constant.Whenthe log-likelihood ratio surpasses a pre-defined threshold,the target is declared detected; otherwise, the sub-thresholdtarget probabilities are maintained as hypotheses. Note, theinverse log-likelihood-ratio posterior represented by temper-ature is −P(θk |zk).

123

Auton Robot

2.2 Predict step: integrating the diffusion equation

The predict step (2) involves updating the likelihood ratio inthe absence of measurements. We allow targets to move ran-domly; we do not estimate their velocities. The target motionmodel is a random walk, which can be described mathemati-cally by the diffusion equation with constant diffusivity, alsocalled the heat equation.

Consider the two-dimensional heat equation with diffu-sivity α, i.e.,

∂P

∂t= α

(∂2P

∂x2+ ∂2P

∂y2

).

The alternating direction implicit (ADI) method (Moin(2010)) is a fractional-step method for numerically integrat-ing the heat equation. ADI employs the difference operatorsAx and Ay representing the spatial derivatives in the x andy directions, respectively. For example, if probability P isdiscretized over n × n grid points, then Ax P is a vector oflength n × n with elements

Pi+1, j − 2Pi, j + Pi−1, j

�x2, i = 1, . . . , n, j = 1, . . . , n.

Let I be the n × n identity matrix. The solution is obtainedfrom P(θk−1|zk−1) using matrix inversion to solve the fol-lowing equation for P(θk |zk−1):

(I − α�t

2Ax

) (I − α�t

2Ay

)P(θk |zk−1)

=(I + α�t

2Ax

) (I + α�t

2Ay

)P(θk−1|zk−1).

To ensure that no information crosses the boundary of thesearch domain, we enforce Neumann boundary conditionsaround the perimeter of Ω by specifying that the normalcomponent of the gradient is zero.

This motion model diffuses likelihood into locations thatare not visited. It promotes visitation of areas of low likeli-hood to ensure that a target has not moved to that location.Note that while a random walk motion model is used in thispaper, it is not necessary for the use of the proposed controlstrategy; any motion model for updating target position maybe used.

2.3 Update step: the sensor measurement model

Consider a measurement data model based on an imperfectsensor with finite field of view. Suppose the criterion locationfor a positive response is chosen such that targetswithin rangeρtarget are detectedwith probability Pd and false alarms occurwith probability Pf per time step. The sensitivity m of each

sensor is (Macmillan and Creelman 2008)

m = z(Pd) − z(Pf ), (5)

where z(·) represents the z-transformation into standard devi-ation units given by the quantile function (Macmillan andCreelman 2008)

z(p) = √2erf−1(2p − 1).

For example, Pd = 0.95 and Pf = 0.1 yields m = 2.92.Letwk represent unit-normal measurement noise in standarddeviation units at time step k. When the target is absent, themeasurement data is zk = wk , whereas when the target ispresent the measurement data is zk = m + wk . Assuming azero-mean Gaussian sensor model (Richards 2005) yields

p(zk |θ−k ) = 1√

2πexp

(− z2k

2

)(6)

p(zk |θ+k ) = 1√

2πexp

(− (zk − m)2

2

). (7)

(The sensor noise variance is absent because zk and m arealready assumed to be in standard deviation units.) The log-likelihood ratio (3) becomes

logL(zk |θk) = − (zk − m)2

2+ z2k

2= m

(zk − m

2

), (8)

where m is a function of the sensor Pd and Pf given by (5).While the methodology above would apply to any sensor

characterized by its Pd and Pf , in this paper the sensor isa camera with a known field of view (FOV) and resolution.Using standard image processing techniques (Nán 2015), thecamera places a bounding box around targets within the FOV.When there is a target present, (8) with zk = m + wk isapplied to the prior uniformly inside a disc of radius ρtargetlocated at the center of the bounding box. In areas of theFOV where there is no target, (8) with zk = wk is applied tothe prior uniformly. Pd and Pf for a camera are complicatedfunctions of the camera resolution, the size of the object inthe FOV, and the quality of the image processing algorithm.For the experiments presented in this paper, Pd and Pf weretreated as tuning parameters for the tracker and were chosento improve the performance of the estimation and controlstrategy.

3 Physics-inspired motion planning

The agent motion is driven by a combination of two artificialforces. The first force guides the vehicle down the gradientof the inverse log-likelihood surface, i.e., the temperature

123

Auton Robot

Fig. 1 Diagram depicting the three emergent behaviors of the pro-posed algorithm. Vehicles in cold areas near a target form a crystallineformation with nearby agents. Vehicles flow like a liquid from warmareas to cold areas. Since speed is proportional to temperature, vehiclesin hot areas travel quickly like gas molecules

surface, in a manner that resembles a flowing liquid. Thespeed of movement down the gradient is determined by thetemperature at the vehicle location: colder temperatures cor-respond to slower speeds. The second force is the gradient ofa Lennard-Jones potential (Lennard-Jones 1924) between theagents. The Lennard-Jones potential is a common approxi-mation used in physical chemistry to model molecular gasdynamics. Figure 1 shows the vehiclesmoving like gasmole-cules in areas where there are no targets and gathering likesolid molecules with other agents where there are likely tobe targets. In this algorithm, as opposed to other physics-inspired swarm controllers, the transitions between states ofmatter (solid, liquid, gas) are emergent.

Let r jk denote the position of agent j ∈ {1, . . . , N } at time

step k. The Lennard-Jones potential is (Lennard-Jones 1924)

V jk =

N∑i �= j

4ε(σ 12||r j

k − r ik ||−12 − σ 6||r jk − r ik ||−6

), (9)

where ε is the depth of the well and σ is the distance atwhich the potential between two agents is zero. An exampleof the Lennard-Jones potential between two agents is shownin Fig. 2. Note that the strength of the potential drops quicklyto zero for large distances. Thus, the interaction between twoagents only happens when they are close to one another.

Let T jk = −P(θk |zk, j) in (4) be the temperature at vehi-

cle j at time k and V jk be the Lennard-Jones potential (9) for

the j th vehicle. We prescribe the desired velocity for eachvehicle by a feedback controller. The desired velocity for the

Fig. 2 Example of Lennard-Jones potential; ε = σ = 1.0

j th vehicle is

r jk = −min(1 + eT

jk , vmax )kP∇T j

k − kV∇V jk , (10)

where kP and kV are control gains and vmax is the maximumspeed of the agent (the same for all agents). The depen-dence on temperature causes the vehicles to slow downwhenthey are near a possible target in order to collect more mea-surements in that area. This behavior, along with the sensoraggregation caused by the Lennard-Jones potential, seeksto increase the chances that a target will be detected. Thealgorithm works well in simulations when the gain on theLennard-Jones potential is one order of magnitude smallerthan the gradient-following gain. Additionally, σ is set toensure that the vehicles do not collide with each other, whichdepends on the scale of the vehicle. Consequently, there areonly three parameters to tune: either kP or kV , ε (the welldepth), and T0 (the detection threshold).

As seen in Eq. (10), we multiply the gradient of the tem-perature potential by a heuristic scaling factor. This scalingfactor, in combination with the Lennard-Jones potential, iswhat causes the states-of-matter behavior to be emergent. Inthe scaling factor, the speed of the vehicle is scaled by thetemperature such that the speed increases when the temper-ature is high, and decreases when the temperature is slow.This causes vehicles to “freeze” in place when they are nearan area of high probability and, conversely, to speed up whenthey are in areas of low probability. The full architecture (inpseudocode form) is shown in Algorithm 1.

Unlike other works in the field, the architecture shownin Algorithm 1 avoids local minima in an informative, non-heuristic fashion. A vehicle may fall into a local minima if itis surrounded by hills of low target likelihood. However, ifthere is no target present underneath the vehicle, the temper-ature will increase (from negative information) to the point

123

Auton Robot

Algorithm 1 Physics-Inspired Target Search and DetectionAlgorithmRequire: N, ε, σ , kP , kV1: Initialize log-likelihood ratio P0(θ |z)2: for k = 1 till end of task do3: for j = 1, . . . , N do4: Get measurement z jk5: Calculate likelihood ratio logL(z jk |θk) using (8)

6: Update: Pk(θ |z) = Pk−1 + logL(z jk |θk)7: end for8: Normalize likelihood ratio: Pk(θ |z) = Pk(θ |z)−sum((Pk(θ |z)))9: Get rk for every vehicle10: for j = 1 . . . N do11: Compute ∇V j

k (rk) and ∇P(θk |zk)12: Calculate desired velocity r j

k

13: Apply control input to platform to achieve r jk

14: end for15: end for

where the vehicle is pushed away from its location. Whilethis does not guarantee convergence to a global minimum,it does ensure that vehicles do not remain in areas of hightarget probability when a target has moved out of the region.Note, this behavior is completely emergent and is providedinherently by connecting the detection scheme to the vehiclecontrol. In addition to informatively handling local minima,we have the following theorem.

Theorem 1 The mutual information between sensor mea-surement zk and target location θk using the sensor modelgiven by (6)–(7) is locally maximized when the vehicle movesalong the gradient of the prior target distribution p(θk).

Proof Let ξk = (xk, yk) be the position of the kth quadrotor.For readability we drop the temporal subscript on all thevariables in this proof. The mutual information between zand θ is

I (z, θ) =∫

z+,z−

∫

Ω

p(θ, z|ξ) log(p(z|θ, ξ)dθdz

−∫

z+,z−p(z|ξ) log p(z|ξ)dz, (11)

where p(z|ξ) is the marginal probability density function ofthe observation in observation space. Since the sensor modelis Gaussian, the logarithm in the first term on the right handside of (11) evaluates to a constant.Hence, the double integralis independent of ξ . Thus, the mutual information is maxi-mized by maximizing the second term, which is the entropyof the observation, H(z|ξ), given the vehicle location. Themarginal probability p(z|ξ) can be written as

p(z|ξ) =∫

θ∈Dp(z|θ, ξ)p(θ)dθ. (12)

Therefore, H(z|ξ) can be expressed as

H(z|ξ) = −∫

z+,z−

∫

θ∈Dp(z|θ, ξ)p(θ)dθ

× log

⎛⎝

∫

θ∈Dp(z|θ, ξ)p(θ)dθ

⎞⎠ dz. (13)

Evaluating the integral over z yields

H(z|ξ) = −∫

θ∈Dp(z+|θ, ξ)p(θ)dθ

× log

⎛⎝

∫

θ∈Dp(z+|θ, ξ)p(θ)dθ

⎞⎠

−∫

θ∈Dp(z−|θ, ξ)p(θ)dθ

log

⎛⎝

∫

θ∈Dp(z−|θ, ξ)p(θ)dθ

⎞⎠ . (14)

Both terms on the right hand side of (14) have the formf (x) = x log(1/x), which is an increasing function of x aslong as x ≤ 1/e ≈ 0.36. In (14), the argument is the integralof p(θ) scaled by either p(z+|θ, ξ)or p(z−|θ, ξ). Since theseare both less than 1/e (see (6)–(7)) and the integral of p(θ)

is less than one in domain D, H(z|ξ) is guaranteed to be anincreasing function of the integral of p(θ).

Because the mutual information is an increasing functionof the integral of p(θ), moving along the gradient of theintegral will maximize the mutual information at the nexttime step. As a first-order approximation, assume that p(θ)

is given by the first-order Taylor expansion

p(θ) ≈ a0 + ax x + ay y, (15)

where it is assumed that coordinates in D are given by thepair (x, y). Then we have

∫

θ∈Dp(θ)dθ ≈

xk+ s2∫

xk− s2

yk+ s2∫

yk− s2

(a0 + ax x + ay y)dydx, (16)

where we have assumed that D is a square region with sidelength s for simplicity. The gradient of (16) yields

∇⎛⎝

∫

θ∈Dp(θ)dθ

⎞⎠ ≈ axs

2ex + ays2ey, (17)

123

Auton Robot

where ex and ey are unit vectors in the x and y directionsrespectively. Taking the gradient of p(θ) directly shows that

∇ p(θ) ≈ ax ex + ayey (18)

= 1

s2∇

⎛⎝

∫

θ∈Dp(θ)dθ

⎞⎠ . (19)

Thus, moving along the gradient of p(θ) is locally the sameas moving along the gradient of its integral, which completesthe proof. �

Theorem 1 shows that the sensor model coupled with agradient control allows each vehicle to greedily maximizethe expected gain in information without explicitly having tocalculate it. This property has significant benefit on the abil-ity of the search strategy to scale with numbers of vehiclesand targets, as compared to the calculationofmutual informa-tion, which generally scales exponentially. (Note, Theorem 1only holds locally.) Since the Lennard-Jones potential dropsoff dramatically with distance, each vehicle need only con-sider the position of other nearby agents. The first term in(10) scales with the calculation of the temperature surface,which scales linearly with the number of vehicles. The sec-ond term, the Lennard-Jones potential, also scales linearlywith the number of vehicles. Therefore, since the two termsadd in the controller, the algorithm scales linearly with thenumber of vehicles. Note that if the algorithm was distrib-uted, the complexity would be fixed by local vehicle densityunder a distance-based communication topology.

Algorithm performance is evaluated via simulation in theoperating domain Ω = [−5 5] × [−5 5] m. Agents andtargets are constrained to remain inΩ at all times. Parameterdefinitions and values used in the simulation are providedin Table 1. The local temperatures of the agents and targetsare obtained via linear interpolation between the grid points.The overall temperature surface is normalized by subtract-ing its mean every time step. To avoid overheating in thetemperature surface, the individual sensor updates are scaledby N , i.e., the number of agents. This scaling has the effectof making the collective detection performance in terms ofthe cumulative number of targets detected over time roughlyindependent of N .

Figure 3 shows the results of the simulation. Figure 3a andb depict snapshots of the vehicles, targets, and temperature(inverse log-likelihood) surface at time steps k = 125 and425 respectively. Vehicles are shown as red circles, undiscov-ered targets as white crosses, and discovered targets as redcrosses. At any moment, there are only three undiscoveredtargets in the domain. Once a target is detected, it turns redand another target appears with uniform probability some-where in the domain. The heat map, which describes the

Table 1 Parameter values and definitions

Parameter Value Definition

N 10 Number of agents

M 3 Instantaneous number of targets

α 0.5 Target diffusivity

σ 1.5 m Repulsive threshold

ε 0.2 Lennard-Jones potential depth

T0 −15 ◦ Target detection threshold

KP 5 Gradient-following gain

KV 0.05 Lennard-Jones gain

vmax 0.3 m/s Maximum agent speed

Pd 0.95 Probability of detection

Pf 0.10 Probability of false alarm

ρtarget α Sensor detection range

n 50 Number of grid points

in each dimension

�x 2/n Grid spacing

�t 0.03 s Time step

temperature, is blue in areas of low temperature (high likeli-hood) and red in areas of high temperature (low likelihood).

In Fig. 3a, one target has been detected. There are severalvehicles scanning the warmer regions like gas molecules,while others are clustering around potential targets in a semi-solid configuration. Figure 3c shows the cumulative numberof targets captured during the simulation (solid line) and theamount of time it takes for the vehicles to fully cover thedomain once (dashed line). The vertical red lines indicatethe times the snapshots were taken. This algorithm followsa typical coverage curve, with approximately linear growthin coverage early on, followed by slower asymptotic conver-gence to full coverage. Also note the target capture rate isapproximately linear.

4 Experimental results

As a demonstration of the proposed technique we imple-mented the algorithm in hardware using a quadrotor testbedin the Laboratory for Autonomous Systems Research locatedat the Naval Research Laboratory. Note that the experimentsconducted in this paper are for proof-of-concept and notintended to serve as a rigorous verification of the algorithm.The testbed is composed of Ascending Technologies Peli-can quadrotors operated in a facility with 115 Vicon motioncapture cameras. Figure 4 shows the prototyping high baywhere the experiments were conducted. The quadrotors areequipped with active LED motion capture markers in orderto be seen by the Vicon cameras.

123

Auton Robot

Fig. 3 a, b Snapshot of the simulation; c The number of targets cap-tured throughout the simulation and the amount of time taken to coverthe entire domain once. Vehicles cluster like solid molecules in cold

regions near the target and explore randomly like gas particles in hotregions, while flowing down the temperature gradient from hot to cold

Fig. 4 Prototyping high bay in the Laboratory for Autonomous Sys-tems Research at the Naval Research Laboratory

The quadrotors each have a downward facing Point GreyChameleon 1.3 megapixel camera with a resolution of 1296× 964 pixels. The camera is equipped with a Tamron CCTVmanual lens to adjust zoom and focus. The camera has aglobal shutter to minimize motion blur and adjustable whitebalance and shutter speed. The quadrotors have an onboardLinux computer that processes camera frames at 15 Hz,which is limited by how fast images can be captured by thecamera. A binary measurement signal sent to the LRT repre-sents whether or not a blob of a predetermined color is in theimage. For the experiments, the targets were Sphero robots,a spherical rolling toy with an orange cover. Figure 5 showsan example of the blob-tracking software used to detect twoSpheros in the field of view of the Point Grey camera.

Pose and target information from each vehicle is sent to alaptop that uses the Robot operating system (ROS) (Quigleyet al. (2009)) to process the data and compute control com-mands in a control node. ROS is an open-source software

Fig. 5 Processed camera image showing twoorangeSpheros identifiedusing a color blob detector

architecture that contains software libraries and tools to buildrobotic applications. The binary signal is generated by usinga standard blob segmentation algorithm (Nán 2015) in a cam-era processing ROS node onboard the vehicle. The vehiclealso has a ROS node to handle passing measurements to thecontrol node running on a laptop as well as receiving positionand control commands from the external laptop and motioncapture system. Control commands are sent to the vehiclethrough a WiFi connection to the message processing nodeat approximately 30 Hz. The full architecture for the experi-ment is shown in Fig. 6.

4.1 Single-vehicle, single-target experiment

To verify the algorithm using the hardware testbed, experi-ments were first conducted with a single vehicle and a single

123

Auton Robot

Fig. 6 Hardware architecture for the experiments. The quadrotors runtwo ROS nodes to process camera data and handle incoming posi-tion measurements and desired velocity. An offboard laptop receivesposition and blob detection measurements to compute the temperaturesurface and calculate the desired velocity

moving target. The Sphero served as an ideal target oncecovered with a protective colored cover, since a color-blobdetection algorithm reliably located it within the image frameof the camera. We commanded the Sphero speed but not itsdirection, yielding a randomwalk behavior due to the cover’sbumpiness.

Figure 7 shows a snapshot of an experiment. The tar-gets are surveilled using the Point Grey camera; color blobs(and false alarms) were found in each image frame. Theblob detections are fed into the LRT to calculate the inverselog-likelihood surface, from which the desired control is cal-culated. Once a target is detected, any blob within a 0.5 m

radius of the detected-target location is ignored by the LRTtracker. The Sphero was commanded to move in a randomwalk, so multiple detections of the same Sphero were possi-ble.

In this experiment, the moving Sphero was detected threetimes as it performed the random walk. The locations wherethe target was detected are indicated by the red crosses. Fig-ure 7a and 7b show snapshots of the experiment at time stepsk = 120 after the first detection and k = 220 after the thirddetection. The number of detections over time is shown inFig. 7c, where the vertical red lines indicate the times of thesnapshot shown in Fig. 7a and 7b. In a real target-trackingapplication, once the target is detected using Algorithm 1the vehicle would switch to another controller to maintaina line-of-sight track of the detected target such that mul-tiple detections are unnecessary. For illustrative purposes,the vehicle in this experiment continued using Algorithm 1even after a detection was called. The single-vehicle experi-ments used the same gains that were used in simulation; theLennard-Jones potential was omitted, as it is pertinent onlyin the multi-vehicle experiment described next.

4.2 Multi-vehicle, multi-target experiment

Amulti-vehicle experiment was also conducted with station-ary targets located at (−0.25,−0.35), (1.60, 0.40), and (1.90,−1.00) meters. Figure 8a and b show snapshots of the inverselog-likelihood surface at time steps k = 60 and 155 with theposition of the undetected targets shown as black crosses anddetected targets shown as red crosses. At the depicted timestep, there is one target that has been recently detected, onethat has yet to be discovered, and a third that in the process ofbeing detected. Figure 8c shows the number of targets cap-tured and the area covered over time. With two vehicles, alltargets were captured within 200 time steps and the vehiclescovered the full domain in approximately 400 time steps,which corresponds to 40 seconds.

Fig. 7 a, b Snapshot of the single-vehicle experiment; c The number of targets captured throughout the experimetn and the amount of time takento cover the entire domain once

123

Auton Robot

Fig. 8 a, b Snapshot of the multi-vehicle experiment; c The number of targets captured throughout the experiment and the amount of time takento cover the entire domain once

Comparing Fig. 3c to Fig. 8c, we see that the cover-age rate in the experiment follows the same patterns as thesimulated experiments, i.e., initial linear growth followedby asymptotic convergence. It was also determined that, ingeneral, the vehicles must have lower gains than in simu-lation. As observed by comparing Fig. 3c to Fig. 8c, lowergains result in a slower capture rate. With higher gains, thevehicles tended to overshoot the targets before the tempera-ture was cool enough to slow them down. The discrepancybetween simulation and experiment can be explained byobserving that the dynamics of the quadrotors are inher-ently second-order, whereas the desired velocity stated in(10) is for vehicles with first-order dynamics. The Pelicansapproximate this control by treating the output of (10) as adesired velocity to be achieved by the onboard flight con-troller.

5 Conclusion

We present a physics-inspired target search algorithm formultiple agents and multiple moving targets. The agentsassimilate noisy measurements from onboard sensors to pro-duce log-likelihood ratio estimates of target positions over theoperating domain. The inverse log-likelihood surface playsthe role of temperature so that agents speed up and slow downalong the temperature (information) gradient. Additionally,vehicles are influenced by aLennard-Jones potential betweenother local vehicles. The Lennard-Jones potential providescollision avoidance as well as local collaboration with othervehicles. The approach is amenable to heterogeneous agentsand to distributing the computation of the log-likelihood sur-face over the agents. Multi-vehicle experiments verify thevehicle routing algorithm. In ongoing work, we are conduct-ing experiments with more quadrotors and moving targets.We are alsomodifying the proposed algorithm for distributedcomputation.

Acknowledgments The authors would like to acknowledge KeithSullivan for the Sphero computer vision tracking software and TomApker and Frank Lagor for discussions pertaining to the designof the control algorithm. This work was performed at the NavalResearch Laboratory and was funded by the Office of Naval Researchunder Grant NumberN0001413WX21045, “MobileAutonomousNavyTeams for Information Surveillance and Search (MANTISS)”, andthe Air Force Office of Scientific Research under DDDAS GrantNo. FA95501310162. The views, positions and conclusions expressedherein reflect only the authors’ opinions and expressly do not reflectthose of the Office of Naval Research, Air Force Office of ScientificResearch, or the Naval Research Laboratory.

References

Almurib, H., Nathan, P., &Kumar, T. (2011). Control and path planningof quadrotor aerial vehicles for search and rescue. In Proceedingsof SICE Annual Conference (SICE), (pp. 700–705).

Apker, T. B., & Potter, M. A. (2012). Robotic swarms as solids, liquidsand gasses. In AAAI Fall Symposium Series: Human Control ofBioinspired Swarms, AAAI Technical Report FS-12-04.

Blanding,W.,Willett, P.,&Bar-Shalom,Y. (2007)Multiple target track-ing using maximum likelihood probabilistic data association. In2007 IEEE Aerospace Conference, (pp. 1–12).

Charrow, B., Kumar, V., & Michael, N. (2015). Approximate repre-sentations for multi-robot control policies that maximize mutualinformation. Autonomous Robots, 37(4), 383–400.

Charrow, B., Liu, S., Kumar, V., & Michael, N. (2015). Information-theoretic mapping using Cauchy–Schwarz quadratic mutual infor-mation. In IEEE International Conference on Robotics andAutomation (ICRA).

Clark, D., & Bell, J. (2005). Bayesian multiple target tracking in for-ward scan sonar images using the phd filter. IEEE Proceedingsof Radar, Sonar and Navigation, 152(5), 327–334. doi:10.1049/ip-rsn:20045068.

Cuevas, E., Echavarria, A., & Ramirez-Ortegon, M. A. (2014). An opti-mization algorithm inspired by the states of matter that improvesthe balance between exploration and exploitation. Applied Intelli-gence, 40(2), 256–272.

Darema, F. (2004). Dynamic data driven applications systems: A newparadigm for application simulations and measurements. In Com-putational Science-ICCS, Vol. 3038 of Lecture Notes in ComputerScience, (pp. 662–669).

Gayle, R., Moss, W., Lin, M. C., & Manocha, D. (2009). Multi-robotcoordination using generalized social potential fields. In Pro-

123

Auton Robot

ceedings of IEEE Conference on Robotics and Automation, (pp.106–113).

Hoffmann, G. M., & Tomlin, C. (2010). Mobile sensor network controlusing mutual information and particle filters. IEEE Transactionson Automatic Control, 55(1), 32–47.

Huang, C. C., &Wang, S. J. (2012). ABayesian hierarchical frameworkfor multitarget labeling and correspondence with ghost suppres-sion over multicamera surveillance system. IEEE Transactions onAutomation Science and Engineering, 9(1), 16–30.

Isler, V., & Magdon-Ismail, M. (2008). Sensor selection in arbitrarydimensions. IEEE Transactions on Automation Science and Engi-neering, 5(4), 651–660.

Julian, B. J., Angermann, M., Schwager, M., & Rus, D. (2012). Distrib-uted robotic sensor networks: An information-theoretic approach.The International Journal of Robotics Research, 31(10), 1134–1154.

Kim, J., & Kim, Y. (2008) Moving ground target tracking in denseobstacle areas using UAVs. In Proceedings of the 17th IFACWorldCongress, (pp. 8552–8557).

Kreucher, C.M., Kastella, K. D., &Hero III, A. O. (2003). Information-based sensor management for multitarget tracking. In OpticalScience and Technology, SPIE’s 48th Annual Meeting, pp. (480–489).

Lennard-Jones, J. (1924). On the determination of molecular fields.Proceedings of the Royal Society of London, 106(738), 463–477.

Macmillan, N.A.,&Creelman, C.D. (2008).Detection theory: A user’sguide (2nd ed.). New York: Psychology Press.

Moin, N. (2010).Fundamentals of engineering numerical analysis (2nded.). Cambridge: Cambridge University Press.

Nán, C. C. L. (2015). cvBlob. http://cvblob.googlecode.com.Peot, M. A., Altshuler, T. W., Breiholz, A., Bueker, R. A., Fertig,

K. W., Hawkins, A. T., & Reddy, S. (2005). Planning sensingactions for UAVs in urban domains. Proceedings of SPIE 5986,Unmanned/Unattended Sensors and Sensor Networks II, 59860J.

Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J.,Wheeler, R., Ng, A. (2009). ROS: an open-source robot operat-ing system. ICRA Workshop on Open Source Software 3(3.2).

Rafi, F., Khan, S., Shafiq, K., & Shah, M. (2006). Autonomous targetfollowing by unmanned aerial vehicles. In Proceedings of SPIE6230, Unmanned Systems Technology VIII, 623010.

Richards, M. A. (2005). Fundamentals of radar signal processing. NewDelhi: McGraw-Hill.

Schwager, M., Slotine, J. J., & Rus, D. (2011). Unifying geometric,probabilistic, and potential field approaches to multi-robot cov-erage control. Robotics research (Vol. 70, pp. 21–38). Berlin:Springer.

Spears, W. M., Spears, D. F., Hamann, J. C., & Heil, R. (2004). Distrib-uted, physics-based control of swarms of vehicles. AutonomousRobots, 17(2–3), 137–162.

Stone, L. D., Barlow, C. A., & Corwin, T. L. (1999). Bayesian multipletarget tracking. Norwood: Artech House.

Willett, P., Ruan, Y., & Streit, R. (2002). Pmht: problems and somesolutions. IEEE Transactions on Aerospace and Electronic Sys-tems, 38(3), 738–754.

Yang, H. C., AbouSleiman, R., Sababha, B., Gjoni, E., Korff, D., &Rawashdeh, O. (2009). Implementation of an autonomous surveil-lance quadrotor system. In Proceedings of AIAA Infotech, AIAA2009–2047, (pp. 1–7).

Nitin Sydney is employed atThe MITRE Corporation. Hereceived the B.S., M.S., andPh.D. degrees in aerospace engi-neering from the University ofMaryland in 2009, 2013, and2015, respectively. His researchinterests include nonlinear con-trol theory, optimal and robustpath planning, and distributedmulti-agent autonomy.

Derek A. Paley is the WillisH. Young Jr. Associate Profes-sor of Aerospace EngineeringEducation in the Department ofAerospace Engineering and theInstitute for Systems Research atthe University of Maryland. Heis the founding director of theCollective Dynamics and Con-trol Laboratory and a memberof the Alfred Gessow RotorcraftCenter, the Maryland RoboticsCenter, the Program in Neuro-science and Cognitive Science,the Burgers Program for Fluid

Dynamics, and the Applied Mathematics & Statistics, and ScientificComputation Program. Paley received the B.S. degree in AppliedPhysics from Yale University in 1997 and the Ph.D. degree in Mechan-ical and Aerospace Engineering from Princeton University in 2007. Hereceived the National Science Foundation CAREER award in 2010, thePresidential Early Career Award for Scientists and Engineers in 2012,and is co-author of Engineering Dynamics: A Comprehensive Intro-duction (Princeton University Press, 2011). Paley’s research interestsare in the area of dynamics and control, including cooperative controlof autonomous vehicles, adaptive sampling with mobile networks, andspatial modeling of biological groups.

Donald Sofge is a ComputerScientist and Roboticist at theNaval Research Laboratory with26 years of experience in Arti-ficial Intelligence and ControlSystems related R&D, and hasserved as PI or co-PI on morethan 20 federally funded R&Dprograms. Don applies nature-inspired computing paradigms tochallenging problems in sensing,artificial intelligence, and controlof robotic systems. His currentresearch focuses on control ofautonomous teams or swarms of

heterogeneous robotic systems.

123