16
Deadlock Analysis and Resolution for Multi-Robot Systems Jaskaran Singh Grover, Changliu Liu, and Katia Sycara ? The Robotics Institute, Carnegie Mellon University {jaskarag,cliu6,sycara}@andrew.cmu.edu Abstract. Collision avoidance for multirobot systems is a well studied problem. Recently, control barrier functions (CBFs) have been proposed for synthesizing controllers that guarantee collision avoidance and goal stabilization for multiple robots. However, it has been noted that reactive control synthesis methods (such as CBFs) are prone to deadlock, an equilibrium of system dynamics that causes robots to come to a standstill before reaching their goals. In this paper, we for- mally derive characteristics of deadlock in a multirobot system that uses CBFs. We propose a novel approach to analyze deadlocks resulting from optimization based controllers (CBFs) by borrowing tools from duality theory and graph enumeration. Our key insight is that system deadlock is characterized by a force- equilibrium on robots and we show how complexity of deadlock analysis increases approximately exponentially with the number of robots. This analysis allows us to interpret deadlock as a subset of the state space, and we prove that this set is non-empty, bounded and located on the boundary of the safety set. Finally, we use these properties to develop a provably correct decentralized algorithm for deadlock resolution which ensures that robots converge to their goals while avoid- ing collisions. We show simulation results of the resolution algorithm for two and three robots and experimentally validate this algorithm on Khepera-IV robots. Keywords: Collision Avoidance, Optimization and Optimal Control 1 Introduction Multirobot systems have been studied thoroughly for solving a variety of complex tasks such as search and rescue [1], sensor coverage [2] and environmental exploration [3]. Global coordinated behaviors result from executing local control laws on individual robots interacting with their neighbors [4], [5]. Typically, the local controllers running on these robots are a combination of a task-based controller responsible for completion of a primary objective and a reactive collision avoidance controller. However, including a hand-engineered safety control no longer guarantees that the original task will be satisfied [6]. This problem becomes all the more pronounced when the number of robots increases. Motivated by this bottleneck, our paper focuses on an algorithmic analysis of the performance-safety trade-offs that result from augmenting a task-based controller with collision avoidance constraints as done using CBF based quadratic programs (QPs) [7]. Although CBF-QPs mediate between safety and performance in a rigorous way, yet ulti- mately they are distributed local controllers. Such approaches exhibit a lack of look-ahead, which causes the robots to be trapped in deadlocks as noted in [8,9,10]. In deadlock, the robots stop while still being away from their goals and persist in this state unless inter- vened. This occurs because robots reach a state where conflict becomes inevitable, i.e. ? This research was supported by the DARPA Cooperative Agreement HR00111820051.

Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

Deadlock Analysis andResolution for Multi-Robot Systems

Jaskaran Singh Grover, Changliu Liu, and Katia Sycara ?

The Robotics Institute, Carnegie Mellon University{jaskarag,cliu6,sycara}@andrew.cmu.edu

Abstract. Collision avoidance for multirobot systems is a well studied problem.Recently, control barrier functions (CBFs) have been proposed for synthesizingcontrollers that guarantee collision avoidance and goal stabilization for multiplerobots. However, it has been noted that reactive control synthesis methods (suchas CBFs) are prone to deadlock, an equilibrium of system dynamics that causesrobots to come to a standstill before reaching their goals. In this paper, we for-mally derive characteristics of deadlock in a multirobot system that uses CBFs.We propose a novel approach to analyze deadlocks resulting from optimizationbased controllers (CBFs) by borrowing tools from duality theory and graphenumeration. Our key insight is that system deadlock is characterized by a force-equilibrium on robots and we show how complexity of deadlock analysis increasesapproximately exponentially with the number of robots. This analysis allows usto interpret deadlock as a subset of the state space, and we prove that this set isnon-empty, bounded and located on the boundary of the safety set. Finally, weuse these properties to develop a provably correct decentralized algorithm fordeadlock resolution which ensures that robots converge to their goals while avoid-ing collisions. We show simulation results of the resolution algorithm for two andthree robots and experimentally validate this algorithm on Khepera-IV robots.

Keywords: Collision Avoidance, Optimization and Optimal Control

1 IntroductionMultirobot systems have been studied thoroughly for solving a variety of complex taskssuch as search and rescue [1], sensor coverage [2] and environmental exploration [3].Global coordinated behaviors result from executing local control laws on individualrobots interacting with their neighbors [4], [5]. Typically, the local controllers runningon these robots are a combination of a task-based controller responsible for completionof a primary objective and a reactive collision avoidance controller. However, including ahand-engineered safety control no longer guarantees that the original task will be satisfied[6]. This problem becomes all the more pronounced when the number of robots increases.Motivated by this bottleneck, our paper focuses on an algorithmic analysis of theperformance-safety trade-offs that result from augmenting a task-based controller withcollision avoidance constraints as done using CBF based quadratic programs (QPs) [7].Although CBF-QPs mediate between safety and performance in a rigorous way, yet ulti-mately they are distributed local controllers. Such approaches exhibit a lack of look-ahead,which causes the robots to be trapped in deadlocks as noted in [8,9,10]. In deadlock, therobots stop while still being away from their goals and persist in this state unless inter-vened. This occurs because robots reach a state where conflict becomes inevitable, i.e.

? This research was supported by the DARPA Cooperative Agreement HR00111820051.

Page 2: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

2 Jaskaran Singh Grover, Changliu Liu, and Katia Sycara

Safe Set<latexit sha1_base64="1VjmQBhobvk5cccilRwcNCkCf/4=">AAAB+HicbVBNSwMxEM3Wr1o/uurRS7AI4qHsVkGPBS8eK7Uf0JaSTWfb0GyyJFmxLv0lXjwo4tWf4s1/Y9ruQVsfDDzem2FmXhBzpo3nfTu5tfWNza38dmFnd2+/6B4cNrVMFIUGlVyqdkA0cCagYZjh0I4VkCjg0ArGNzO/9QBKMynuzSSGXkSGgoWMEmOlvlvsRoF8TOskBFwHM+27Ja/szYFXiZ+REspQ67tf3YGkSQTCUE607vhebHopUYZRDtNCN9EQEzomQ+hYKkgEupfOD5/iU6sMcCiVLWHwXP09kZJI60kU2M6ImJFe9mbif14nMeF1L2UiTgwIulgUJhwbiWcp4AFTQA2fWEKoYvZWTEdEEWpsVgUbgr/88ippVsr+Rblyd1mqnmdx5NExOkFnyEdXqIpuUQ01EEUJekav6M15cl6cd+dj0Zpzspkj9AfO5w+oD5MC</latexit>

Deadlock<latexit sha1_base64="vhpgd2O/8mKSpyjQ53/Y7iuZN3I=">AAAB+HicbVBNSwMxEJ31s9aPrnr0EiyCeCi7VdBjQQ8eK9gPaJeSzaZtaLJZkqxYl/4SLx4U8epP8ea/MW33oK0PBh7vzTAzL0w408bzvp2V1bX1jc3CVnF7Z3ev5O4fNLVMFaENIrlU7RBryllMG4YZTtuJoliEnLbC0fXUbz1QpZmM7804oYHAg5j1GcHGSj231BWhfMxuKI64JKNJzy17FW8GtEz8nJQhR73nfnUjSVJBY0M41rrje4kJMqwMI5xOit1U0wSTER7QjqUxFlQH2ezwCTqxSoT6UtmKDZqpvycyLLQei9B2CmyGetGbiv95ndT0r4KMxUlqaEzmi/opR0aiaQooYooSw8eWYKKYvRWRIVaYGJtV0YbgL768TJrVin9eqd5dlGtneRwFOIJjOAUfLqEGt1CHBhBI4Rle4c15cl6cd+dj3rri5DOH8AfO5w8cRpNO</latexit>

t = 0<latexit sha1_base64="sci/Wgfdr6OLr68DzoOWG0Tt/TE=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSp4KkkV9CIUvHisaD+gDWWz3bRLN5uwOxFK6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777aysrq1vbBa2its7u3v7pYPDpolTzXiDxTLW7YAaLoXiDRQoeTvRnEaB5K1gdDv1W09cGxGrRxwn3I/oQIlQMIpWesAbt1cquxV3BrJMvJyUIUe9V/rq9mOWRlwhk9SYjucm6GdUo2CST4rd1PCEshEd8I6likbc+Nns1Ak5s0qfhLG2pZDM1N8TGY2MGUeB7YwoDs2iNxX/8zophtd+JlSSIldsvihMJcGYTP8mfaE5Qzm2hDIt7K2EDammDG06RRuCt/jyMmlWK95FpXp/Wa6d5nEU4BhO4Bw8uIIa3EEdGsBgAM/wCm+OdF6cd+dj3rri5DNH8AfO5w/Io41f</latexit>

t = 1<latexit sha1_base64="4UodqnZsNohBPadkh5Pu+9cilN0=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSp4KkkV9CIUvHisaD+gDWWz3bRLN5uwOxFK6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777aysrq1vbBa2its7u3v7pYPDpolTzXiDxTLW7YAaLoXiDRQoeTvRnEaB5K1gdDv1W09cGxGrRxwn3I/oQIlQMIpWesAbr1cquxV3BrJMvJyUIUe9V/rq9mOWRlwhk9SYjucm6GdUo2CST4rd1PCEshEd8I6likbc+Nns1Ak5s0qfhLG2pZDM1N8TGY2MGUeB7YwoDs2iNxX/8zophtd+JlSSIldsvihMJcGYTP8mfaE5Qzm2hDIt7K2EDammDG06RRuCt/jyMmlWK95FpXp/Wa6d5nEU4BhO4Bw8uIIa3EEdGsBgAM/wCm+OdF6cd+dj3rri5DNH8AfO5w/KJ41g</latexit>

t = 0<latexit sha1_base64="sci/Wgfdr6OLr68DzoOWG0Tt/TE=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSp4KkkV9CIUvHisaD+gDWWz3bRLN5uwOxFK6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777aysrq1vbBa2its7u3v7pYPDpolTzXiDxTLW7YAaLoXiDRQoeTvRnEaB5K1gdDv1W09cGxGrRxwn3I/oQIlQMIpWesAbt1cquxV3BrJMvJyUIUe9V/rq9mOWRlwhk9SYjucm6GdUo2CST4rd1PCEshEd8I6likbc+Nns1Ak5s0qfhLG2pZDM1N8TGY2MGUeB7YwoDs2iNxX/8zophtd+JlSSIldsvihMJcGYTP8mfaE5Qzm2hDIt7K2EDammDG06RRuCt/jyMmlWK95FpXp/Wa6d5nEU4BhO4Bw8uIIa3EEdGsBgAM/wCm+OdF6cd+dj3rri5DNH8AfO5w/Io41f</latexit>

t = 0<latexit sha1_base64="sci/Wgfdr6OLr68DzoOWG0Tt/TE=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSp4KkkV9CIUvHisaD+gDWWz3bRLN5uwOxFK6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777aysrq1vbBa2its7u3v7pYPDpolTzXiDxTLW7YAaLoXiDRQoeTvRnEaB5K1gdDv1W09cGxGrRxwn3I/oQIlQMIpWesAbt1cquxV3BrJMvJyUIUe9V/rq9mOWRlwhk9SYjucm6GdUo2CST4rd1PCEshEd8I6likbc+Nns1Ak5s0qfhLG2pZDM1N8TGY2MGUeB7YwoDs2iNxX/8zophtd+JlSSIldsvihMJcGYTP8mfaE5Qzm2hDIt7K2EDammDG06RRuCt/jyMmlWK95FpXp/Wa6d5nEU4BhO4Bw8uIIa3EEdGsBgAM/wCm+OdF6cd+dj3rri5DNH8AfO5w/Io41f</latexit>t = 1

<latexit sha1_base64="4UodqnZsNohBPadkh5Pu+9cilN0=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSp4KkkV9CIUvHisaD+gDWWz3bRLN5uwOxFK6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777aysrq1vbBa2its7u3v7pYPDpolTzXiDxTLW7YAaLoXiDRQoeTvRnEaB5K1gdDv1W09cGxGrRxwn3I/oQIlQMIpWesAbr1cquxV3BrJMvJyUIUe9V/rq9mOWRlwhk9SYjucm6GdUo2CST4rd1PCEshEd8I6likbc+Nns1Ak5s0qfhLG2pZDM1N8TGY2MGUeB7YwoDs2iNxX/8zophtd+JlSSIldsvihMJcGYTP8mfaE5Qzm2hDIt7K2EDammDG06RRuCt/jyMmlWK95FpXp/Wa6d5nEU4BhO4Bw8uIIa3EEdGsBgAM/wCm+OdF6cd+dj3rri5DNH8AfO5w/KJ41g</latexit>

t = 1<latexit sha1_base64="4UodqnZsNohBPadkh5Pu+9cilN0=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSp4KkkV9CIUvHisaD+gDWWz3bRLN5uwOxFK6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777aysrq1vbBa2its7u3v7pYPDpolTzXiDxTLW7YAaLoXiDRQoeTvRnEaB5K1gdDv1W09cGxGrRxwn3I/oQIlQMIpWesAbr1cquxV3BrJMvJyUIUe9V/rq9mOWRlwhk9SYjucm6GdUo2CST4rd1PCEshEd8I6likbc+Nns1Ak5s0qfhLG2pZDM1N8TGY2MGUeB7YwoDs2iNxX/8zophtd+JlSSIldsvihMJcGYTP8mfaE5Qzm2hDIt7K2EDammDG06RRuCt/jyMmlWK95FpXp/Wa6d5nEU4BhO4Bw8uIIa3EEdGsBgAM/wCm+OdF6cd+dj3rri5DNH8AfO5w/KJ41g</latexit>

t = end<latexit sha1_base64="4JQ2uIsFMVn1FO473n9eKCt+7P0=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69BKvgqSRV0ItQ8OKxgmkLbSibzaZdutmE3YlQQn+DFw+KePUHefPfuG1z0NYHA4/3ZpiZF6SCa3Scb6u0tr6xuVXeruzs7u0fVA+P2jrJFGUeTUSiugHRTHDJPOQoWDdVjMSBYJ1gfDfzO09MaZ7IR5ykzI/JUPKIU4JG8vCWyXBQrTl1Zw57lbgFqUGB1qD61Q8TmsVMIhVE657rpOjnRCGngk0r/UyzlNAxGbKeoZLETPv5/NipfW6U0I4SZUqiPVd/T+Qk1noSB6YzJjjSy95M/M/rZRjd+DmXaYZM0sWiKBM2JvbsczvkilEUE0MIVdzcatMRUYSiyadiQnCXX14l7Ubdvaw3Hq5qzbMijjKcwClcgAvX0IR7aIEHFDg8wyu8WdJ6sd6tj0VrySpmjuEPrM8fo8mOeg==</latexit>

t = end<latexit sha1_base64="4JQ2uIsFMVn1FO473n9eKCt+7P0=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69BKvgqSRV0ItQ8OKxgmkLbSibzaZdutmE3YlQQn+DFw+KePUHefPfuG1z0NYHA4/3ZpiZF6SCa3Scb6u0tr6xuVXeruzs7u0fVA+P2jrJFGUeTUSiugHRTHDJPOQoWDdVjMSBYJ1gfDfzO09MaZ7IR5ykzI/JUPKIU4JG8vCWyXBQrTl1Zw57lbgFqUGB1qD61Q8TmsVMIhVE657rpOjnRCGngk0r/UyzlNAxGbKeoZLETPv5/NipfW6U0I4SZUqiPVd/T+Qk1noSB6YzJjjSy95M/M/rZRjd+DmXaYZM0sWiKBM2JvbsczvkilEUE0MIVdzcatMRUYSiyadiQnCXX14l7Ubdvaw3Hq5qzbMijjKcwClcgAvX0IR7aIEHFDg8wyu8WdJ6sd6tj0VrySpmjuEPrM8fo8mOeg==</latexit>

Fig. 1: Two robots moving towards each other fall in deadlock. System state convergesto the boundary of safe set.

a control favoring goal stabilization will violate safety (see red dot in Fig. 1). Hence, theonly feasible strategy is to remain static. Although small perturbations can steer the sys-tem away from deadlock, there is no guarantee that robots will not fall back in deadlock.To circumvent these issues, this work addresses the following technical questions:

1. What are the characteristics of a system in deadlock ?2. What all geometric configurations of robots are admissible in deadlock ?3. How can we leverage this information to provably exit deadlock using decentralized

controllers?

To address these questions, we first review technical definitions for CBF based QPs[10] to synthesize controllers for collision avoidance and goal stabilization in section 3.In section 4, we recall the definition of deadlock and use KKT conditions to motivatea novel set theoretic interpretation of deadlock with an eye towards devising controllersthat evade/exit this set. We use graph enumeration to highlight the combinatorialcomplexity of geometric configurations of robots admissible in deadlock. Following thisdevelopment, in section 5 and section 6, we focus on the easier to analyze cases for twoand three robots respectively and examine mathematical properties of the deadlock setfor these cases. We show that this set is on the boundary of the safety set, is non-emptyand bounded. In section 7, we show how to design a provably-correct decentralizedcontroller to make the robots exit deadlock. We demonstrate this strategy on twoand three robots in simulation, and experimentally on Khepera-IV robots. Finally, weconclude with directions for future work.

2 Prior Work

Several existing methods provide inspiration for the results presented here. Of these,two are especially relevant: in the first category, we describe prior methods for collisionavoidance and in the second, we focus on deadlock resolution.

2.1 Prior Work on Avoidance ControlAvoidance control is a well-studied problem with immediate applications for planningcollision-free motions for multirobot systems. Classical avoidance control assumes a worstcase scenario with no cooperation between robots [11,12] Cooperative collision avoidance

Page 3: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

Deadlock Analysis for Multirobot Systems 3

is explored in [13,14] where avoidance control laws are computed using value functions.Velocity obstacles have been proposed in [15] for motion planning in dynamic environ-ments. They select avoidance maneuvers outside of robot’s velocity obstacles to avoidstatic and moving obstacles by means of a tree-search. While this method is prone to un-desirable oscillations, the authors in [16,17,18] propose reciprocal velocity obstacles thatare immune to such oscillations. More recently, control barrier function based controllershave been used in [6,10] to mediate between safety and performance using QPs.

2.2 Prior Work on Deadlock ResolutionThe importance of coordinating motions of multiple robots while simultaneously ensuringsafety, performance and deadlock prevention has been acknowledged in works as earlyas in [9]. Here, authors proposed scheduling algorithms to asynchronously coordinatemotions of two manipulators to ensure that their trajectories remain collision anddeadlock free. In the context of mobile robots, [19] identified the presence of deadlocksin a cooperative scenario using mobile robot troops. To the best of our knowledge, [20]were the first to propose algorithms for deadlock resolution specifically for multiplemobile robots. Their strategy for collision avoidance modifies planned paths by insertingidle times and resolves deadlocks by asking the trajectory planners of each robot toplan an alternative trajectory until deadlock is resolved. Authors in [21] proposedcoordination graphs to resolve deadlocks in robots navigating through narrow corridors.[10,22] added perturbation terms to their controllers for avoiding deadlock.

Differently from these, we characterize analytical properties of system states when indeadlock. We explicitly analyze controls from CBF based QPs and demonstrate thatintuitive explanations for systems in deadlock are indeed recovered using duality. Ouranalysis can be extended to reveal bottlenecks of any optimization based controllersynthesis method. Additionally, we use graph enumeration to highlight the complexityof this analysis. We do not consider additive perturbations for resolving deadlocks,since there are no formal guarantees. Instead, we use feedback linearization and thegeometric properties recovered from duality to guide the design of a provably correctcontroller that ensures safety, performance and deadlock resolution.

3 Avoidance Control with CBFs: ReviewIn this section, we review CBF based QPs used for synthesizing controllers that mediatebetween safety (collision avoidance) and performance (goal-stabilization) for multirobotsystems. We refer the reader to [10] for a comprehensive treatment on this subject,since our work builds on top of their approach. Assume that we have N mobile robots,each of which follows double-integrator dynamics:[

pivi

]=

[vi0

]+

[OI

]ui, (1)

where pi=(xi,yi)∈R2 represents the position of robot i, vi∈R2 represents its velocityand ui∈R2 represents the acceleration (i.e. control). The collective state of robot i is de-noted by zi=(pi,vi) and the collective state of the multirobot system is denoted as Z=(z1,z2,...,zN). Assume that each robot has maximum allowable acceleration limits |ui|≤αi that represent actuator constraints. The problem of goal stabilization with avoidancecontrol requires that each robot i must reach a goal pdi while avoiding collisions with ev-ery other robot j 6=i. For reaching a goal, assume that there is a prescribed PD controller

Page 4: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

4 Jaskaran Singh Grover, Changliu Liu, and Katia Sycara

ui(zi)=−kp(pi−pdi)−kvvi with kp,kv>0. This controller is chosen as a nominal ref-erence controller because by itself, it ensures exponential stabilization of each robot to itsgoal. However, there is no guarantee that the resulting trajectories will be collision free.Based on [10], a safety constraint is formulated for every pair of robots to ensure

mutually collision free motions. This constraint is mathematically posed by defining afunction that maps the joint state space of robots i and j to a real-valued safety indexi.e. h :R4×R4−→R. For a desired safety margin distance Ds, this index is defined as

hij=√2(αi+αj)(

∥∥∆pij∥∥−Ds)+∆pTij∆vij∥∥∆pij∥∥ . (2)

Robots i and j are considered to be collision-free if their states (zi,zj) are such thathij(zi,zj)≥0. We define “safe set” as the 0-level superset of hij i.e. Cij :={(zi,zj)∈R8 |hij(zi,zj)≥0}. The boundary of the safe set is

∂Cij={(zi,zj)∈R4|h(zi,zj)=0} (3)Assuming that the initial positions of robots i and j are in the safe set Cij, we wouldlike to synthesize controls ui and uj that ensure that future states of the robots i and jalso stay in Cij. This can be achieved by ensuring that

dhijdt≥−κ(hij), (4)

where we choose, κ(h):=h3. For the given choice of h, (4) can be rewritten as−∆pTij∆uij≤bij, where (5)

bij=∥∥∆pij∥∥h3ij+ (αi+αj)∆p

Tij∆vij√

2(αi+αj)(∥∥∆pij∥∥−Ds)+‖∆vij‖2−

(∆pTij∆vij)2∥∥∆pij∥∥2 (6)

This constraint is distributed on robots i and j as:

−∆pTijui≤αi

αi+αjbij and ∆p

Tijuj≤

αjαi+αj

bij (7)

Therefore, any ui and uj that satisfy (7) will ensure collision free trajectories for robotsi and j in the multirobot system. Note that these constraints are linear in ui and uj fora given state (zi,zj). Therefore, the feasible set of controls is convex. Assuming roboti wants to avoid collisions with its M neighbors, there will be M collision avoidanceconstraints. To mediate between safety and goal stabilization, a QP is posed thatcomputes a controller closest to the PD control ui(zi) (in 2-norm) and satisfies theM collision avoidance constraints:

minimizeui

‖ui−ui(zi)‖22

subject to −∆pTijui≤αi

αi+αjbij j∈{1,...,M}

|ui|≤αi

(8)

This QP has (M+4) constraints (M from collision avoidance with M neighbors andfour from acceleration limits). Each robot i executes a local version of this QP andcomputes its optimal u∗i at every time step. As long as the QP remains feasible, thegenerated control u∗i ensures collision avoidance of robot i with its neighbors. In thenext section, we derive an analytical expression for u∗i as a function of (z1,...,zN)to analyze the closed-loop dynamics of the ego robot and use this to investigate theincidence of deadlocks resulting from this technique.

Page 5: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

Deadlock Analysis for Multirobot Systems 5

0 5 10 15 20-4

-2

0

2

4

0 5 10 15 20-4

-2

0

2

4

(a) Positions

0 5 10 15 20-5

0

5

0 5 10 15 20-5

0

5

(b) Accelerations (control inputs)

Fig. 2: Positions and accelerations of robots falling in deadlock. Note thatlimt→∞px,y 6=pdx,y yet limt→∞ax,y=0. Video: https://tinyurl.com/y4ylzwh8

4 Analysis of N Robot Deadlock

We reviewed the formulation of multirobot collision avoidance and goal stablizationusing the framework of CBF based QPs. In this section, we will show that this approachcan result in deadlocks (depending on the initial conditions and goals of robots). Wewant to analyze qualitative properties of a robot in deadlock. Towards that end, wewill investigate the KKT conditions [23] of the problem in (8). Our goal is to use theseconditions to compute properties of geometric configurations of robots in deadlockand then exploit these properties to make the robots exit deadlock. Fig. 2 shows thestates of two robots that have fallen in deadlock while executing controllers based on(8). Notice from Fig. 2(a) that the positions of robots have converged, but not totheir respective goals. Therefore, the outputs from the prescribed PD controller willstill be non-zero after convergence. However, the control inputs from (8) have alreadyconverged to zero Fig. 2(b). From these observations, deadlock is defined as follows [10]

Definition 1. A robot i is in deadlock if u∗i =0, vi=0, ui 6=0 and pi 6=pdiIn simpler terms, for a robot to be in deadlock, it should be static i.e. its velocity should

be zero, and the output from the QP based controller should also be zero, even thoughthe reference PD controller reports non-zero acceleration since the robot is not at itsintended goal. We now look at the KKT conditions for the optimization problem in (8).

4.1 KKT Conditions

Recall that each robot i computes a control by solving a local QP as in (8). Define

aj :=−∆pij and bj= αiαi+αj

bij.We will drop subscript i and implicitly assume that the

QP is being solved for the ego robot. Hence, we rewrite (8) as:

minimizeu

‖u−u‖22subject to Au≤ b

(9)

where A :=(aT1 ;...;aTM ;eT1 ;...;−eT2 ) and b :=(b1;...;bM ;α;...;α). Let ak denote the k’th

row of A and bk denote the k’th element of b. The Lagrangian for (9) is

L(u,µ)=‖u−u‖22+M+4∑k=1

µk(aTku−bk) (10)

Let (u∗,µ∗) be the optimal primal-dual solution to (9). The KKT conditions are

Page 6: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

6 Jaskaran Singh Grover, Changliu Liu, and Katia Sycara

1. Stationarity: ∇uL(u,µ)|(u∗,µ∗)=0

=⇒u∗=u− 1

2

M+4∑k=1

µ∗kaTk . (11)

2. Primal Feasibility

aTku∗≤ bk ∀k∈{1,2,...,M+4} (12)

3. Dual Feasibility

µ∗k≥0 ∀k∈{1,2,...,M+4} (13)4. Complementary Slackness

µ∗k ·(aTku∗−bk)=0 ∀k∈{1,2,...,M+4} (14)Define the set of active and inactive constraints as follows:

A(u∗)={k∈{1,2,...,M+4}|aTku∗= bk} (15)

IA(u∗)={k∈{1,2,...,M+4}|aTku∗<bk} (16)

Using complementary slackness from (14), we deduce

µ∗k=0 ∀k∈IA(u∗) (17)Therefore, we can restrict the sum in (11) to only the set of active constraints

u∗=u− 1

2

∑k∈A(u∗)

µ∗kaTk (18)

4.2 KKT Conditions for the deadlock case

From Def. 1, we know that in deadlock, u∗=0, u 6=0 and v=0. We conclude:1. In deadlock, u∗ 6=u i.e. the solution to the QP is not equal to the prescribed PD

controller (which means that u is infeasible in deadlock i.e. aT u� b).2. u∗=0=⇒u∗ 6=±α. This implies that at least the last four constraints in Au≤ b

are inactive i.e. {M+1,M+2,M+3,M+4}∈IA(u∗) in deadlock.

Using these observations, we rewrite the KKT conditions for the deadlock case:1. Stationarity: ∇uL(u,µ)|(0,µ∗)=0

=⇒ u= 1

2

∑k∈A(u∗)

µ∗kaTk (19)

2. Primal Feasibilitybk≥0 ∀k∈{1,2,...,M+4} (20)

3. Dual Feasibilityµ∗k≥0 ∀k∈{1,2,...,M+4} (21)

4. Complementary Slacknessµ∗k ·(aTku∗−bk)=0

=⇒ µ∗k ·bk=0 ∀j∈{1,2,...,M+4} (22)Based on these conditions, we will now motivate a set-theoretic interpretation ofdeadlock. Assume that the state of the ego robot is z=(p,v) and it has M neighborsdenoted as Znb.. Define P ∈R2×4 and V ∈R2×4 appropriately to extract the positionand velocity components from z i.e. p=Pz and v=V z. Finally, define D as:

D(z |Znb.)={z∈R4 |u∗(z)=0,u(z) 6=0,V z=0,µ∗k(Z)>0 ∀ k∈A(u∗)} (23)

The set D is defined as the set of all states of the ego robot which satisfy the

Page 7: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

Deadlock Analysis for Multirobot Systems 7

criteria of being in deadlock. We have combined the conditions of deadlock into a settheoretic definition. Note that for each robot, its set of deadlock states depends onthe states of its neighboring robots. This is because the Lagrange multipliers dependon the states of all robots. The motivation behind stating this definition is to interpretdeadlock as a bonafide set in the state space of the ego robot and derive a controlstrategy that makes the robot evade/exit this set. We now rewrite this definition inmore easily interpretable conditions. From (18) and (19), note that

u∗(z)=0⇐⇒ u(z)= 1

2

∑k∈A(u∗(z))

µ∗kak (24)

Since ak=−∆pik=−P(z−zk), we rewrite (24) as:u∗(z)=0⇐⇒ u(z)=−1

2

∑k∈A(u∗(z))

µ∗kP(z−zk) (25)

We will use this condition to replace the u∗(z) = 0 criterion in the def. of D in(23). Secondly, we know that prescribed controller u(z) is a PD controller. Define thegoal state as zd=(pd,0). Noting that u(z) 6=0 and v=0,

u(z)=−kp(p−pd)−kvv 6=0⇐⇒ P(z−zd) 6=0 (26)This criterion is restating that in deadlock the ego robot is not at its goal. The finalcondition is that the velocity of the ego robot is zero i.e. v=0⇐⇒ V z=0. Combiningthese conditions, we rewrite the definition of the deadlock from (23) as follows:

D(z |Znb.)={z∈R4|u(z)=−12

∑k∈A(u∗(z))

µ∗kP(z−zk),P(z−zd) 6=0,V z=0,

µ∗k>0 ∀ k∈A(u∗(z))} (27)Building on the definition of one robot deadlock, we motivate system deadlock to bethe set of states where all robots are in deadlock and is defined as

Dsystem={(z1,z2,···,zN)∈R4N |zi∈D(zi |Zinb.) ∀i∈{1,2,···,N}) (28)For the rest of the paper, we will focus our analysis on system deadlock. This isbecause the case where only a subset of robots are in deadlock can be decomposed intosubproblems where a subset is in system deadlock and the remaining robots free to move.The next section focuses on the geometric complexity analysis of system deadlock.

4.3 Graph Enumeration based Complexity Analysis of DeadlockThe Lagrange multipliers µ∗k are in general, a nonlinear function of the state of robots z.Their values depend on which constraints are active/inactive (an example calculation isshown in (32)). An active constraint will in-turn determine the set of possible geometricconfigurations that the robots can take when they are in deadlock (sections 5 and 6)and this in turn will guide the design of our deadlock resolution algorithm (section 7).Therefore, we are interested in deriving all possible combinations of active/inactiveconstraints that the robots can assume once in deadlock. But first we derive upperand lower bounds for the number of valid configurations in system deadlock.

We can interpret an active collision avoidance constraint between robots i and j as anundirected edge between vertices i and j in a graph formed by N labeled vertices, whereeach vertex represents a robot. The following property (which follows from symmetry)allows the edges to be undirected.

Lemma 1. If robot i and j are both in deadlock and i′s constraint with j is active(inactive), then j′s constraint with i is also active (inactive).

Page 8: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

8 Jaskaran Singh Grover, Changliu Liu, and Katia Sycara

Upper Bound Given N vertices, there are NC2 distinct pairs of edges possible. Theoverall system can have any subset of those edges. Since a set with NC2 members has

2NC2 subsets, we conclude that there are 2

NC2 possible graphs. In other words, given N

robots, the number of configurations that are admissible in deadlock is 2NC2. However,

this number is an upper bound because it includes cases where a given vertex can bedisconnected from all other vertices, which is not valid in system deadlock as shown next.

Lower Bound We further impose the restriction that each vertex in the graph haveat-least one edge i.e. each robot have at-least one constraint active with some otherrobot. This is because if a robot has no active constraints i.e. µ∗k=0 ∀k then from (24),we will get u∗(z)=u(z)=0 which would contradict the definition of deadlock for thatrobot and hence contradict system deadlock. From this observation, it follows that the setof graphs that are valid in system deadlock is a superset of connected simple graphs thatcan be formed by N labeled vertices. This is because there could be graphs that are notsimply connected yet admissible in deadlock. While this argument is based on algebraicqualifiers resulting from the ‘edge’ interpretation of collision avoidance constraints, itis possible that some simply connected graphs may not be geometrically feasible dueto restrictions imposed by Euclidean geometry. Graphs that are (a) simply-connected(to enforce deadlock for each robot), (b) have N labelled vertices (since each robothas an ID), (c) are embedded in R2 (since the robots/environment are planar), (d)have Euclidean distance between connected vertices equal to Ds, (e) that betweenunconnected vertices greater than Ds, and (f) have exactly one or two edges for eachvertex, necessarily represent admissible geometric configurations of robots in systemdeadlock. The reason for qualifiers (d) and (e) is explained in the proof of theorem 1.(f) is needed because the decision variables in (9) are in R2, so there can be either oneor two active constraints per ego robot. The number of graphs meeting qualifiers (a)and (b) can be obtained using the following recurrence relation [24]

dN=2NC2− 1

N

N−1∑k=1

k NCk2N−kC2dk (29)

For N={1, 2, 3, 4}, this number is {1, 1, 4, 38}. The number of graphs meeting quali-fiers (a), (c) and (d) can be obtained by calculating the number of connected matchstickgraphs on N nodes [25]. The number of graphs meeting (b) and (d) was obtained in [26]and is exponential in N2 (for unit distance graphs). A lower bound for graphs satisfyingall qualifiers (a)-(f) can be shown to be 0.5(N+1)(N−1)! as follows (N≥3). Consider acyclic graph whose each node is the vertex of an N regular polygon with sideDs. Such agraph necessarily satisfies (a)-(f). Re-arrangements of its vertices gives rise to 0.5(N−1)!graphs. Likewise, a graph with nodes along an open chain also satisfies (a)-(f), andgives 0.5N ! rearrangements. Thus, the total is 0.5(N−1)!+0.5N !=0.5(N+1)(N−1)!It is well known that factorial overtakes exponential, thus highlighting the increase inthe number of geometric configurations. Our MATLAB simulations show that the exactnumber of configurations for N={1, 2, 3, 4} are {1, 1, 4, 18} whereas our bound gives{1, 1, 4, 15}. This simulation demonstrates the explosion in the number of possible geo-metric configurations that are admissible in system deadlock with increasing number ofrobots. Therefore for further analysis, we will restrict to the case of two and three robots.

5 Two-Robot DeadlockIn section 4, we proposed a set-theoretic definition of deadlock for a specific robotin an N robot system. In this section, we will refine the KKT conditions derived

Page 9: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

Deadlock Analysis for Multirobot Systems 9

in section 4.2 for the case of two robots in the system. This setting reveals severalimportant underlying characteristics of the system that are extendable to the N robotcase, as will be shown for N=3. One key feature of a two-robot system is that a singlerobot by itself cannot be in deadlock i.e. either both robots are in deadlock or neither.This is because the sole collision avoidance constraint is symmetric due to Lemma 1.Hence, a two-robot system can only exhibit system deadlock. Additionally since theego robot avoids collision only with the one other robot, there is no sum in (24) i.e.

u∗(z)=0⇐⇒ u(z)= 1

2µ∗a (30)

The left hand side of this equation is u(z)=−kp(pego−pd). The right hand side is12µ∗a=−1

2µ∗(pego−pneighbor). Writing this another way, we have −kp(pego−pd)+

12µ∗(pego−pneighbor)=0. The first term as represents an attractive force that pulls the

ego robot towards goal pd. Since µ∗>0, the second term represents a repulsive force

pushing the ego robot away from its neighbor. Thus, system deadlock occurs whenthe net force due to attraction and repulsion on each robot vanishes (see Fig. 3(a)).We now define the system deadlock set Dsystem using (27) and (28):

Dsystem={(z1,z2)∈R8 |u1=1

2µ∗1a1, u2=

1

2µ∗2a2, µ

∗1>0, µ∗2>0,

(P(z1−zd1),P(z2−zd2)) 6=(0,0), (V z1,V z2)=(0,0)}. (31)where a1 = −a2 = −(p1−p2). Next, we derive analytical expressions for the La-grange multipliers µ∗1,µ

∗2. Depending on whether the collision avoidance constraint is

active/inactive at the optimum, there are two cases:

Case 1: The constraint aTu≤ b is active at u=u∗ i.e. aTu∗= b=⇒ aT

(u− 1

2µ∗a

)= b

=⇒ µ∗=2aT u−b‖a‖22

(32)

Case 2: The constraint aTu≤ b is inactive at u=u∗ i.e. aTu∗<b. From comple-mentary slackness, it follows µ∗=0 and hence u∗=u. However, this contradicts thedefinition of deadlock. Hence, case 2 can never arise in deadlock.

5.1 Characteristics of two-robot deadlock

We now analyze qualitative properties of the system deadlock set towards synthesizing acontroller that will enable the robots to exit this set. We will show that when deadlockoccurs, (1) the two robots are separated by the safety distance, (2) deadlock set isnon-empty and (3) bounded and of measure zero.

Theorem 1 (Safety Margin Apart). In deadlock, the two robots are separated bythe safety distance and the robots are on the verge of violating safety (see Fig. 1, 3(a))

Proof. In (32), we proved that the collision avoidance constraint is active in deadlock.Since both robots are in deadlock, we know that both of their collision avoidance

constraints are active i.e. aT1 u∗1= b12 , aT2 u

∗2= b21 and u∗1=0 and u∗2=0. This implies

b12= b21=0. Using (6) and (7) and that in deadlock, (v1,v2)=(0,0) we get

b12=α1

α1+α2‖∆p12‖h312=0=⇒ h12=0 (33)

Page 10: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

10 Jaskaran Singh Grover, Changliu Liu, and Katia Sycara

Recall h12 from (2) and using that (v1,v2)=(0,0), we get

h12(z1,z2)=√2(α1+α2)(‖∆p12‖−Ds) (34)

Therefore, h12=0⇐⇒‖∆p12‖=Ds. Assuming QP is feasible, we disregard ‖∆p12‖=0.Therefore, ‖∆p12‖=Ds. Additionally, recalling the definition from ∂C from (3) wededuce that, in deadlock, (z1,z2)∈∂C i.e. Dsystem⊂∂C.

This result confirms our intuition, because if the robots are separated by more thanthe safety distance, then they will have wiggle room to move because they are not attheir goals and u 6=0. However, the ability to move, albeit with small velocity wouldcontradict the definition of deadlock. We now propose a family of states that are alwaysin the system deadlock set Dsystem.Theorem 2 (Dsystem is Non-Empty). ∀ kp, kv,Ds > 0,∃ a family of states(z∗1,z

∗2)∈Dsystem. These states are such that the robots and their goals are all collinear.

Proof. To prove this theorem, we propose a set of candidate states (z∗1,z∗2) and show

that they satisfy the definition of deadlock (31). See Fig. 3(a) for an illustration ofgeometric quantities referred to in this proof.

Let p∗1 = αpd1 +(1−α)pd2 and p∗2 = p∗1−Dseβ where β = tan−1(

yd2−yd1xd2−xd1

) and

α∈(0,1). Note that p∗1,p∗2,pd1,pd2 are collinear by construction. Let z∗1=(p∗1,0) andz∗2=(p∗2,0). Then we will show that Z∗=(z∗1,z

∗2)∈Dsystem. Note that

a1=−(p∗1−p∗2)=−Dseβu1=−kp(p∗1−pd1) (35)

From definition, eβ=1DG

(xd2−xd1,yd2−yd1) where DG=∥∥pd2−pd1∥∥ is the distance

between the goals. Therefore, we havep∗1−pd1 =−(1−α)pd1+(1−α)pd2

=(1−α)DGeβ (36)Substituting (36) in (35) gives

u1=−kp(1−α)DGeβ (37)From (35) and (37), we deduce that Lagrange multiplier µ1

µ1=2aT1 u1

‖a1‖22=2kp(1−α)

DGDs

>0 ∀α∈(0,1)

=⇒ 1

2µ1a1=−

1

22kp(1−α)

DGDs

Dseβ=u1 (38)

Hence, in (38), we have shown that u1=12µ1a1 which is one condition in the definition

of the deadlock set. Similarly, we can show that u2=12µ2a2. Also note that in (38)

we have shown that the Lagrange multiplier µ1 is positive, which is another conditionin (31). We can similarly show that µ2>0. Finally, note that for our choice of states,v∗1=v

∗2=0 and we have restricted α∈(0,1) so we can ensure that p∗i 6=pdi Hence, the

proposed states are always in deadlock.

Theorem 3 (Dsystem is bounded). The system deadlock set is measure zero.

Proof. Following the definition of p∗1 and p∗2 from theorem 1 and theorem 2, we canshow that when two robots are in deadlock, their positions satisfy∥∥(p1−pd1)∥∥+∥∥(p2−pd2)∥∥=Ds+DGThis can be verified by straightforward substitution. From this constraint it is evident,that the deadlock set is not “large”, bounded and of measure zero. That is why, randomperturbations are one feasible way to resolve deadlock.

Page 11: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

Deadlock Analysis for Multirobot Systems 11

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

a1 = ��p

12

<latexit sha1_base64="aA9QXuRNY9EHCa180daiSgc2LhA=">AAACF3icbVDLSsNAFJ3UV62vqEs3g0VwY0hqQTdCQRcuK9gHNCFMJtN26GQSZiZCCfkLN/6KGxeKuNWdf+OkzUJbDwxzOOde7r0nSBiVyra/jcrK6tr6RnWztrW9s7tn7h90ZZwKTDo4ZrHoB0gSRjnpKKoY6SeCoChgpBdMrgu/90CEpDG/V9OEeBEacTqkGCkt+ablBjEL5TTSX4Zy34FX8My9IUwh+NtKcj9zGrlv1m3LngEuE6ckdVCi7ZtfbhjjNCJcYYakHDh2orwMCUUxI3nNTSVJEJ6gERloylFEpJfN7srhiVZCOIyFflzBmfq7I0ORLPbTlRFSY7noFeJ/3iBVw0svozxJFeF4PmiYMqhiWIQEQyoIVmyqCcKC6l0hHiOBsNJR1nQIzuLJy6TbsJxzq3HXrLeaZRxVcASOwSlwwAVogVvQBh2AwSN4Bq/gzXgyXox342NeWjHKnkPwB8bnD9fSn64=</latexit>

� + ⇡<latexit sha1_base64="jUaCXoLXWp5ReLQwcHUYh7j/pGE=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSIIQkmqYI8FLx4r2A9pStlsN+3S3STsToQS+iu8eFDEqz/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGSxHxJgqUvJNoTlUgeTsY38789hPXRsTRA04S3lN0GIlQMIpWevQDjvTST0S/VHYr7hxklXg5KUOORr/05Q9ilioeIZPUmK7nJtjLqEbBJJ8W/dTwhLIxHfKupRFV3PSy+cFTcm6VAQljbStCMld/T2RUGTNRge1UFEdm2ZuJ/3ndFMNaLxNRkiKP2GJRmEqCMZl9TwZCc4ZyYgllWthbCRtRTRnajIo2BG/55VXSqla8q0r1/rpcr+VxFOAUzuACPLiBOtxBA5rAQMEzvMKbo50X5935WLSuOfnMCfyB8/kDdRSQJg==</latexit>

-4 -3 -2 -1 0 1 2 3 4 5-4

-3

-2

-1

0

1

2

3

4

�<latexit sha1_base64="5SbAXYCDLz6mp12N1T52o+MOmfY=">AAAB7HicbVBNS8NAEN3Ur1q/qoIXL8EieCpJPejBQ8GLxwqmLbShbLaTdulmE3YnSgj9DV48KCLe/EHe/AH+D7cfB219MPB4b4aZeUEiuEbH+bIKK6tr6xvFzdLW9s7uXnn/oKnjVDHwWCxi1Q6oBsEleMhRQDtRQKNAQCsYXU/81j0ozWN5h1kCfkQHkoecUTSS1w0Aaa9ccarOFPYyceekUj9qPiTZ93ujV/7s9mOWRiCRCap1x3US9HOqkDMB41I31ZBQNqID6BgqaQTaz6fHju1To/TtMFamJNpT9fdETiOtsygwnRHFoV70JuJ/XifF8NLPuUxSBMlmi8JU2Bjbk8/tPlfAUGSGUKa4udVmQ6ooQ5NPyYTgLr68TJq1qnterd2aNK7IDEVyTE7IGXHJBamTG9IgHmGEk0fyTF4saT1Zr9bbrLVgzWcOyR9YHz+KJpIu</latexit>

f31rep

<latexit sha1_base64="RV1MeyfwH4JSDwaOM559NQkzc5U=">AAAB83icbVC7SgNBFL3rM8ZX1EawGQxCqrCbFFoGbCwjmAds1jA7mU2GzM4OM7NCWPIbaSwUsfVLrLTTr3HyKDTxwIXDOfdy7z2h5Ewb1/1y1tY3Nre2czv53b39g8PC0XFTJ6kitEESnqh2iDXlTNCGYYbTtlQUxyGnrXB4PfVbD1Rplog7M5I0iHFfsIgRbKzUie6zqjfuZorKcbdQdMvuDGiVeAtSrJ1++5PS+0e9W/js9BKSxlQYwrHWvudKE2RYGUY4Hec7qaYSkyHuU99SgWOqg2x28xhdWKWHokTZEgbN1N8TGY61HsWh7YyxGehlbyr+5/mpia6CjAmZGirIfFGUcmQSNA0A9ZiixPCRJZgoZm9FZIAVJsbGlLcheMsvr5JmpexVy5Vbm4YLc+TgDM6hBB5cQg1uoA4NICBhAk/w7KTOo/PivM5b15zFzAn8gfP2A0gklYU=</latexit>

f32rep

<latexit sha1_base64="8ACMmvx2vkVsTQIvohHGw167qjs=">AAAB83icbVC7SgNBFL3rM8ZX1EawGQxCqrCbFFoGbCwjmAdsYpidzCZDZmeHmVkhLPsbaSwUsfVLrLTTr3HyKDTxwIXDOfdy7z2B5Ewb1/1y1tY3Nre2czv53b39g8PC0XFTx4kitEFiHqt2gDXlTNCGYYbTtlQURwGnrWB0PfVbD1RpFos7M5a0G+GBYCEj2FipE96n1UrWSxWVWa9QdMvuDGiVeAtSrJ1++5PS+0e9V/js9GOSRFQYwrHWvudK002xMoxwmuU7iaYSkxEeUN9SgSOqu+ns5gxdWKWPwljZEgbN1N8TKY60HkeB7YywGeplbyr+5/mJCa+6KRMyMVSQ+aIw4cjEaBoA6jNFieFjSzBRzN6KyBArTIyNKW9D8JZfXiXNStmrliu3Ng0X5sjBGZxDCTy4hBrcQB0aQEDCBJ7g2UmcR+fFeZ23rjmLmRP4A+ftB0mvlYY=</latexit>

f3datt

<latexit sha1_base64="F5cyOt7G+dOoSImd3HviJ++nK1Y=">AAAB83icbVDLSsNAFJ3UV62vqEs3g1VwVZJ2oTsLblxWsQ9oY5lMJu3QySTM3Agl5DfcuFDErT/jTvAz/ACnj4W2HrhwOOde7r3HTwTX4DifVmFldW19o7hZ2tre2d2z9w9aOk4VZU0ai1h1fKKZ4JI1gYNgnUQxEvmCtf3R1cRvPzCleSzvYJwwLyIDyUNOCRipF95ntSDvZwQg79tlp+JMgZeJOyflun1yezn4/mr07Y9eENM0YhKoIFp3XScBLyMKOBUsL/VSzRJCR2TAuoZKEjHtZdObc3xqlACHsTIlAU/V3xMZibQeR77pjAgM9aI3Ef/zuimEF17GZZICk3S2KEwFhhhPAsABV4yCGBtCqOLmVkyHRBEKJqaSCcFdfHmZtKoVt1ap3pg0HDRDER2hY3SGXHSO6ugaNVATUZSgR/SMXqzUerJerbdZa8GazxyiP7DefwCxtpUP</latexit>

p3<latexit sha1_base64="mmn5wt5Q9+RM0Sjt/ah7e/FQf5s=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0laQY8VLx4r2lpoQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCoreNUMWyxWMSqE1CNgktsGW4EdhKFNAoEPgbjm5n/+IRK81g+mEmCfkSHkoecUWOl+6Rf75crbtWdg6wSLycVyNHsl796g5ilEUrDBNW667mJ8TOqDGcCp6VeqjGhbEyH2LVU0gi1n81PnZIzqwxIGCtb0pC5+nsio5HWkyiwnRE1I73szcT/vG5qwis/4zJJDUq2WBSmgpiYzP4mA66QGTGxhDLF7a2EjaiizNh0SjYEb/nlVdKuVb16tXZ3UWlc53EU4QRO4Rw8uIQG3EITWsBgCM/wCm+OcF6cd+dj0Vpw8plj+APn8wcDiY2d</latexit>

e�<latexit sha1_base64="gCdXGSES8icp12Aphsp5s09cQpM=">AAACBXicbVDLSsNAFJ3UV62vqEtdBIvgqiRVsMuCG5cV7APaEiaT23boJBNmboQSsnHjr7hxoYhb/8Gdf+P0sdDWA8MczrmXe+8JEsE1uu63VVhb39jcKm6Xdnb39g/sw6OWlqli0GRSSNUJqAbBY2giRwGdRAGNAgHtYHwz9dsPoDSX8T1OEuhHdBjzAWcUjeTbp70RxawXSBHqSWS+DPLcNwIgzX277FbcGZxV4i1ImSzQ8O2vXihZGkGMTFCtu56bYD+jCjkTkJd6qYaEsjEdQtfQmEag+9nsitw5N0roDKQyL0Znpv7uyGikpzuayojiSC97U/E/r5vioNbPeJykCDGbDxqkwkHpTCNxQq6AoZgYQpniZleHjaiiDE1wJROCt3zyKmlVK95lpXp3Va7XFnEUyQk5IxfEI9ekTm5JgzQJI4/kmbySN+vJerHerY95acFa9ByTP7A+fwAVGJmN</latexit>

frep<latexit sha1_base64="fnyG6bfGksDBQcm9d9vhQs/XvL8=">AAAB7nicbVDLSgNBEOyNrxhfUY9eBoPgKexGQY8BLx4jmAckIcxOepMhs7PDzKwQlnyEFw+KePV7vPk3TpI9aGJBQ1HVTXdXqAQ31ve/vcLG5tb2TnG3tLd/cHhUPj5pmSTVDJssEYnuhNSg4BKblluBHaWRxqHAdji5m/vtJ9SGJ/LRThX2YzqSPOKMWie1o0GmUc0G5Ypf9Rcg6yTISQVyNAblr94wYWmM0jJBjekGvrL9jGrLmcBZqZcaVJRN6Ai7jkoao+lni3Nn5MIpQxIl2pW0ZKH+nshobMw0Dl1nTO3YrHpz8T+vm9rotp9xqVKLki0XRakgNiHz38mQa2RWTB2hTHN3K2FjqimzLqGSCyFYfXmdtGrV4Kpae7iu1P08jiKcwTlcQgA3UId7aEATGEzgGV7hzVPei/fufSxbC14+cwp/4H3+AKTJj7Y=</latexit>

frep<latexit sha1_base64="fnyG6bfGksDBQcm9d9vhQs/XvL8=">AAAB7nicbVDLSgNBEOyNrxhfUY9eBoPgKexGQY8BLx4jmAckIcxOepMhs7PDzKwQlnyEFw+KePV7vPk3TpI9aGJBQ1HVTXdXqAQ31ve/vcLG5tb2TnG3tLd/cHhUPj5pmSTVDJssEYnuhNSg4BKblluBHaWRxqHAdji5m/vtJ9SGJ/LRThX2YzqSPOKMWie1o0GmUc0G5Ypf9Rcg6yTISQVyNAblr94wYWmM0jJBjekGvrL9jGrLmcBZqZcaVJRN6Ai7jkoao+lni3Nn5MIpQxIl2pW0ZKH+nshobMw0Dl1nTO3YrHpz8T+vm9rotp9xqVKLki0XRakgNiHz38mQa2RWTB2hTHN3K2FjqimzLqGSCyFYfXmdtGrV4Kpae7iu1P08jiKcwTlcQgA3UId7aEATGEzgGV7hzVPei/fufSxbC14+cwp/4H3+AKTJj7Y=</latexit>

fatt<latexit sha1_base64="wIXDzMFbYJKsZRpPXJZ2FMASVeo=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeCF48V7Ae0oWy2m3bpZhN2J0IJ+RFePCji1d/jzX/jts1BWx8MPN6bYWZekEhh0HW/ndLG5tb2Tnm3srd/cHhUPT7pmDjVjLdZLGPdC6jhUijeRoGS9xLNaRRI3g2md3O/+8S1EbF6xFnC/YiOlQgFo2ilbjjMKGI+rNbcursAWSdeQWpQoDWsfg1GMUsjrpBJakzfcxP0M6pRMMnzyiA1PKFsSse8b6miETd+tjg3JxdWGZEw1rYUkoX6eyKjkTGzKLCdEcWJWfXm4n9eP8Xw1s+ESlLkii0XhakkGJP572QkNGcoZ5ZQpoW9lbAJ1ZShTahiQ/BWX14nnUbdu6o3Hq5rTbeIowxncA6X4MENNOEeWtAGBlN4hld4cxLnxXl3PpatJaeYOYU/cD5/AKfAj7g=</latexit>

fatt<latexit sha1_base64="wIXDzMFbYJKsZRpPXJZ2FMASVeo=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeCF48V7Ae0oWy2m3bpZhN2J0IJ+RFePCji1d/jzX/jts1BWx8MPN6bYWZekEhh0HW/ndLG5tb2Tnm3srd/cHhUPT7pmDjVjLdZLGPdC6jhUijeRoGS9xLNaRRI3g2md3O/+8S1EbF6xFnC/YiOlQgFo2ilbjjMKGI+rNbcursAWSdeQWpQoDWsfg1GMUsjrpBJakzfcxP0M6pRMMnzyiA1PKFsSse8b6miETd+tjg3JxdWGZEw1rYUkoX6eyKjkTGzKLCdEcWJWfXm4n9eP8Xw1s+ESlLkii0XhakkGJP572QkNGcoZ5ZQpoW9lbAJ1ZShTahiQ/BWX14nnUbdu6o3Hq5rTbeIowxncA6X4MENNOEeWtAGBlN4hld4cxLnxXl3PpatJaeYOYU/cD5/AKfAj7g=</latexit>

Ds<latexit sha1_base64="kd2F4Qg7/dtYFWs4c99n+gU4M/w=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSJ4Kkkt6LGgB48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2Z++4lrI2L1iJOE+xEdKhEKRtFKD7d90y+V3Yo7B1klXk7KkKPRL331BjFLI66QSWpM13MT9DOqUTDJp8VeanhC2ZgOeddSRSNu/Gx+6pScW2VAwljbUkjm6u+JjEbGTKLAdkYUR2bZm4n/ed0Uw2s/EypJkSu2WBSmkmBMZn+TgdCcoZxYQpkW9lbCRlRThjadog3BW355lbSqFe+yUr2vleu1PI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEdmI2k</latexit>

e✓(t)<latexit sha1_base64="Zot5QIGzIEVMv3w9AldKasf5BNQ=">AAACCXicbVA9SwNBEN2LXzF+RS1tFoMQm3AXBVMGbCwjmETIHWFvM0mW7H2wOyeE41ob/4qNhSK2/gM7/42b5ApNfLDs470ZZub5sRQabfvbKqytb2xuFbdLO7t7+wflw6OOjhLFoc0jGal7n2mQIoQ2CpRwHytggS+h60+uZ373AZQWUXiH0xi8gI1CMRScoZH6ZeqOGaauH8mBngbmSyHL+qmLY0BWxfOsX67YNXsOukqcnFRIjla//OUOIp4EECKXTOueY8fopUyh4BKykptoiBmfsBH0DA1ZANpL55dk9MwoAzqMlHkh0rn6uyNlgZ7taSoDhmO97M3E/7xegsOGl4owThBCvhg0TCTFiM5ioQOhgKOcGsK4EmZXysdMMY4mvJIJwVk+eZV06jXnola/vaw0G3kcRXJCTkmVOOSKNMkNaZE24eSRPJNX8mY9WS/Wu/WxKC1Yec8x+QPr8we+pZr0</latexit>

✓(t)<latexit sha1_base64="q/j84qCeYurHXHKzyv1mbANBrUE=">AAAB8HicbVDLSgNBEJyNrxhfUY9eFoMQL2E3CuYY8OIxgnlIsoTZSW8yZGZ2mekVQshXePGgiFc/x5t/4yTZgyYWNBRV3XR3hYngBj3v28ltbG5t7+R3C3v7B4dHxeOTlolTzaDJYhHrTkgNCK6giRwFdBINVIYC2uH4du63n0AbHqsHnCQQSDpUPOKMopUeezgCpGW87BdLXsVbwF0nfkZKJEOjX/zqDWKWSlDIBDWm63sJBlOqkTMBs0IvNZBQNqZD6FqqqAQTTBcHz9wLqwzcKNa2FLoL9ffElEpjJjK0nZLiyKx6c/E/r5tiVAumXCUpgmLLRVEqXIzd+ffugGtgKCaWUKa5vdVlI6opQ5tRwYbgr768TlrVin9Vqd5fl+q1LI48OSPnpEx8ckPq5I40SJMwIskzeSVvjnZenHfnY9mac7KZU/IHzucPQ9+QBQ==</latexit>

a2 = ��p

12

<latexit sha1_base64="vWTU2fiPpwwWz9Qi+4Fau3xhTnI=">AAACF3icbVDLSsNAFJ3UV62vqEs3g0VwY0iiYDdCQRcuK9gHNCFMJpN26OTBzEQoIX/hxl9x40IRt7rzb5y0WWjrgWEO59zLvff4KaNCmua3VltZXVvfqG82trZ3dvf0/YOeSDKOSRcnLOEDHwnCaEy6kkpGBiknKPIZ6fuT69LvPxAuaBLfy2lK3AiNYhpSjKSSPN1w/IQFYhqpL0eFZ8MreObcECYR/G2lhZdbduHpTdMwZ4DLxKpIE1ToePqXEyQ4i0gsMUNCDC0zlW6OuKSYkaLhZIKkCE/QiAwVjVFEhJvP7irgiVICGCZcvVjCmfq7I0eRKPdTlRGSY7HoleJ/3jCTYcvNaZxmksR4PijMGJQJLEOCAeUESzZVBGFO1a4QjxFHWKooGyoEa/HkZdKzDevcsO8umu1WFUcdHIFjcAoscAna4BZ0QBdg8AiewSt40560F+1d+5iX1rSq5xD8gfb5A9qon7M=</latexit>

(a) Two Robot Equilibrium

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

a1 = ��p

12

<latexit sha1_base64="aA9QXuRNY9EHCa180daiSgc2LhA=">AAACF3icbVDLSsNAFJ3UV62vqEs3g0VwY0hqQTdCQRcuK9gHNCFMJtN26GQSZiZCCfkLN/6KGxeKuNWdf+OkzUJbDwxzOOde7r0nSBiVyra/jcrK6tr6RnWztrW9s7tn7h90ZZwKTDo4ZrHoB0gSRjnpKKoY6SeCoChgpBdMrgu/90CEpDG/V9OEeBEacTqkGCkt+ablBjEL5TTSX4Zy34FX8My9IUwh+NtKcj9zGrlv1m3LngEuE6ckdVCi7ZtfbhjjNCJcYYakHDh2orwMCUUxI3nNTSVJEJ6gERloylFEpJfN7srhiVZCOIyFflzBmfq7I0ORLPbTlRFSY7noFeJ/3iBVw0svozxJFeF4PmiYMqhiWIQEQyoIVmyqCcKC6l0hHiOBsNJR1nQIzuLJy6TbsJxzq3HXrLeaZRxVcASOwSlwwAVogVvQBh2AwSN4Bq/gzXgyXox342NeWjHKnkPwB8bnD9fSn64=</latexit>

� + ⇡<latexit sha1_base64="jUaCXoLXWp5ReLQwcHUYh7j/pGE=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSIIQkmqYI8FLx4r2A9pStlsN+3S3STsToQS+iu8eFDEqz/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGSxHxJgqUvJNoTlUgeTsY38789hPXRsTRA04S3lN0GIlQMIpWevQDjvTST0S/VHYr7hxklXg5KUOORr/05Q9ilioeIZPUmK7nJtjLqEbBJJ8W/dTwhLIxHfKupRFV3PSy+cFTcm6VAQljbStCMld/T2RUGTNRge1UFEdm2ZuJ/3ndFMNaLxNRkiKP2GJRmEqCMZl9TwZCc4ZyYgllWthbCRtRTRnajIo2BG/55VXSqla8q0r1/rpcr+VxFOAUzuACPLiBOtxBA5rAQMEzvMKbo50X5935WLSuOfnMCfyB8/kDdRSQJg==</latexit>

-4 -3 -2 -1 0 1 2 3 4 5-4

-3

-2

-1

0

1

2

3

4

�<latexit sha1_base64="5SbAXYCDLz6mp12N1T52o+MOmfY=">AAAB7HicbVBNS8NAEN3Ur1q/qoIXL8EieCpJPejBQ8GLxwqmLbShbLaTdulmE3YnSgj9DV48KCLe/EHe/AH+D7cfB219MPB4b4aZeUEiuEbH+bIKK6tr6xvFzdLW9s7uXnn/oKnjVDHwWCxi1Q6oBsEleMhRQDtRQKNAQCsYXU/81j0ozWN5h1kCfkQHkoecUTSS1w0Aaa9ccarOFPYyceekUj9qPiTZ93ujV/7s9mOWRiCRCap1x3US9HOqkDMB41I31ZBQNqID6BgqaQTaz6fHju1To/TtMFamJNpT9fdETiOtsygwnRHFoV70JuJ/XifF8NLPuUxSBMlmi8JU2Bjbk8/tPlfAUGSGUKa4udVmQ6ooQ5NPyYTgLr68TJq1qnterd2aNK7IDEVyTE7IGXHJBamTG9IgHmGEk0fyTF4saT1Zr9bbrLVgzWcOyR9YHz+KJpIu</latexit>

f31rep

<latexit sha1_base64="RV1MeyfwH4JSDwaOM559NQkzc5U=">AAAB83icbVC7SgNBFL3rM8ZX1EawGQxCqrCbFFoGbCwjmAds1jA7mU2GzM4OM7NCWPIbaSwUsfVLrLTTr3HyKDTxwIXDOfdy7z2h5Ewb1/1y1tY3Nre2czv53b39g8PC0XFTJ6kitEESnqh2iDXlTNCGYYbTtlQUxyGnrXB4PfVbD1Rplog7M5I0iHFfsIgRbKzUie6zqjfuZorKcbdQdMvuDGiVeAtSrJ1++5PS+0e9W/js9BKSxlQYwrHWvudKE2RYGUY4Hec7qaYSkyHuU99SgWOqg2x28xhdWKWHokTZEgbN1N8TGY61HsWh7YyxGehlbyr+5/mpia6CjAmZGirIfFGUcmQSNA0A9ZiixPCRJZgoZm9FZIAVJsbGlLcheMsvr5JmpexVy5Vbm4YLc+TgDM6hBB5cQg1uoA4NICBhAk/w7KTOo/PivM5b15zFzAn8gfP2A0gklYU=</latexit>

f32rep

<latexit sha1_base64="8ACMmvx2vkVsTQIvohHGw167qjs=">AAAB83icbVC7SgNBFL3rM8ZX1EawGQxCqrCbFFoGbCwjmAdsYpidzCZDZmeHmVkhLPsbaSwUsfVLrLTTr3HyKDTxwIXDOfdy7z2B5Ewb1/1y1tY3Nre2czv53b39g8PC0XFTx4kitEFiHqt2gDXlTNCGYYbTtlQURwGnrWB0PfVbD1RpFos7M5a0G+GBYCEj2FipE96n1UrWSxWVWa9QdMvuDGiVeAtSrJ1++5PS+0e9V/js9GOSRFQYwrHWvudK002xMoxwmuU7iaYSkxEeUN9SgSOqu+ns5gxdWKWPwljZEgbN1N8TKY60HkeB7YywGeplbyr+5/mJCa+6KRMyMVSQ+aIw4cjEaBoA6jNFieFjSzBRzN6KyBArTIyNKW9D8JZfXiXNStmrliu3Ng0X5sjBGZxDCTy4hBrcQB0aQEDCBJ7g2UmcR+fFeZ23rjmLmRP4A+ftB0mvlYY=</latexit>

f3datt

<latexit sha1_base64="F5cyOt7G+dOoSImd3HviJ++nK1Y=">AAAB83icbVDLSsNAFJ3UV62vqEs3g1VwVZJ2oTsLblxWsQ9oY5lMJu3QySTM3Agl5DfcuFDErT/jTvAz/ACnj4W2HrhwOOde7r3HTwTX4DifVmFldW19o7hZ2tre2d2z9w9aOk4VZU0ai1h1fKKZ4JI1gYNgnUQxEvmCtf3R1cRvPzCleSzvYJwwLyIDyUNOCRipF95ntSDvZwQg79tlp+JMgZeJOyflun1yezn4/mr07Y9eENM0YhKoIFp3XScBLyMKOBUsL/VSzRJCR2TAuoZKEjHtZdObc3xqlACHsTIlAU/V3xMZibQeR77pjAgM9aI3Ef/zuimEF17GZZICk3S2KEwFhhhPAsABV4yCGBtCqOLmVkyHRBEKJqaSCcFdfHmZtKoVt1ap3pg0HDRDER2hY3SGXHSO6ugaNVATUZSgR/SMXqzUerJerbdZa8GazxyiP7DefwCxtpUP</latexit>

p3<latexit sha1_base64="mmn5wt5Q9+RM0Sjt/ah7e/FQf5s=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0laQY8VLx4r2lpoQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCoreNUMWyxWMSqE1CNgktsGW4EdhKFNAoEPgbjm5n/+IRK81g+mEmCfkSHkoecUWOl+6Rf75crbtWdg6wSLycVyNHsl796g5ilEUrDBNW667mJ8TOqDGcCp6VeqjGhbEyH2LVU0gi1n81PnZIzqwxIGCtb0pC5+nsio5HWkyiwnRE1I73szcT/vG5qwis/4zJJDUq2WBSmgpiYzP4mA66QGTGxhDLF7a2EjaiizNh0SjYEb/nlVdKuVb16tXZ3UWlc53EU4QRO4Rw8uIQG3EITWsBgCM/wCm+OcF6cd+dj0Vpw8plj+APn8wcDiY2d</latexit>

e�<latexit sha1_base64="gCdXGSES8icp12Aphsp5s09cQpM=">AAACBXicbVDLSsNAFJ3UV62vqEtdBIvgqiRVsMuCG5cV7APaEiaT23boJBNmboQSsnHjr7hxoYhb/8Gdf+P0sdDWA8MczrmXe+8JEsE1uu63VVhb39jcKm6Xdnb39g/sw6OWlqli0GRSSNUJqAbBY2giRwGdRAGNAgHtYHwz9dsPoDSX8T1OEuhHdBjzAWcUjeTbp70RxawXSBHqSWS+DPLcNwIgzX277FbcGZxV4i1ImSzQ8O2vXihZGkGMTFCtu56bYD+jCjkTkJd6qYaEsjEdQtfQmEag+9nsitw5N0roDKQyL0Znpv7uyGikpzuayojiSC97U/E/r5vioNbPeJykCDGbDxqkwkHpTCNxQq6AoZgYQpniZleHjaiiDE1wJROCt3zyKmlVK95lpXp3Va7XFnEUyQk5IxfEI9ekTm5JgzQJI4/kmbySN+vJerHerY95acFa9ByTP7A+fwAVGJmN</latexit>

frep<latexit sha1_base64="fnyG6bfGksDBQcm9d9vhQs/XvL8=">AAAB7nicbVDLSgNBEOyNrxhfUY9eBoPgKexGQY8BLx4jmAckIcxOepMhs7PDzKwQlnyEFw+KePV7vPk3TpI9aGJBQ1HVTXdXqAQ31ve/vcLG5tb2TnG3tLd/cHhUPj5pmSTVDJssEYnuhNSg4BKblluBHaWRxqHAdji5m/vtJ9SGJ/LRThX2YzqSPOKMWie1o0GmUc0G5Ypf9Rcg6yTISQVyNAblr94wYWmM0jJBjekGvrL9jGrLmcBZqZcaVJRN6Ai7jkoao+lni3Nn5MIpQxIl2pW0ZKH+nshobMw0Dl1nTO3YrHpz8T+vm9rotp9xqVKLki0XRakgNiHz38mQa2RWTB2hTHN3K2FjqimzLqGSCyFYfXmdtGrV4Kpae7iu1P08jiKcwTlcQgA3UId7aEATGEzgGV7hzVPei/fufSxbC14+cwp/4H3+AKTJj7Y=</latexit>

frep<latexit sha1_base64="fnyG6bfGksDBQcm9d9vhQs/XvL8=">AAAB7nicbVDLSgNBEOyNrxhfUY9eBoPgKexGQY8BLx4jmAckIcxOepMhs7PDzKwQlnyEFw+KePV7vPk3TpI9aGJBQ1HVTXdXqAQ31ve/vcLG5tb2TnG3tLd/cHhUPj5pmSTVDJssEYnuhNSg4BKblluBHaWRxqHAdji5m/vtJ9SGJ/LRThX2YzqSPOKMWie1o0GmUc0G5Ypf9Rcg6yTISQVyNAblr94wYWmM0jJBjekGvrL9jGrLmcBZqZcaVJRN6Ai7jkoao+lni3Nn5MIpQxIl2pW0ZKH+nshobMw0Dl1nTO3YrHpz8T+vm9rotp9xqVKLki0XRakgNiHz38mQa2RWTB2hTHN3K2FjqimzLqGSCyFYfXmdtGrV4Kpae7iu1P08jiKcwTlcQgA3UId7aEATGEzgGV7hzVPei/fufSxbC14+cwp/4H3+AKTJj7Y=</latexit>

fatt<latexit sha1_base64="wIXDzMFbYJKsZRpPXJZ2FMASVeo=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeCF48V7Ae0oWy2m3bpZhN2J0IJ+RFePCji1d/jzX/jts1BWx8MPN6bYWZekEhh0HW/ndLG5tb2Tnm3srd/cHhUPT7pmDjVjLdZLGPdC6jhUijeRoGS9xLNaRRI3g2md3O/+8S1EbF6xFnC/YiOlQgFo2ilbjjMKGI+rNbcursAWSdeQWpQoDWsfg1GMUsjrpBJakzfcxP0M6pRMMnzyiA1PKFsSse8b6miETd+tjg3JxdWGZEw1rYUkoX6eyKjkTGzKLCdEcWJWfXm4n9eP8Xw1s+ESlLkii0XhakkGJP572QkNGcoZ5ZQpoW9lbAJ1ZShTahiQ/BWX14nnUbdu6o3Hq5rTbeIowxncA6X4MENNOEeWtAGBlN4hld4cxLnxXl3PpatJaeYOYU/cD5/AKfAj7g=</latexit>

fatt<latexit sha1_base64="wIXDzMFbYJKsZRpPXJZ2FMASVeo=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mqoMeCF48V7Ae0oWy2m3bpZhN2J0IJ+RFePCji1d/jzX/jts1BWx8MPN6bYWZekEhh0HW/ndLG5tb2Tnm3srd/cHhUPT7pmDjVjLdZLGPdC6jhUijeRoGS9xLNaRRI3g2md3O/+8S1EbF6xFnC/YiOlQgFo2ilbjjMKGI+rNbcursAWSdeQWpQoDWsfg1GMUsjrpBJakzfcxP0M6pRMMnzyiA1PKFsSse8b6miETd+tjg3JxdWGZEw1rYUkoX6eyKjkTGzKLCdEcWJWfXm4n9eP8Xw1s+ESlLkii0XhakkGJP572QkNGcoZ5ZQpoW9lbAJ1ZShTahiQ/BWX14nnUbdu6o3Hq5rTbeIowxncA6X4MENNOEeWtAGBlN4hld4cxLnxXl3PpatJaeYOYU/cD5/AKfAj7g=</latexit>

Ds<latexit sha1_base64="kd2F4Qg7/dtYFWs4c99n+gU4M/w=">AAAB6nicbVBNS8NAEJ34WetX1aOXxSJ4Kkkt6LGgB48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2Z++4lrI2L1iJOE+xEdKhEKRtFKD7d90y+V3Yo7B1klXk7KkKPRL331BjFLI66QSWpM13MT9DOqUTDJp8VeanhC2ZgOeddSRSNu/Gx+6pScW2VAwljbUkjm6u+JjEbGTKLAdkYUR2bZm4n/ed0Uw2s/EypJkSu2WBSmkmBMZn+TgdCcoZxYQpkW9lbCRlRThjadog3BW355lbSqFe+yUr2vleu1PI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEdmI2k</latexit>

e✓(t)<latexit sha1_base64="Zot5QIGzIEVMv3w9AldKasf5BNQ=">AAACCXicbVA9SwNBEN2LXzF+RS1tFoMQm3AXBVMGbCwjmETIHWFvM0mW7H2wOyeE41ob/4qNhSK2/gM7/42b5ApNfLDs470ZZub5sRQabfvbKqytb2xuFbdLO7t7+wflw6OOjhLFoc0jGal7n2mQIoQ2CpRwHytggS+h60+uZ373AZQWUXiH0xi8gI1CMRScoZH6ZeqOGaauH8mBngbmSyHL+qmLY0BWxfOsX67YNXsOukqcnFRIjla//OUOIp4EECKXTOueY8fopUyh4BKykptoiBmfsBH0DA1ZANpL55dk9MwoAzqMlHkh0rn6uyNlgZ7taSoDhmO97M3E/7xegsOGl4owThBCvhg0TCTFiM5ioQOhgKOcGsK4EmZXysdMMY4mvJIJwVk+eZV06jXnola/vaw0G3kcRXJCTkmVOOSKNMkNaZE24eSRPJNX8mY9WS/Wu/WxKC1Yec8x+QPr8we+pZr0</latexit>

✓(t)<latexit sha1_base64="q/j84qCeYurHXHKzyv1mbANBrUE=">AAAB8HicbVDLSgNBEJyNrxhfUY9eFoMQL2E3CuYY8OIxgnlIsoTZSW8yZGZ2mekVQshXePGgiFc/x5t/4yTZgyYWNBRV3XR3hYngBj3v28ltbG5t7+R3C3v7B4dHxeOTlolTzaDJYhHrTkgNCK6giRwFdBINVIYC2uH4du63n0AbHqsHnCQQSDpUPOKMopUeezgCpGW87BdLXsVbwF0nfkZKJEOjX/zqDWKWSlDIBDWm63sJBlOqkTMBs0IvNZBQNqZD6FqqqAQTTBcHz9wLqwzcKNa2FLoL9ffElEpjJjK0nZLiyKx6c/E/r5tiVAumXCUpgmLLRVEqXIzd+ffugGtgKCaWUKa5vdVlI6opQ5tRwYbgr768TlrVin9Vqd5fl+q1LI48OSPnpEx8ckPq5I40SJMwIskzeSVvjnZenHfnY9mac7KZU/IHzucPQ9+QBQ==</latexit>

a2 = ��p

12

<latexit sha1_base64="vWTU2fiPpwwWz9Qi+4Fau3xhTnI=">AAACF3icbVDLSsNAFJ3UV62vqEs3g0VwY0iiYDdCQRcuK9gHNCFMJpN26OTBzEQoIX/hxl9x40IRt7rzb5y0WWjrgWEO59zLvff4KaNCmua3VltZXVvfqG82trZ3dvf0/YOeSDKOSRcnLOEDHwnCaEy6kkpGBiknKPIZ6fuT69LvPxAuaBLfy2lK3AiNYhpSjKSSPN1w/IQFYhqpL0eFZ8MreObcECYR/G2lhZdbduHpTdMwZ4DLxKpIE1ToePqXEyQ4i0gsMUNCDC0zlW6OuKSYkaLhZIKkCE/QiAwVjVFEhJvP7irgiVICGCZcvVjCmfq7I0eRKPdTlRGSY7HoleJ/3jCTYcvNaZxmksR4PijMGJQJLEOCAeUESzZVBGFO1a4QjxFHWKooGyoEa/HkZdKzDevcsO8umu1WFUcdHIFjcAoscAna4BZ0QBdg8AiewSt40560F+1d+5iX1rSq5xD8gfb5A9qon7M=</latexit>

(b) Three Robot Equilibrium

Fig. 3: Force Equilibrium in Deadlock

6 Three Robot DeadlockFollowing the ideas developed for two robot deadlock, we now describe the three robotcase. We will demonstrate that properties such as robots being on the verge of safetyviolation (theorem 4) and non-emptiness (theorem 5) are retained in this case as well.We are interested in analyzing system deadlock, which occurs when u∗i =0, vi=0 andui 6=0 ∀i∈{1,2,3}. Since we are studying system deadlock, each robot will have at-leastone active collision avoidance constraint (each robot has two constraints in total). Notethat the system deadlock set Dsystem for three robots is defined analogously to (31).

Theorem 4 (Safety Margin Apart). In system deadlock, either all three robotsare separated by the safety margin or exactly two pairs of robots are separated by thesafety margin.

Proof. The proof is kept brief because it is similar to the proof of theorem 1. Basedon the number of constraints that are allowed to be active per robot, all geometricconfigurations can be clubbed in two categories :

Category A - This arises when all collision avoidance constraints of each robot are

active i.e. aTiju∗i = bij=0⇐⇒

∥∥∆pij∥∥=Ds ∀j∈{1,2,3}\i ∀i∈{1,2,3}. As a result,each robot is separated by Ds from every other robot (Fig. 4(a)).

Category B - This arises when there is exactly one robot with both its constraintsactive (robot i in Fig. 4(b)), and the remaining two robots (j and k) have exactly oneconstraint active each. Hence, robot i is separated by Ds from the other two. Afterrelabeling of indices, category B results in three rearrangements.

Theorem 5 (Non-emptiness). ∀ kp,kv,Ds,R > 0 and pdi =Re2π(i−1)/3 wherei={1,2,3}, ∃ (z∗1,z

∗2,z∗3)∈Dsystem where z∗i =(p∗i ,0) and p

∗i is proposed as follows:

Configuration A: p∗i =Ds√3e2π(i−1)

3 +πwhere i={1,2,3}

Configuration B: p∗1=Dseπ, p∗2=0, p∗3=Dseπ3 if robot 2 has both constraints active.

Proof. This proof is similar to the proof of theorem 2 so it is skipped. Some remarks:1. In the statement of this theorem, we have predefined the desired goal positions unlike

the statement of theorem 2. The candidate positions of the robots that we propose arein Dsystem are valid with respect to these given goals. We have derived a similar non-emptiness result for arbitrary goals but are not including it here for the sake of brevity.

Page 12: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

12 Jaskaran Singh Grover, Changliu Liu, and Katia Sycara

(a) Category A (b) Category B

Fig. 4: Categories of geometric configurations in system deadlock of three robots

2. For configuration B, we proposed one set of positions that is valid in deadlock,however there is continuous family of positions that can be valid in configurationB. The representation of this family can be found in the supplementary material.

7 Deadlock Resolution

We now use the properties of geometric configurations derived in section 5 and section 6to synthesize a strategy that (1) gets the robots out of deadlock, (2) ensures their safetyand (3) makes them converge to their goals. One approach to achieve these objectives isto detect the incidence of deadlock while the CBF-QP controller is running on the robotsand once detected, any small non-zero perturbation to the control will instantaneouslygive a non-zero velocity to the robots. Thereafter, CBF-QPs can take charge again andwe can hope that using this controller the system state will come out of deadlock at-leastfor a short time. This has two limitations however; firstly, since it was the CBF-QPcontroller that led to deadlock, there is no guarantee that the system will not fall backin deadlock again. Secondly, perturbations can violate safety and even lead to degradedperformance. Therefore, we propose a controller which ensures that goal stabilization,safety and deadlock resolution are met with guarantees. We demonstrate this algorithmfor the two and three robot cases. Extension to N≥4 is left for future work since N≥4admits a large number of geometric configurations that are valid in deadlock. Referto Fig. 5 for a schematic of our approach. This algorithm is described here:

1. The algorithm starts by executing controls derived from CBF-QP in Phase 1. Thisensures movement of robots to the goals and safety by construction. To detect the inci-dence of deadlock, we continuously compare ‖u∗‖,‖v‖,‖p−pd‖ against small thresh-olds. If satisfied, we switch control to phase 2, otherwise, phase 1 continues to operate.

2. In this phase, we rotate the robots around each other to swap positions whilemaintaining the safe distance.(a) For the two-robot case, we calculate u1

fl(t) and u2fl(t) using feedback lineariza-

tion (see supplementary material) to ensure that ‖∆p12‖=Ds and rotation

(θ=−kp(θ−β)−kvθ=⇒∆pT12∆v12=0) (See Fig. 3(b) for θ,β). This rotationand distance invariance guarantees safety i.e. h12=0. Adding an extra constraintu1fl+u

2fl=0 still ensures that the problem is well posed and additionally makes

the centroid static.(b) For the three-robot case, we calculate u1

fl(t),u3fl(t),u

3fl(t) to ensure that

‖∆p12‖= ‖∆p23‖= ‖∆p31‖=Ds and rotation (θ =−kp(θ−β)−kvθ =⇒

Page 13: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

Deadlock Analysis for Multirobot Systems 13

Phase 1: CBF-QP Phase 2: FL Phase 3: PD Control

minimizeu

||u � u||22subject to Au b

<latexit sha1_base64="u3+uX8uDpnoSFbHaKQ3K5hDEOlg=">AAADBHicbVLLjtMwFHXCa1peHdjBxqICsaFqwoJZDmLDskh0ZqS6VI5z25qxnYwfI4qVBRt+hQ0LEGLLR7Djb3CTMI+WKyW5OefeY99jZ6Xgxg6Hf6L4ytVr12/sdLo3b92+c7e3e+/AFE4zGLNCFPooowYEVzC23Ao4KjVQmQk4zI5frfnDU9CGF+qtXZUwlXSh+JwzagM0240ekAwWXHk4cTVUdYmgGQg/SqpupyWp4AsFeQA6TzBxKg+KYD3JCpGblQwf76rKEwsfrJdccck/QtWUh4awAdu8LjXgZ5gs6ZbMefm7dJZiQppVa23jsvfALLbFmbjlIgf/strQJgJO/pEXqawKnCqUkxnoMCCo/Hy8+u/MiVmvPxwM68DbSdImfdTGaNb7TfKCOQnKMkGNmSTD0k491ZYzAUHeGSgpO6YLmIRUUQlm6utDrPDjgOR4XujwKItr9GKHp9KsZwiVktql2eTW4P+4ibPzvannqnQWFGsWmjsRPMTrG4FzroOlYhUSyjQPe8VsSTVlNpxyN5iQbI68nRykg+T5IH2T9vfT1o4d9BA9Qk9Rgl6gffQajdAYsehT9CX6Fn2PP8df4x/xz6Y0jtqe++hSxL/+At3q93Y=</latexit>

ui = �kp(pi � pdi) � kvvi

<latexit sha1_base64="knLMMqFMabNhlQgM1FB4aBH1Iuw=">AAACQXicbVA7SwNBGNzzGePrjFjZLAYhFgl3EdFGCNhYRjAPSI5jb28vWbL3YHcvEI5rrfwBtv4RG/+Bnb2NhSK2Nu4lKWLiwLLDzHzst+NEjAppGK/a0vLK6tp6biO/ubW9s6vvFZoijDkmDRyykLcdJAijAWlIKhlpR5wg32Gk5QyuMr81JFzQMLiVo4hYPuoF1KMYSSXZervbRzLpOiFzxchXVxKnqU3hJSwP7Kg0a0RKL88JiWvT9ERFh7PGUCVtvWhUjDHgIjGnpFgrPN49nB3c1239peuGOPZJIDFDQnRMI5JWgrikmJE0340FiRAeoB7pKBognwgrGTeQwmOluNALuTqBhGN1diJBvsiWU0kfyb6Y9zLxP68TS+/CSmgQxZIEePKQFzMoQ5jVCV3KCZZspAjCnKpdIe4jjrBUpedVCeb8lxdJs1oxTyvVG9WGASbIgUNwBErABOegBq5BHTQABk/gDXyAT+1Ze9e+tO9JdEmbzuyDP9B+fgGAUrYi</latexit>

Deadlock Detected

Feasibility Detected

d||�p12||dt

! 0 i.e. ||�p12|| ! Ds

✓ �! �<latexit sha1_base64="pseqz3lKCOgidUJpOwIdPjb0Jko=">AAACqHiclVFNbxMxEPUuFNpQIMCxF4uoCHFY7QYkOFaiB6RegtQ0RXG08npnE6tee2XPQiNrfxv/gRv/ps6HRGh76Ui2n968Gc9H0SjpME3/RvGjx3tPnu4f9J4dPn/xsv/q9YUzrRUwFkYZe1lwB0pqGKNEBZeNBV4XCibF1deVf/ITrJNGn+OygVnN51pWUnAMVN7/zSrLhS8pCyrcXKegkLPCqNIt6/D4pst9Nuz+STpfYkeZlfMFcmvNL5rSdyxorz2lDOE6VOJlAklHafew1LtJT3NHGQuZcQEhTBk93/GyIpB5f5Am6droXZBtwYBsbZT3/7DSiLYGjUJx56ZZ2uDMc4tSKOh6rHXQcHHF5zANUPMa3MyvB93R48CUtDI2HI10ze5GeF67VWNBWXNcuNu+FXmfb9pi9WXmpW5aBC02H1Wtomjoamu0lBYEqmUAXFgZaqViwcPmMOy2F4aQ3W75LrgYJtnHZPj90+Ak3Y5jnxyRt+Q9ychnckK+kREZExEdR2fReTSOP8SjeBL/2EjjaBvzhvxncXEDcPzVOw==</latexit>

i = {1, 2}<latexit sha1_base64="OKQLU5+PWrcldBIT7DtVhOmXJOw=">AAAB8HicbVC7SgNBFL0bXzG+Vu20GQyChYTdWGgjBGwsLBIwD8kuYXYymwyZnV1mZoWw5AssbSwUsfU3/AM7/8JPcPIoNPHAhcM593LvPUHCmdKO82XllpZXVtfy64WNza3tHXt3r6HiVBJaJzGPZSvAinImaF0zzWkrkRRHAafNYHA19pv3VCoWi1s9TKgf4Z5gISNYG+mOXXqZe1r2Rh276JScCdAicWekWLFr3x83Bw/Vjv3pdWOSRlRowrFSbddJtJ9hqRnhdFTwUkUTTAa4R9uGChxR5WeTg0fo2ChdFMbSlNBoov6eyHCk1DAKTGeEdV/Ne2PxP6+d6vDCz5hIUk0FmS4KU450jMbfoy6TlGg+NAQTycytiPSxxESbjAomBHf+5UXSKJfcs1K5ZtJwYIo8HMIRnIAL51CBa6hCHQhE8AjP8GJJ68l6td6mrTlrNrMPf2C9/wAm4pLs</latexit>

(a) Deadlock resolution for two robots

Phase 1: CBF-QP Phase 2: FL Phase 3: PD Control

minimizeu

||u � u||22subject to Au b

<latexit sha1_base64="u3+uX8uDpnoSFbHaKQ3K5hDEOlg=">AAADBHicbVLLjtMwFHXCa1peHdjBxqICsaFqwoJZDmLDskh0ZqS6VI5z25qxnYwfI4qVBRt+hQ0LEGLLR7Djb3CTMI+WKyW5OefeY99jZ6Xgxg6Hf6L4ytVr12/sdLo3b92+c7e3e+/AFE4zGLNCFPooowYEVzC23Ao4KjVQmQk4zI5frfnDU9CGF+qtXZUwlXSh+JwzagM0240ekAwWXHk4cTVUdYmgGQg/SqpupyWp4AsFeQA6TzBxKg+KYD3JCpGblQwf76rKEwsfrJdccck/QtWUh4awAdu8LjXgZ5gs6ZbMefm7dJZiQppVa23jsvfALLbFmbjlIgf/strQJgJO/pEXqawKnCqUkxnoMCCo/Hy8+u/MiVmvPxwM68DbSdImfdTGaNb7TfKCOQnKMkGNmSTD0k491ZYzAUHeGSgpO6YLmIRUUQlm6utDrPDjgOR4XujwKItr9GKHp9KsZwiVktql2eTW4P+4ibPzvannqnQWFGsWmjsRPMTrG4FzroOlYhUSyjQPe8VsSTVlNpxyN5iQbI68nRykg+T5IH2T9vfT1o4d9BA9Qk9Rgl6gffQajdAYsehT9CX6Fn2PP8df4x/xz6Y0jtqe++hSxL/+At3q93Y=</latexit>

ui = �kp(pi � pdi) � kvvi

<latexit sha1_base64="knLMMqFMabNhlQgM1FB4aBH1Iuw=">AAACQXicbVA7SwNBGNzzGePrjFjZLAYhFgl3EdFGCNhYRjAPSI5jb28vWbL3YHcvEI5rrfwBtv4RG/+Bnb2NhSK2Nu4lKWLiwLLDzHzst+NEjAppGK/a0vLK6tp6biO/ubW9s6vvFZoijDkmDRyykLcdJAijAWlIKhlpR5wg32Gk5QyuMr81JFzQMLiVo4hYPuoF1KMYSSXZervbRzLpOiFzxchXVxKnqU3hJSwP7Kg0a0RKL88JiWvT9ERFh7PGUCVtvWhUjDHgIjGnpFgrPN49nB3c1239peuGOPZJIDFDQnRMI5JWgrikmJE0340FiRAeoB7pKBognwgrGTeQwmOluNALuTqBhGN1diJBvsiWU0kfyb6Y9zLxP68TS+/CSmgQxZIEePKQFzMoQ5jVCV3KCZZspAjCnKpdIe4jjrBUpedVCeb8lxdJs1oxTyvVG9WGASbIgUNwBErABOegBq5BHTQABk/gDXyAT+1Ze9e+tO9JdEmbzuyDP9B+fgGAUrYi</latexit>

Deadlock Detected

Feasibility Detected

d||�pij ||dt

! 0 i.e. ||�pij || ! Ds

8i, j 2 {1, 2, 3}, i 6= j and ✓ �! �<latexit sha1_base64="IZFwSnjMsQ0wZtwUfn3hyYu1m74=">AAAC3HiclVJLbxMxEPYurxJeAY5cRkRUHKJoN0WCYyV64Fgk0hbFYeX1ehOnXnuxZ6GRteLCAYS48sO48Tv4AzgPRGi5MJLtT998M2PPOK+VdJgkP6L40uUrV6/tXO/cuHnr9p3u3XtHzjSWixE3ytiTnDmhpBYjlKjESW0Fq3IljvPT50v/8TthnTT6FS5qManYVMtScoaByro/d2lpGfcF0CDD9XYgFDKaG1W4RRUOX7eZl/P2j6T1BbZArZzOkFlr3kMCNEjPPABFcRZu4uVADFqA9v8yb+c8yBxQCrtAS2OZUl725yGd1EB92h/292jbBwlUi7cw/10emC5CVYozEUopo6dbGWkeyKzbSwbJyuAiSDegRzZ2mHW/08LwphIauWLOjdOkxolnFiVXou3Qxoma8VM2FeMANauEm/jVcFp4FJgCwgPC0ggrdjvCs8otmxGUFcOZO+9bkv/yjRssn0281HWDQvN1obJRgAaWk4ZCWsFRLQJg3MpwV+AzFoaN4T90QhPS80++CI6Gg3RvMHz5pLefbNqxQx6Qh+QxSclTsk9ekEMyIjx6HX2IPkWf4zfxx/hL/HUtjaNNzH3yl8XffgEITukD</latexit>

i = {1, 2, 3}<latexit sha1_base64="0y0aS1grckDSCESY/BdbfvagqHA=">AAAB8nicbVC7SgNBFL3rM8bXqp02i0GwCGE3KbQRAjYWFgmYB2SXMDuZTYbMziwzs0JY8gXWNhaK2PoX/oGdf+EnOHkUmnjgwuGce7n3njBhVGnX/bJWVtfWNzZzW/ntnd29ffvgsKlEKjFpYMGEbIdIEUY5aWiqGWknkqA4ZKQVDq8nfuueSEUFv9OjhAQx6nMaUYy0kTr0ys+8YrlY8cddu+CW3CmcZeLNSaFq178/bo8fal370+8JnMaEa8yQUh3PTXSQIakpZmSc91NFEoSHqE86hnIUExVk05PHzplRek4kpCmunan6eyJDsVKjODSdMdIDtehNxP+8TqqjyyCjPEk14Xi2KEqZo4Uz+d/pUUmwZiNDEJbU3OrgAZIIa5NS3oTgLb68TJrlklcplesmDRdmyMEJnMI5eHABVbiBGjQAg4BHeIYXS1tP1qv1NmtdseYzR/AH1vsPBdmTXw==</latexit>

(b) Deadlock resolution for three robots

Fig. 5: Deadlock Resolution Algorithm Schematic

∆pT12∆v12=0) (See Fig. 3(b) for θ,β). This guarantees safety i.e. h12=h23=h31=0. Similarly, we impose u1

fl+u2fl+u

3fl=0 to make the centroid static.

3. Once the robots swap their positions, their new positions will ensure that prescribedPD controllers will be feasible in the future. Thus, after convergence of Phase 2(which happens in finite time), control switches to Phase 3, which simply uses theprescribed PD controllers. This phase guarantees that the distance between robotsis non-decreasing and safety is maintained as we prove in theorem 6.

Fig. 6 shows simulation and experimental results from running this strategy on two(6(a)) and three (Fig. 6(c)) robots. Experiments were conducted using Khepera 4nonholonomic robots (6(b)). Note that for nonholnomic robots, we noticed from exper-iments and simulations that deadlock only occurs if the body frames of both robots areperfectly aligned with one another at t=0. Since this alignment is difficult to establishin experiments, we simulated a virtual deadlock at t=0 i.e. assumed that the initialposition of robots are ones that are in deadlock.We next prove that this strategy ensures resolution of deadlock and convergence

of robots to their goals. The proof of this theorem will exploit the geometric propertiesof deadlock we derived in theorem 1 and theorem 2. We prove this theorem for N=2since the proof for N=3 is a trivial extension of N=2.

Theorem 6. Assuming that PD controllers are overdamped and DG>Ds, this strategyensures that (1) the robots will never fall in deadlock and (2) converge to their goals.

Proof. We would like to show that once phase three control begins, the robots willnever fall back in deadlock. We will do this by showing that the distance between therobots is non-decreasing, once phase three control starts. We break this proof into threeparts consistent with the three phases:

Phase 1 → Phase 2: Let t= t1 be the time at which phase 1 ends (and phase 2starts) i.e. when robots fall in deadlock. In theorem 1 we showed that in deadlock‖∆p21‖=Ds, and in theorem 2 we showed that the positions of robots and theirgoals are collinear. So at the end of phase 1, ∆p21(t1) =Dseβ+π. The goal vector

Page 14: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

14 Jaskaran Singh Grover, Changliu Liu, and Katia Sycara

0 5 10 15 20 25 30-4

-3

-2

-1

0

1

2

3

4

5

(a) Two Robots Sims.

0 20 40 60 80-3

-2

-1

0

1

2

3

(b) Expmts. with Kheperas.

0 5 10 15 20-4

-3

-2

-1

0

1

2

3

4

(c) Three Robots Sims.

Fig. 6: Positions of robots from deadlock resolution algorithm. In all figures, final positionsconverge to desired positions in Phase 3. Videos at https://tinyurl.com/y4ylzwh8

∆pd21(t):=pd2−pd1 =DGeβ ∀t>0. Moreover, since the robots are static in deadlock,∆v21(t1)=0

Phase 2 −→ Phase 3: The initial condition of phase two is the final conditionof phase one i.e. ∆p21(t1)=Dseβ+π and ∆v21(t1)=0. In phase two, we use feedbacklinearization to rotate the assembly of robots making sure that the distance betweenthem stays at Ds, until the orientation of the vector ∆p21(t)=Dseθ(t) aligns with∆pd21 =DGeβ (see supplementary for controller derivation). Once done, ∃ a time t2at which θ(t2)=β. Moreover, at t=t2, the robots are no longer moving, hence theirvelocities are zero, hence, ∆p21(t2)=Dseβ, ∆v21(t2)=0. These states are the finalcondition for phase 2 and initial for phase 3.

Phase 3 −→∞ : In this phase, the initial conditions are ∆p21(t2) = Dseβ and∆v21(t2)=0. Also, note that the dynamics of phase 3 control are specified by theprescribed PD controllers. The dynamics of relative positions and velocities are:

∆p21=∆v21

∆v21=−kp(∆p21−∆pd21)−kv∆v21, (39)

where ∆pd21 =DGeβ. Now, we will do a coordinate change as described next. Let∆p21 :=R−β∆p21 and ∆v21 :=R−β∆v21. The initial conditions in these coordinatesare ∆p21(t2)=R−βDseβ =(Ds,0) and ∆v21(t2)=0 i.e. ∆px21(t2)=Ds,∆p

y21(t2)=

0,∆vx21(t2)=0 and ∆vy21(t2)=0. The dynamics in new coordinates are:

∆ ˙p21=∆v21

∆ ˙v21=−kp(∆p21−R−β∆pd21)−kv∆v21. (40)

Using these coordinates, note that R−β∆pd21 =(DG,0). Note from the dynamics andthe initial conditions for the y components of relative position and velocities that theonly solution is the zero solution i.e. ∆py21(t)≡0 and ∆vy21(t)≡0 ∀ t≥t2. As for the xcomponent, we can compute the solution to be ∆px21(t)=c1e

ω1(t−t2)+c2eω2(t−t2)+DG

and ∆vx21(t)=c1ω1eω1(t−t2)+c2ω2e

ω2(t−t2). Here

ω1,2=1

2

(−kv±

√k2v−4kp

), c1=

ω2(DG−Ds)ω1−ω2

, c2=−ω1(DG−Ds)ω1−ω2

,

and ω1 − ω2 = −√k2v−4kp and ω1ω2 = kp. After substituting these values, we

get, ∆vx21(t) =kp(Ds−DG)(eω1(t−t2)−eω2(t−t2))√

k2v−4kp. Now, from the assumptions that PD

Page 15: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

Deadlock Analysis for Multirobot Systems 15

controllers are overdamped i.e kv, k2v − 4kp > 0 and DG > Ds, it follows that

∆vx21(t)=kp(Ds−DG)(eω1(t−t2)−eω2(t−t2))√

k2v−4kp≥ 0 and ∆px21(t)=

DG−Ds√(k2v−4kp)

(ω1e

ω2(t−t2)−

ω2eω1(t−t2)

)+DG≥Ds≥0. Finally, note that d‖∆p12(t)‖

dt =∆pT21(t)∆v21(t)‖∆p21(t)‖

≥0. Hence,the distance between the robots is non-decreasing i.e. the robots never fall in deadlock.Additionally, since the robots use a PD-type controller, their positions exponentiallystabilize to their goals.

8 Conclusions

In this paper, we analyzed the characteristic properties of deadlock that results fromusing CBF based QPs for avoidance control in multirobot systems. We demonstratedhow to interpret deadlock as a subset of the state space and proved that in deadlock,the robots are on the verge of violating safety. Additionally, we showed that this set isnon-empty and bounded. Using these properties, we devised corrective control algorithmto force the robots out of deadlock and ensure task completion. We also demonstratedthat the number of valid geometric configurations in deadlock increases approximatelyexponentially with the number of robots which makes the analysis and resolution forN≥4 complex. There are several directions we would like to explore in future. Firstly,we want to extend this to N≥4 case. In the N=4 case, we determined a large numberof admissible geometric configurations. We find that there exist bijections among someof these configurations depending on the number of total active constraints. We believethis property can be exploited to reduce the complexity down to the equivalenceclasses of these bijections. We will exploit this line of approach to simplify analysisfor N≥4 cases. Secondly, we are interested in identifying the basin of attraction of thedeadlock set to formally characterize all initial conditions of robots that lead to deadlock.Tools from backwards reachability set calculation can be used to compute the basin ofattraction. Finally, although we focused on CBF based QPs for analysis, we will extendthis to other reactive methods such as velocity obstacles and tools using value functions,and explore the properties that make a particular algorithm immune to deadlock.

References

1. G. Kantor, S. Singh, R. Peterson, D. Rus, A. Das, V. Kumar, G. Pereira, and J. Spletzer,“Distributed search and rescue with robot and sensor teams,” in Field and Service Robotics.Springer, 2003, pp. 529–538.

2. J. Cortes, S. Martinez, T. Karatas, and F. Bullo, “Coverage control for mobile sensingnetworks,” IEEE Transactions on robotics and Automation, vol. 20, no. 2, pp. 243–255, 2004.

3. W. Burgard, M. Moors, C. Stachniss, and F. E. Schneider, “Coordinated multi-robotexploration,” IEEE Transactions on robotics, vol. 21, no. 3, pp. 376–386, 2005.

4. P. Ogren, M. Egerstedt, and X. Hu, “A control lyapunov function approach to multiagentcoordination,” IEEE Transactions on Robotics and Automation, vol. 18, no. 5, pp.847–851, 2002.

5. R. Olfati-Saber, J. A. Fax, and R. M. Murray, “Consensus and cooperation in networkedmulti-agent systems,” Proceedings of the IEEE, vol. 95, no. 1, pp. 215–233, 2007.

6. U. Borrmann, L. Wang, A. D. Ames, and M. Egerstedt, “Control barrier certificatesfor safe swarm behavior,” IFAC-PapersOnLine, vol. 48, no. 27, pp. 68–73, 2015.

Page 16: Deadlock Analysis and Resolution for Multi-Robot Systemsrobotics.cs.rutgers.edu/wafr2020/wp-content/...Jaskaran Singh Grover, Changliu Liu, and Katia Sycara? The Robotics Institute,

16 Jaskaran Singh Grover, Changliu Liu, and Katia Sycara

7. A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada, “Control barrier function basedquadratic programs for safety critical systems,” IEEE Transactions on Automatic Control,vol. 62, no. 8, pp. 3861–3876, 2017.

8. S. Petti and T. Fraichard, “Safe motion planning in dynamic environments,” in 2005IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2005,pp. 2210–2215.

9. P. O’DONNELL, “Deadlockfree and collision-free coordination of two robot manipulators.”in Proc. IEEE International Conference on Robotics and Automation, 1989, pp. 484–489.

10. L. Wang, A. D. Ames, and M. Egerstedt, “Safety barrier certificates for collisions-freemultirobot systems,” IEEE Transactions on Robotics, vol. 33, no. 3, pp. 661–674, 2017.

11. G. Leitmann and J. Skowronski, “Avoidance control,” Journal of optimization theoryand applications, vol. 23, no. 4, pp. 581–591, 1977.

12. ——, “A note on avoidance control,” Optimal Control Applications and Methods, vol. 4,no. 4, pp. 335–342, 1983.

13. D. M. Stipanovic, P. F. Hokayem, M. W. Spong, and D. D. Siljak, “Cooperative avoidancecontrol for multiagent systems,” Journal of dynamic systems, measurement, and control,vol. 129, no. 5, pp. 699–707, 2007.

14. P. F. Hokayem, D. M. Stipanovic, and M. W. Spong, “Coordination and collision avoidancefor lagrangian systems with disturbances,” Applied Mathematics and Computation, vol.217, no. 3, pp. 1085–1094, 2010.

15. P. Fiorini and Z. Shiller, “Motion planning in dynamic environments using velocityobstacles,” The International Journal of Robotics Research, vol. 17, no. 7, pp. 760–772, 1998.

16. J. Van den Berg, M. Lin, and D. Manocha, “Reciprocal velocity obstacles for real-timemulti-agent navigation,” in 2008 IEEE International Conference on Robotics andAutomation. IEEE, 2008, pp. 1928–1935.

17. J. Van Den Berg, S. J. Guy, M. Lin, and D. Manocha, “Reciprocal n-body collisionavoidance,” in Robotics research. Springer, 2011, pp. 3–19.

18. D. Wilkie, J. Van Den Berg, and D. Manocha, “Generalized velocity obstacles,” in 2009IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2009,pp. 5573–5578.

19. H. Yamaguchi, “A cooperative hunting behavior by mobile-robot troops,” the InternationalJournal of robotics Research, vol. 18, no. 9, pp. 931–940, 1999.

20. M. Jager and B. Nebel, “Decentralized collision avoidance, deadlock detection, and deadlockresolution for multiple mobile robots,” in Proceedings 2001 IEEE/RSJ InternationalConference on Intelligent Robots and Systems. Expanding the Societal Role of Roboticsin the the Next Millennium (Cat. No. 01CH37180), vol. 3. IEEE, 2001, pp. 1213–1219.

21. Y. Li, K. Gupta, and S. Payandeh, “Motion planning of multiple agents in virtualenvironments using coordination graphs,” in Proceedings of the 2005 IEEE InternationalConference on Robotics and Automation. IEEE, 2005, pp. 378–383.

22. E. J. Rodrıguez-Seda, D. M. Stipanovic, and M. W. Spong, “Guaranteed collisionavoidance for autonomous systems with acceleration constraints and sensing uncertainties,”Journal of Optimization Theory and Applications, vol. 168, no. 3, pp. 1014–1038, 2016.

23. S. Boyd and L. Vandenberghe, Convex optimization. Cambridge university press, 2004.24. H. S. Wilf, generatingfunctionology. AK Peters/CRC Press, 2005.25. E. Weisstein. [Online]. Available: https://oeis.org/A30379226. N. Alon and A. Kupavskii, “Two notions of unit distance graphs,” Journal of

Combinatorial Theory, Series A, vol. 125, pp. 1–17, 2014.