Cooperative Control of Dynamical Systems Applications to Autonomous Vehicles

Cooperative Control of Dynamical Systems

Zhihua Qu

Cooperative Control

of Dynamical Systems

Applications to Autonomous Vehicles

123

Zhihua Qu, PhD

School of Electrical Engineering

and Computer Science

University of Central Florida

Orlando, FL 32816

USA

ISBN 978-1-84882-324-2 e-ISBN 978-1-84882-325-9

DOI 10.1007/978-1-84882-325-9

A catalogue record for this book is available from the British Library

Library of Congress Control Number: 2008940267

© 2009 Springer-Verlag London Limited

Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted

under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or

transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case

of reprographic reproduction in accordance with the terms of licences issued by the Copyright Licensing

Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers.

The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of a

specific statement, that such names are exempt from the relevant laws and regulations and therefore free for

general use.

The publisher makes no representation, express or implied, with regard to the accuracy of the information

contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that

may be made.

Cover design: eStudio Calamar S.L., Girona, Spain

Printed on acid-free paper

9 8 7 6 5 4 3 2 1

springer.com

To

Xinfan ZhangE. Vivian Qu and M. Willa Qu

Preface

Stability theory has allowed us to study both qualitative and quantitativeproperties of dynamical systems, and control theory has played a key role indesigning numerous systems. Contemporary sensing and communication net-works enable collection and subscription of geographically-distributed infor-mation and such information can be used to enhance significantly the perfor-mance of many of existing systems. Through a shared sensing/communicationnetwork, heterogeneous systems can now be controlled to operate robustly andautonomously; cooperative control is to make the systems act as one groupand exhibit certain cooperative behavior, and it must be pliable to physicaland environmental constraints as well as be robust to intermittency, latencyand changing patterns of the information flow in the network. This bookattempts to provide a detailed coverage on the tools of and the results onanalyzing and synthesizing cooperative systems. Dynamical systems underconsideration can be either continuous-time or discrete-time, either linear ornon-linear, and either unconstrained or constrained.

Technical contents of the book are divided into three parts. The first partconsists of Chapters 1, 2, and 4. Chapter 1 provides an overview of coopera-tive behaviors, kinematical and dynamical modeling approaches, and typicalvehicle models. Chapter 2 contains a review of standard analysis and designtools in both linear control theory and non-linear control theory. Chapter 4 isa focused treatment of non-negative matrices and their properties, multiplica-tive sequence convergence of non-negative and row-stochastic matrices, andthe presence of these matrices and sequences in linear cooperative systems.

The second part of the book deals with cooperative control designs thatsynthesize cooperative behaviors for dynamical systems. In Chapter 5, lineardynamical systems are considered, the matrix-theoretical approach developedin Chapter 4 is used to conclude cooperative stability in the presence of local,intermittent, and unpredictable changes in their sensing and communicationnetwork, and a class of linear cooperative controls is designed based onlyon relative measurements of neighbors’ outputs. In Chapter 6, cooperativestability of heterogeneous non-linear systems is considered, a comparative

viii Preface

and topology-based Lyapunov argument and the corresponding comparisontheorem on cooperative stability are introduced, and cooperative controls aredesigned for several classes of non-linear networked systems.

As the third part, the aforementioned results are applied to a team ofunmanned ground and aerial vehicles. It is revealed in Chapter 1 that thesevehicles belong to the class of so-called non-holonomic systems. Accordingly, inChapter 3, properties of non-holonomic systems are investigated, their canon-ical form is derived, and path planning and control designs for an individualnon-holonomic system are carried out. Application of cooperative control tovehicle systems can be found in Sections 5.3, 6.5 and 6.6.

During the last 18 years at University of Central Florida, I have developedand taught several new courses, including “EEL4664 Autonomous RoboticSystems,” “EEL6667 Planning and Control for Mobile Robotic Systems,” and“EEL6683 Cooperative Control of Networked and Autonomous Systems.” Inrecent years I also taught summer short courses and seminars on these topicsat several universities abroad. This book is the outgrowth of my course notes,and it incorporates many research results in the most recent literature. Whenteaching senior undergraduate students, I have chosen to cover mainly all thematrix results in Chapters 2 and 4, to focus upon linear cooperative systemsin Chapter 5, and to apply cooperative control to simple vehicle models (withthe aid of dynamic feedback linearization in Chapter 3). At the graduate level,many of our students have already taken our courses on linear system theoryand on non-linear systems, and hence they are able to go through most ofthe materials in the book. In analyzing and designing cooperative systems,autonomous vehicles are used as examples. Most students appear to find thisa happy pedagogical practice, since they become familiar with both theoryand application(s).

I wish to express my indebtedness to the following research collaboratorsfor their useful comments and suggestions: Kevin L. Conrad, Mark Falash,Richard A. Hull, Clinton E. Plaisted, Eytan Pollak, and Jing Wang. My thanksgo to former postdoctors and students in our Controls and Robotics Labora-tories; in particular, Jing Wang generated many of the simulation results forthe book and provided assistance in preparing a few sections of Chapters 1and 3, and Jian Yang coded the real-time path planning algorithm and gen-erated several figures in Chapter 3. I also wish to thank Thomas Ditzinger,Oliver Jackson, Sorina Moosdorf, and Aislinn Bunning at Springer and Cor-nelia Kresser at le-tex publishing services oHG for their assistance in gettingthe manuscript ready for publication.

My special appreciation goes to the following individuals who profession-ally inspired and supported me in different ways over the years: Tamer Basar,Theodore Djaferis, John F. Dorsey, Erol Gelenbe, Abraham H. Haddad, Mo-hamed Kamel, Edward W. Kamen, Miroslav Krstic, Hassan K. Khalil, PetarV. Kokotovic, Frank L. Lewis, Marwan A. Simaan, Mark W. Spong, and YoraiWardi.

Preface ix

Finally, I would like to acknowledge the following agencies and companieswhich provided me research grants over the last eight years: Army ResearchLaboratory, Department of Defense, Florida High Tech Council, Florida SpaceGrant Consortium, Florida Space Research Initiative, L-3 CommunicationsLink Simulation & Training, Lockheed Martin Corporation, Microtronic Inc.,NASA Kennedy Space Center, National Science Foundation (CISE, CMMI,and MRI), Oak Ridge National Laboratory, and Science Applications Inter-national Corporation (SAIC).

Zhihua QuOrlando, FloridaOctober 2008

Contents

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Cooperative, Pliable and Robust Systems . . . . . . . . . . . . . . . . . . 1

1.1.1 Control Through an Intermittent Network . . . . . . . . . . . 21.1.2 Cooperative Behaviors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.1.3 Pliable and Robust Systems . . . . . . . . . . . . . . . . . . . . . . . . 10

1.2 Modeling of Constrained Mechanical Systems . . . . . . . . . . . . . . . 111.2.1 Motion Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.2.2 Kinematic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.2.3 Dynamic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.2.4 Hamiltonian and Energy . . . . . . . . . . . . . . . . . . . . . . . . . . . 151.2.5 Reduced-order Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151.2.6 Underactuated Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

1.3 Vehicle Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181.3.1 Differential-drive Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . 181.3.2 A Car-like Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201.3.3 Tractor-trailer Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231.3.4 A Planar Space Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251.3.5 Newton’s Model of Rigid-body Motion . . . . . . . . . . . . . . 261.3.6 Underwater Vehicle and Surface Vessel . . . . . . . . . . . . . . . 291.3.7 Aerial Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311.3.8 Other Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

1.4 Control of Heterogeneous Vehicles . . . . . . . . . . . . . . . . . . . . . . . . 341.5 Notes and Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

2 Preliminaries on Systems Theory . . . . . . . . . . . . . . . . . . . . . . . . . 392.1 Matrix Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.2 Useful Theorems and Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.2.1 Contraction Mapping Theorem . . . . . . . . . . . . . . . . . . . . . 432.2.2 Barbalat Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

xii Contents

2.2.3 Comparison Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452.3 Lyapunov Stability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

2.3.1 Lyapunov Direct Method . . . . . . . . . . . . . . . . . . . . . . . . . . 512.3.2 Explanations and Enhancements . . . . . . . . . . . . . . . . . . . . 542.3.3 Control Lyapunov Function . . . . . . . . . . . . . . . . . . . . . . . . 582.3.4 Lyapunov Analysis of Switching Systems . . . . . . . . . . . . 59

2.4 Stability Analysis of Linear Systems . . . . . . . . . . . . . . . . . . . . . . . 622.4.1 Eigenvalue Analysis of Linear Time-invariant Systems . . 622.4.2 Stability of Linear Time-varying Systems . . . . . . . . . . . . 632.4.3 Lyapunov Analysis of Linear Systems . . . . . . . . . . . . . . . 66

2.5 Controllability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 692.6 Non-linear Design Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

2.6.1 Recursive Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732.6.2 Feedback Linearization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 752.6.3 Optimal Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 772.6.4 Inverse Optimality and Lyapunov Function . . . . . . . . . . 78

2.7 Notes and Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

3 Control of Non-holonomic Systems . . . . . . . . . . . . . . . . . . . . . . . . 813.1 Canonical Form and Its Properties . . . . . . . . . . . . . . . . . . . . . . . . 81

3.1.1 Chained Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 823.1.2 Controllability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 853.1.3 Feedback Linearization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 873.1.4 Options of Control Design . . . . . . . . . . . . . . . . . . . . . . . . . 903.1.5 Uniform Complete Controllability . . . . . . . . . . . . . . . . . . . 923.1.6 Equivalence and Extension of Chained Form . . . . . . . . . . 96

3.2 Steering Control and Real-time Trajectory Planning . . . . . . . . . 983.2.1 Navigation of Chained Systems . . . . . . . . . . . . . . . . . . . . . 983.2.2 Path Planning in a Dynamic Environment . . . . . . . . . . . . 1043.2.3 A Real-time and Optimized Path Planning Algorithm . 109

3.3 Feedback Control of Non-holonomic Systems . . . . . . . . . . . . . . . 1163.3.1 Tracking Control Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 1183.3.2 Quadratic Lyapunov Designs of Feedback Control . . . . . 1203.3.3 Other Feedback Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

3.4 Control of Vehicle Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1303.4.1 Formation Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1313.4.2 Multi-objective Reactive Control . . . . . . . . . . . . . . . . . . . 136


4 Matrix Theory for Cooperative Systems . . . . . . . . . . . . . . . . . . 1534.1 Non-negative Matrices and Their Properties . . . . . . . . . . . . . . . . 153

4.1.1 Reducible and Irreducible Matrices . . . . . . . . . . . . . . . . . . 1544.1.2 Perron-Frobenius Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 1554.1.3 Cyclic and Primitive Matrices . . . . . . . . . . . . . . . . . . . . . . 158

Contents xiii

4.2 Importance of Non-negative Matrices . . . . . . . . . . . . . . . . . . . . . . 1614.2.1 Geometrical Representation of Non-negative Matrices . . 1654.2.2 Graphical Representation of Non-negative Matrices . . . . 166

4.3 M-matrices and Their Properties . . . . . . . . . . . . . . . . . . . . . . . . . 1674.3.1 Diagonal Dominance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1674.3.2 Non-singular M-matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 1684.3.3 Singular M-matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1704.3.4 Irreducible M-matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1724.3.5 Diagonal Lyapunov Matrix . . . . . . . . . . . . . . . . . . . . . . . . . 1744.3.6 A Class of Interconnected Systems . . . . . . . . . . . . . . . . . . 176

4.4 Multiplicative Sequence of Row-stochastic Matrices . . . . . . . . . 1774.4.1 Convergence of Power Sequence . . . . . . . . . . . . . . . . . . . . . 1784.4.2 Convergence Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1794.4.3 Sufficient Conditions on Convergence . . . . . . . . . . . . . . . . 1874.4.4 Necessary and Sufficient Condition on Convergence . . . 188


5 Cooperative Control of Linear Systems . . . . . . . . . . . . . . . . . . . 1955.1 Linear Cooperative System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

5.1.1 Characteristics of Cooperative Systems . . . . . . . . . . . . . . 1965.1.2 Cooperative Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1985.1.3 A Simple Cooperative System . . . . . . . . . . . . . . . . . . . . . . 200

5.2 Linear Cooperative Control Design . . . . . . . . . . . . . . . . . . . . . . . . 2015.2.1 Matrix of Sensing and Communication Network . . . . . . . 2025.2.2 Linear Cooperative Control . . . . . . . . . . . . . . . . . . . . . . . . 2035.2.3 Conditions of Cooperative Controllability . . . . . . . . . . . . 2065.2.4 Discrete Cooperative System . . . . . . . . . . . . . . . . . . . . . . . 209

5.3 Applications of Cooperative Control . . . . . . . . . . . . . . . . . . . . . . . 2105.3.1 Consensus Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2105.3.2 Rendezvous Problem and Vector Consensus . . . . . . . . . . 2125.3.3 Hands-off Operator and Virtual Leader . . . . . . . . . . . . . . 2125.3.4 Formation Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2165.3.5 Synchronization and Stabilization of Dynamical

Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2225.4 Ensuring Network Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . 2235.5 Average System and Its Properties . . . . . . . . . . . . . . . . . . . . . . . . 2265.6 Cooperative Control Lyapunov Function . . . . . . . . . . . . . . . . . . . 229

5.6.1 Fixed Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2305.6.2 Varying Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

5.7 Robustness of Cooperative Systems . . . . . . . . . . . . . . . . . . . . . . . 2385.8 Integral Cooperative Control Design . . . . . . . . . . . . . . . . . . . . . . 2485.9 Notes and Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

xiv Contents

6 Cooperative Control of Non-linear Systems . . . . . . . . . . . . . . . 2536.1 Networked Systems with Balanced Topologies . . . . . . . . . . . . . . 2546.2 Networked Systems of Arbitrary Topologies . . . . . . . . . . . . . . . . 256

6.2.1 A Topology-based Comparison Theorem . . . . . . . . . . . . . 2566.2.2 Generalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

6.3 Cooperative Control Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2666.3.1 Systems of Relative Degree One . . . . . . . . . . . . . . . . . . . . 2676.3.2 Systems in the Feedback Form . . . . . . . . . . . . . . . . . . . . . 2706.3.3 Affine Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2746.3.4 Non-affine Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2756.3.5 Output Cooperation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

6.4 Discrete Systems and Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 2786.5 Driftless Non-holonomic Systems . . . . . . . . . . . . . . . . . . . . . . . . . 281

6.5.1 Output Rendezvous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2816.5.2 Vector Consensus During Constant Line Motion . . . . . . . 285

6.6 Robust Cooperative Behaviors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2886.6.1 Delayed Sensing and Communication . . . . . . . . . . . . . . . . 2906.6.2 Vehicle Cooperation in a Dynamic Environment . . . . . . . 294


References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

List of Figures

1.1 Block diagram of a standard feedback control system . . . . . . . . . 31.2 Block diagram of a networked control system . . . . . . . . . . . . . . . . 31.3 Ad hoc sensor network for mobile robots . . . . . . . . . . . . . . . . . . . . . 41.4 Boid’s neighborhood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.5 Cooperative behaviors of animal flocking . . . . . . . . . . . . . . . . . . . . 61.6 Synchronized motion of planar motion particles . . . . . . . . . . . . . . 81.7 Zones for the maintain-formation motor schema . . . . . . . . . . . . . . 91.8 Voronoi diagram for locating mobile sensors . . . . . . . . . . . . . . . . . 101.9 A differential-drive vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191.10 A car-like robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211.11 A fire truck . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231.12 An n-trailer system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241.13 A planar space robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251.14 A surface vessel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301.15 An unmanned aerial vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321.16 Hierarchical control structure for autonomous vehicle system . . . 35

2.1 Incremental motion in terms of Lie bracket . . . . . . . . . . . . . . . . . . 71

3.1 Two-dimensional explanation of the Brockett condition . . . . . . . . 913.2 Trajectories under sinusoidal inputs . . . . . . . . . . . . . . . . . . . . . . . . . 1013.3 Trajectories under piecewise-constant inputs . . . . . . . . . . . . . . . . . 1033.4 Trajectories under polynomial inputs . . . . . . . . . . . . . . . . . . . . . . . 1053.5 Physical envelope of a car-like robot . . . . . . . . . . . . . . . . . . . . . . . . 1073.6 Trajectory planning in a dynamic environment . . . . . . . . . . . . . . . 1083.7 Illustration of collision avoidance in the transformed plane . . . . . 1133.8 The geometrical meaning of performance index . . . . . . . . . . . . . . . 1153.9 The optimized trajectory under the choices in Table 3.1 . . . . . . . 1173.10 Performance under Tracking Control 3.74 . . . . . . . . . . . . . . . . . . . . 1213.11 Performance of Stabilizing Controls 3.86 and 3.84 . . . . . . . . . . . . 1253.12 Performance under Controls 3.88 and 3.94 . . . . . . . . . . . . . . . . . . . 127

xvi List of Figures

3.13 Illustration of a formation in 2-D . . . . . . . . . . . . . . . . . . . . . . . . . . . 1313.14 Performance under formation decomposition and control . . . . . . 1333.15 Performance under a leader-follower formation control . . . . . . . . 1353.16 Vector field of ∂P (q − qd, q − qo)/∂q under (3.107) and (3.108) . 1403.17 Reactive Control 3.118: a static obstacle . . . . . . . . . . . . . . . . . . . . . 1463.18 Performance of reactive control: static and moving obstacles . . . 1483.19 Reactive Control 3.118: static and moving obstacles . . . . . . . . . . . 149

4.1 Examples of graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

5.1 Motion alignment and cooperative behavior of self-drivenparticles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

5.2 Velocity consensus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2115.3 Vector consensus: convergence per channel . . . . . . . . . . . . . . . . . . . 2135.4 Rendezvous under cooperative controls . . . . . . . . . . . . . . . . . . . . . . 2145.5 Rendezvous to the desired target position . . . . . . . . . . . . . . . . . . . 2165.6 Performance under Cooperative Formation Control 5.28 . . . . . . . 2185.7 Performance under Cooperative Formation Control 5.29 . . . . . . . 2215.8 Rendezvous in the presence of measurement bias . . . . . . . . . . . . . 2455.9 Robustification via a virtual leader . . . . . . . . . . . . . . . . . . . . . . . . . 2465.10 Integral cooperative control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

6.1 Performance of linear cooperative system . . . . . . . . . . . . . . . . . . . . 2686.2 Performance of non-linear cooperative system . . . . . . . . . . . . . . . . 2696.3 Extended response of the non-linear cooperative system in

Fig. 6.2(a) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2706.4 Performance of System 6.24 under Cooperative Control 6.25 . . . 2736.5 Performance under Cooperative Control 6.30 . . . . . . . . . . . . . . . . . 2766.6 Output consensus of three vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . 2846.7 Control inputs for the vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2866.8 Vector consensus of chained systems . . . . . . . . . . . . . . . . . . . . . . . . 2896.9 Cooperative Control 6.49 and 6.51 . . . . . . . . . . . . . . . . . . . . . . . . . . 2916.10 Consensus under latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2936.11 Performance under Cooperative Reactive Control 6.60 . . . . . . . . 2966.12 Cooperative Reactive Control 6.60 . . . . . . . . . . . . . . . . . . . . . . . . . . 2976.13 Responses over [0, 30] and under Cooperative Reactive Control

6.60 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2996.14 Responses over [30, 80] and under Cooperative Reactive

Control 6.60 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3006.15 Responses over [80, 160] and under Cooperative Reactive

Control 6.60 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

1

Introduction

In this chapter, the so-called cooperative and pliable systems and their char-acteristics are described. It is shown through simple examples that cooperativeand pliable behaviors emerge naturally in many complex systems and are alsodesirable for the operation of engineering systems such as autonomous vehi-cles. Indeed, a team of heterogeneous vehicles can autonomously interact witheach other and their surroundings and exhibit cooperative and pliable behav-iors upon implementing a multi-level networked-based local-feedback controlon the vehicles.

Heterogeneous entities in cooperative systems may be as simple as scalaragents without any dynamics, or may be those described by a second-orderpoint-mass model, or may be as complicated as nonlinear dynamical systemswith nonholonomic motion constraints. In this chapter, standard modelingmethods and basic models of typical vehicles are reviewed, providing the foun-dation for analysis and control of autonomous vehicle systems.

1.1 Cooperative, Pliable and Robust Systems

Although there isn’t a universal definition, a cooperative, pliable and robustsystem consisting of several sub-systems should have the following character-istics:

(a) Trajectories of all the sub-systems in a cooperative system move collab-oratively toward achieving a common objective. In stability and controltheory, the common objective correspond to equilibrium points. Differentfrom a standard stability analysis problem, equilibrium points of a coop-erative system may not be selected a priori. Rather, the equilibrium pointreached by the cooperative system may depend upon such factors as ini-tial conditions, changes of the system dynamics, and influence from itsenvironment. Later in Chapter 5, cooperative stability will be defined toquantify mathematically the characteristic of sub-systems moving towardthe common objective.

2 1 Introduction

(b) Associated with a cooperative system, there is a network of either sensorsor communication links or a mix of both. The sensing/communicationnetwork provides the means of information exchange among the sub-systems and, unless sensor control or a communication protocol is explic-itly considered, its changes can be modeled over time as binary variablesof either on or off. Different from a standard control problem, feedbackpatterns of the network over time may not be known a priori. As such, notonly do individual sub-systems satisfy certain controllability conditions,but also the networked system must require cooperative controllability.In addition to intermittency, the network may also suffer from limitedbandwidth, latency, noises, etc.

(c) The cooperative system generally evolves in a dynamically changing phys-ical environment. As exogenous dynamics, the environment has a directimpact on the status of sensing and communication network. Anothermajor impact of the environment is the geometrical constraints on themaneuverable sub-space. In the process of achieving the common goal,motion of all the sub-systems must be robust in the sense that their tra-jectories comply with changes of the environment.

(d) Motion by each of the sub-systems may also be subject to constraintsin kinematics or dynamics or both. As a result, control design must alsobe pliable in the sense that the resulting motion trajectory complies withthese constraints.

Control through an intermittent network, cooperative behaviors of multipleentities, and pliable and robust systems are of particular interest and henceare discussed subsequently.

1.1.1 Control Through an Intermittent Network

Figure 1.1 shows the standard configuration of a feedback control system, inwhich the reference input r(t) is given and the control objective is to makeoutput y(t) converge to r(t). Typically, feedback of sensor S is of most impor-tance in the system, the sensor model is usually known, and its informationflow is either continuous or continually available at a prescribed series of timeinstants. Plant P is mostly known, while unknowns such as disturbances anduncertainties may be considered. Depending upon plant dynamics and its un-knowns, control C can be designed. For instance, linear and non-linear controlscan be designed, an adaptive law can be introduced if P contains unknownparameters, and robust control can be added to compensate for uncertaintiesin P, etc. It is well known that, under any control, the sensitivity function ofthe closed-loop system with respect to uncertainties in S always has a valueclose to 1. As such, most feedback control systems do not perform well if thefeedback is significantly delayed or if the feedback is available on an irregular,sporadic, or unpredictable basis.

Through a high-speed and reliable data network, feedback control systemscan be implemented. Figure 1.2 shows a network control system in which

1.1 Cooperative, Pliable and Robust Systems 3

)(tr )(tyC P

S

Fig. 1.1. Block diagram of a standard feedback control system

1P

2P

qP

1C

2C

qC

NetworkN

Hands-off

Operator

Fig. 1.2. Block diagram of a networked control system

multiple plants are admissible. Compared to Fig. 1.1, the networked controlsystem has the same setup. The main difference is that information generallyflows through sensing or communication network N because the plants as wellas control stations are often geographically distributed. A dedicated networkcan be used to ensure stability and performance of networked control systems.If feasible and deemed to be necessary, a centralized control with global infor-mation can be designed to achieve the optimal performance. For robustness,it is better to have each of the plants stabilized by its local control, whilethe network is to provide information sharing toward synthesizing or adjust-ing reference inputs to the plants. Nonetheless, the presence of the networkedloop could cause problems for stability and performance of the overall system.Indeed, for large-scale interconnected systems, decentralized control (feedbackcontrol individually installed at each of the plants) is often desired.

Unless stated otherwise, the networked control problem studied in thisbook is the one in which information flow through network N is intermit-tent, of limited bandwidth and range, and in certain directions, cannot bepredicted by any deterministic or stochastic model, and may have significantlatency. Examples of such a network include certain wireless communication

4 1 Introduction

Sensor radius

Fig. 1.3. Ad hoc sensor network for mobile robots

networks, ad hoc sensor networks, etc. Figure 1.3 shows an ad hoc sensornetwork among a group of mobile robots. As robots move relative to eachother, feedback information available to a given robot is only about thoseneighboring robots in a certain region of its vicinity. Consequently, motioncontrol of the mobile robot is restricted by the network to be local feedback(only from those neighboring robots). And, the control objective is for therobotic vehicles to exhibit certain group behavior (which is the topic of thesubsequent subsection). Should an operator be involved to inject commandsignals through the same network, a virtual vehicle can be added into thegroup to represent the operator. The questions to be answered in this bookinclude the following. (i) Controllability via a network and networked con-trol design: given the intermittent nature of the network, the (necessary andsufficient) condition needs to be found under which a network-enabled localfeedback control can be designed to achieve the control objective. (ii) Vehiclecontrol design and performance: based on models of robotic vehicles, vehiclefeedback controls are designed to ensure certain performance.

1.1.2 Cooperative Behaviors

Complex systems such as social systems, biological systems, science and engi-neering systems often consist of heterogeneous entities which have individualcharacteristics, interact with each other in various ways, and exhibit certaingroup phenomena. Since interactions existing among the entities may bequite sophisticated, it is often necessary to model these complex systems asnetworked systems. Cooperative behaviors refer to their group phenomena


angle

distance

Fig. 1.4. Boid’s neighborhood

and characteristics, and they are manifestation of the algorithms or controlsgoverning each of the entities in relationship to its local environment. To il-lustrate the basics without undue complexity, let us consider a group of finiteentities whose dynamics are overlooked in this subsection. In what follows,sample systems and their typical cooperative behaviors are outlined. Math-ematically, these cooperative behaviors could all be captured by the limit of[Gi(yi) − Gj(yj)] → 0, where yi is the output of the ith entity, and Gi(·) is acertain (set-based) mapping associated with the ith entity. It is based on thisabstraction that the so-called cooperative stability will be defined in Chapter5. Analysis and synthesis of linear and non-linear cooperative models as wellas their behaviors will systematically be carried out in Chapters 5 and 6.

Boid’s Model of Animal Flocking

As often observed, biological systems such as the groups of ants, fish, birdsand bacteria reveal some amazing cooperative behaviors in their motion. Com-puter simulation and animation have been used to generate generic simulatedflocking creatures called boids [217]. To describe interactions in a flock of birdsor a school of fish, the so-called boid’s neighborhood can be used. As shownin Fig. 1.4, it is parameterized in terms of a distance and an angle, and ithas been used as a computer animation/simulation model. The behavior ofeach boid is influenced by other boids within its neighborhood. To generateor capture cooperative behaviors, three common steering rules of cohesion,separation and alignment can be applied. By cohesion, an entity moves to-ward the center of mass of its neighboring entities; by separation, the entity ismoving away from the nearest boids in order to avoid possible collision; andby alignment, the entity adjusts its heading according to the average head-ing of those boids in the neighborhood. Graphically, the rules of separation,alignment and cohesion are depicted in Fig. 1.5.

6 1 Introduction

Separation Alignment Cohesion

Fig. 1.5. Cooperative behaviors of animal flocking

Couzin Model of Animal Motion

The Couzin model [45] can be used to describe how the animals in a groupmake their movement decision with a limited pertinent information. For agroup of q individuals moving at a constant velocity, heading φ′

i of the ithindividual is determined by

φ′i(k + 1) =

φi(k + 1) + ωiφdi

‖φi(k + 1) + ωiφdi ‖

, (1.1)

where φdi is the desired direction (along certain segment of a migration route

to some resource), 0 ≤ ωi ≤ 1 is a weighting constant,⎧

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎩

φi(k + 1) = −∑

j∈N ′i

xj(k) − xi(k)

dij(k), if N ′

i is not empty

φi(k + 1) =∑

j∈Ni

xj(k) − xi(k)

dij(k)+

∑

j∈Ni∪i

φj(k)

‖φj(k)‖ , if N ′i is empty

(1.2)xi(t) is the position vector of the ith individual, dij(k) = ‖xj(k) − xi(k)‖is the norm-based distance between the ith and jth individuals, α is theminimum distance between i and j, β represents the range of interaction,Ni(k) = j : 1 ≤ j ≤ q, dij(k) ≤ β is the index set of individual i’sneighbors, and N ′

i = j : 1 ≤ j ≤ q, dij(k) < α is the index set of thoseneighbors that are too close. In (1.1), the individuals with ωi > 0 are informedabout their preferred motion direction, and the rest of individuals with ωi = 0are not. According to (1.2), collision avoidance is the highest priority (which isachieved by moving away from each other) and, if collision is not expected, theindividuals will tend to attract toward and align with each other. Experimentsshow that, using the Couzin model of (1.1), only a very small proportion ofindividuals needs to be informed in order to guide correctly the group motionand that the informed individuals do not have to know their roles.


Aggregation and Flocking of Swarms

Aggregation is a basic group behavior of biological swarms. Simulation modelshave long been studied and used by biologists to animate aggregation [29, 178].Indeed, an aggregation behavior can be generated using a simple model. Forinstance, an aggregation model based on an artificial potential field functionis given by

xi =

q∑

j=1,j =i

(

c1 − c2e− ‖xi−xj‖2

c3

)

(xj − xi), (1.3)

where xi ∈ ℜn, i ∈ 1, · · · , q is the index integer, and ci are positive constants.It is shown in [76] that, if all the members move simultaneously and shareposition information with each other, aggregation is achieved under Model1.3 in the sense that all the members of the swarm converge into a smallregion around the swarm center defined by

∑qi=1 xi/q. Model 1.3 is similar

to the so-called artificial social potential functions [212, 270]. Aggregationbehaviors can also be studied by using probabilistic method [240] or discrete-time models [75, 137, 138].

Flocking motion is another basic group behavior of biological swarms.Loosely speaking, flocking motion of a swarm is characterized by the commonvelocity to which velocities of the swarm members all converge. A simpleflocking model is

vi(k + 1) = vi(k) +

q∑

j=1

aij(xi, xj)[vj(k) − vi(k)], (1.4)

where vi(t) is the velocity of the ith member, the influence on the ith memberby the jth member is quantified by the non-linear weighting term of

aij(xi, xj) =k0

(k21 + ‖xi − xj‖2)k2

,

kl are positive constants, and xi is the location of the ith member. Charac-teristics of the flocking behavior can be analyzed in terms of parameters ki

and initial conditions [46].

Synchronized Motion

Consider a group of q particles which move in a plane and have the same linearspeed v. As shown in Fig. 1.6, motion of the particles can be synchronized byaligning their heading angles. A localized alignment algorithm is given by theso-called Vicsek model [263]:

θi(k + 1) =1

1 + ni(k)

⎛

⎝θi(k) +∑

j∈Ni(k)

θj(k)

⎞

⎠ , (1.5)

8 1 Introduction

jp

ip

lp

v

v

v

j

i

l

Fig. 1.6. Synchronized motion of planar motion particles

where ℵ is the set of all non-negative integers, k ∈ ℵ is the discrete-time index,Ni(k) is the index set of particle i’s neighbors at time k, and ni(k) is thenumber of entries in set Ni(k). Essentially, Model 1.5 states that the headingof any given particle is adjusted regularly to the average of those headings ofits neighbors. It is shown by experimentation and computer simulation [263]or by a graph theoretical analysis [98] that, if sufficient information is sharedamong the particles, headings of the particles all converge to the same value.Closely related to particle alignment is the stability problem of nano-particlesfor friction control [82, 83].

Synchronization arises naturally from such physical systems as coupledoscillators. For the set of q oscillators with angles θi and natural frequenciesωi, their coupling dynamics can be described by the so-called Kuramoto model[247]:

θi = ωi +∑

j∈Ni

kij sin(θj − θi), (1.6)

where Ni is the coupling set of the ith oscillator, and kij are positive constantsof coupling strength. In the case that ωi = ω for all i and that there is noisolated oscillator, it can be shown [136] that angles θi(t) have the same limit.Alternatively, by considering the special case of θi ∈ (−π/2, π/2) and wi = 0for all i, Model 1.6 is converted under transformation xi = tan θi to

xi =∑

j∈Ni

kji

√

1 + x2j

√

1 + x2i

(xj − xi). (1.7)

Model 1.7 can be used as the alternative to analyze local synchronization ofcoupled oscillators [161].

Artificial Behaviors of Autonomous Vehicles

In [13, 150], artificial behaviors of mobile robots are achieved by institut-ing certain basic maneuvers and a set of rules governing robots’ interactions


Vehicle 2

Vehicle 1

Zone 2

Zone 1

Zone 3

Fig. 1.7. Zones for the maintain-formation motor schema

and decision-making. Specifically, artificial formation behaviors are defined in[13] in terms of such maneuvers as move-to-goal, avoid-static-obstacle, avoid-robot, maintain-formation, etc. These maneuvers are called motor schemas,and each of them generates a motion vector of the direction and magnitudeto represent the corresponding artificial behavior under the current sensoryinformation. For instance, for a vehicle undertaking the maintain-formationmotor schema, its direction and magnitude of motion vector is determined bygeometrical relationship between its current location and the correspondingdesired formation position. As an example, Fig. 1.7 depicts one circular zoneand two ring-like zones, all concentric at the desired formation location ofvehicle 1 (in order to maintain a formation with respect to vehicle 2). Then,the motion direction of vehicle 1 is that pointing toward the desired locationfrom its current location, and the motion magnitude is set to be zero if thevehicle is in zone 1 (the circular zone called dead zone), or to be propor-tional to the distance between its desired and current locations if the vehiclelies in zone 2 (the inner ring, also called the controlled zone), or to be themaximum value if the vehicle enters zone 3 (the outer ring). Once motionvectors are calculated under all relevant motor schemas, the resulting motionof artificial behavior can be computed using a set of specific rules. Clearly,rule-based artificial behaviors are intuitive and imitate certain animal groupbehaviors. On-going research is to design analytical algorithms and/or explicitprotocols of neighboring-feedback cooperation for the vehicles by synthesizinganalytical algorithms of emulating animal group behaviors, by making vehiclemotion comply with a dynamically changing and uncertain environment, andby guaranteeing their mission completion and success.

Coverage Control of Mobile Sensor Network

A group of mobile vehicles can be deployed to undertake such missions assearch and exploration, surveillance and monitoring, and rescue and recovery.In these applications, vehicles need to coordinate their motion to form an ad-

10 1 Introduction

hoc communication network or act individually as a mobile sensor. Towardthe goal of motion cooperation, a coverage control problem needs to be solvedto determine the optimal spatial resource allocation [44]. Given a convex poly-tope Ω and a distribution density function φ(·), the coverage control problemof finding optimal static locations of P = [p1, · · · , pn]T for n sensors/robotscan be mathematically formulated as

P = minpi,Wi

n∑

i=1

∫

Wi

‖q − pi‖dφ(q),

where partition Wi ⊂ Ω is the designated region for the ith sensor at positionpi ∈ Wi. If distribution density function φ(·) is uniform in Ω, partitions Wi

are determined solely by distance as

Wi = q ∈ Q | ‖q − pi‖ ≤ ‖q − pj‖, ∀j = i,

and so are pi. In this case, the solutions are given by the Voronoi diagram inFig. 1.8: a partition of space into cells, each of which consists of the pointscloser to one pi than to any others pj. If distribution density function φ(·)is not uniform, solutions of Wi and pi are weighted by the distribution den-sity function. Dynamically, gradient descent algorithms can be used to solvethis coverage control problem in an adaptive, distributed, and asynchronousmanner [44].

Fig. 1.8. Voronoi diagram for locating mobile sensors

1.1.3 Pliable and Robust Systems

A control system shown in Fig. 1.1 is said to be pliable if all the constraints ofthe plant are met. Should the plant be a mechanical system, it will be shownin Sections 1.2 and 1.3 that the constraints may be kinematic or dynamicor both. A networked control system in Fig. 1.2 is pliable if all the plantssatisfy their constraints and if all the network-based controls conform withinformation flow in the network.

1.2 Modeling of Constrained Mechanical Systems 11

In addition to the sensing and communication network, the networkedcontrol system must also be robust by complying to the changes (if any) in itsphysical environment. For an autonomous multi-vehicle system, this meansthat the vehicles can avoid static and moving obstacles and that they canalso avoid each other. The latter is of particular importance for cooperativecontrol because many of cooperative behaviors require that vehicles stay inclose vicinity of each other while moving.

Indeed, as performance measures, autonomous vehicles are required to beboth cooperative, pliable, and robust. The multiple control objectives for thevehicle systems and for cooperative, robust and pliable systems in generalimply that multi-level controls need to be properly designed, which is thesubject of Section 1.4.

1.2 Modeling of Constrained Mechanical Systems

Most vehicles are constrained mechanical systems, and their models can bederived using fundamental principles in rigid-body mechanics and in terms oflumped parameters. In this section, standard modeling techniques and stepsare summarized. Unless stated otherwise, all functions are assumed to besmooth.

1.2.1 Motion Constraints

Let ℜn be the configuration space, q ∈ ℜn be the vector of generalized coordi-nates, and q ∈ ℜn be the vector of generalized velocities. A matrix expressionof k motion constraints is

A(q, t)q + B(q, t) = 0 ∈ ℜk, (1.8)

where 0 < k < n and A(q, t) ∈ ℜk×n. Constraints 1.8 are said to be holo-nomic or integrable if they correspond to algebraic equations only in terms ofconfiguration variables of q as

β(q, t) = 0 ∈ ℜk.

Therefore, given the set of holonomic constraints, the matrices in (1.8) can becalculated as

A(q, t) =∂β(q, t)

∂q, B(q, t) =

∂β(q, t)

∂t;

conversely, given the constraints in (1.8), their integrability can be determined(using the Frobenius theorem which is to be stated in Section 2.6.2 and givesnecessary and sufficient conditions), or the following equations can be verifiedas necessary conditions: for all meaningful indices i, j, l:

∂aij(q, t)

∂ql=

∂ail(q, t)

∂qj=

∂2βi(q, t)

∂ql∂qj,

∂aij(q, t)

∂t=

∂bi(q, t)

∂qj=

∂2βi(q, t)

∂t∂qj,

12 1 Introduction

where aij(q, t) denote the (i, j)th element of A(·), and bj(q, t) denote the jthelement of B(·). If constraints are not integrable, they are said to be non-holonomic.

Constraints 1.8 are called rheonomic if they explicitly depend upon time,and scleronomic otherwise. Constraints 1.8 are called driftless if they are lin-ear in velocities or B(q, t) = 0. If B(q, t) = 0, the constraints have drift, whichwill be discussed in Section 1.2.6. Should Constraints 1.8 be both scleronomicand driftless, they reduce to the so-called Pfaffian constraints of form

A(q)q = 0 ∈ ℜk. (1.9)

If Pfaffian constraints are integrable, the holonomic constraints become

β(q) = 0 ∈ ℜk. (1.10)

Obviously, holonomic constraints of (1.10) restrict the motion of q on an (n−k)-dimensional hypersurface (or sub-manifold). These geometric constraintscan always be satisfied by reducing the number of free configuration variablesfrom n to (n − k). For instance, both a closed-chain mechanical device andthe two robots holding a common object are holonomic constraints. On theother hand, a rolling motion without side slipping is the typical non-holonomicconstraint in the form of (1.9). These non-holonomic constraints require thatall admissible velocities belong to the null space of matrix A(q), while themotion in the configuration space of q ∈ ℜn is not limited. To ensure non-holonomic constraints, a kinematic model and the corresponding kinematiccontrol need to be derived, which is the subject of the next subsection.

1.2.2 Kinematic Model

Consider a mechanical system kinematically subject to non-holonomic velocityconstraints in the form of (1.9). Since A(q) ∈ ℜk×n with k < n, it is alwayspossible to find a rank-(n− k) matrix G(q) ∈ ℜn×(n−k) such that columns ofG(q), gj(q), consist of smooth and linearly independent vector fields spanningthe null space of A(q), that is,

A(q)G(q) = 0, or A(q)gj(q) = 0, j = 1, · · · , n − k. (1.11)

It then follows from (1.9) and (1.11) that q can be expressed in terms of alinear combination of vector fields gi(q) and as

q = G(q)u =

n−k∑

i=1

gi(q)ui, (1.12)

where auxiliary input u =[

u1 · · · un−k

]T ∈ ℜm is called kinematic control.Equation 1.12 gives the kinematic model of a non-holonomic system, and thenon-holonomic constraints in (1.9) are met if the corresponding velocities areplanned by or steered according to (1.12). Properties of Model 1.12 will bestudied in Section 3.1.


1.2.3 Dynamic Model

The dynamic equation of a constrained (non-holonomic) mechanical systemcan be derived using variational principles. To this end, let L(q, q) be theso-called Lagrangian defined by

L(q, q) = K(q, q) − V (q),

where K(·) is the kinetic energy of the system, and V (q) is the potentialenergy. Given time interval [t0, tf ] and terminal conditions q(t0) = q0 andq(tf ) = qf , we can define the so-called variation q(t, ǫ) as a smooth mappingsatisfying q(t, 0) = q(t), q(t0, ǫ) = q0 and q(tf , ǫ) = qf . Then, quantity

δq(t)=

∂q(t, ǫ)

∂ǫ

∣

∣

∣

∣

ǫ=0

is the virtual displacement corresponding to the variation, and its boundaryconditions are

δq(t0) = δq(tf ) = 0. (1.13)

In the absence of external force and constraint, the Hamilton principlestates that the system trajectory q(t) is the stationary solution with respectto the time integral of the Lagrangian, that is,

∂

∂ǫ

∫ tf

t0

L(q(t, ǫ), q(t, ǫ))dt

∣

∣

∣

∣

ǫ=0

= 0.

Using the chain rule, we can compute the variation of the integral correspond-ing to variation q(t, ǫ) and express it in terms of its virtual displacement as

∫ tf

t0

[

(

∂L

∂q

)T

δq +

(

∂L

∂q

)T

δq

]

dt = 0.

Noting that δq = d(δq)/dt, integrating by parts, and substituting the bound-ary conditions in (1.13) yield

∫ tf

t0

[

(

− d

dt

∂L

∂q+

∂L

∂q

)T

δq

]

dt = 0. (1.14)

In the presence of any input τ ∈ ℜm, the Lagrange-d’Alembert principlegeneralizes the Hamilton principle as

∂

∂ǫ

∫ tf

t0

L(q(t, ǫ), q(t, ǫ))dt

∣

∣

∣

∣

ǫ=0

+

∫ tf

t0

[B(q)τ ]T δq

dt = 0, (1.15)

where B(q) ∈ ℜn×m maps input τ into forces/torques, and [B(q)τ ]T δq isthe virtual work done by force τ with respect to virtual displacement δq.

14 1 Introduction

Repeating the derivations leading to (1.14) yields the so-called Lagrange-Eulerequation of motion:

d

dt

(

∂L

∂q

)

− ∂L

∂q= B(q)τ. (1.16)

If present, constraints in (1.9) exert a constrained force vector F on the sys-tem which undergoes a virtual displacement δq. As long as the non-holonomicconstraints are enforced by these forces, the system can be thought to beholonomic but subject to the constrained forces, that is, its equation of mo-tion is given by (1.16) with B(q)τ replaced by F . The work done by theseforces is FT δq. The d’Alembert principle about the constrained forces is that,with respect to any virtual displacement consistent with the constraints, theconstrained forces do no work as

FT δq = 0, (1.17)

where virtual displacement δq is assumed to satisfy the constraint equationof (1.9), that is,

A(q)δq = 0. (1.18)

Note that δq and q are different since, while virtual displacement δq satis-fies only the constraints, the generalized velocity q satisfies both the velocityconstraints and the equation of motion. Comparing (1.17) and (1.18) yields

F = AT (q)λ,

where λ ∈ ℜk is the Lagrange multiplier. The above equation can also beestablished using the same argument that proves the Lagrange multiplier the-orem.

In the presence of both external input τ ∈ ℜm and the non-holonomicconstraints of (1.9), the Lagrange-d’Alembert principle also gives (1.15) ex-cept that δq satisfies (1.18). Combining the external forces and constrainedforces, we obtain the following Euler-Lagrange equation, also called Lagrange-d’Alembert equation:

d

dt

(

∂L

∂q

)

− ∂L

∂q= AT (q)λ + B(q)τ. (1.19)

Note that, in general, substituting a set of Pfaffian constraints into the La-grangian and then applying the Lagrange-d’Alembert equations render equa-tions that are not correct. See Section 1.4 in Chapter 6 of [168] for an illus-trative example.

The Euler-Lagrange equation or Lagrange-d’Alembert equation is conve-nient to derive and use because it does not need to account for any forcesinternal to the system and it is independent of coordinate systems. These twoadvantages are beneficial, especially in handling a multi-body mechanical sys-tem with moving coordinate frames. The alternative Newton-Euler methodwill be introduced in Section 1.3.5.


1.2.4 Hamiltonian and Energy

If inertia matrix

M(q)=

∂2L

∂q2

is non-singular, Lagrangian L is said to be regular. In this case, the Legendretransformation changes the state from [qT qT ]T to [qT pT ]T , where

p =∂L

∂q

is the so-called momentum. Then, the so-called Hamiltonian is defined as

H(q, p)= pT q − L(q, q). (1.20)

It is straightforward to verify that, if τ = 0, the Lagrange-Euler Equation of(1.16) becomes the following Hamilton equations:

q =∂H

∂p, p = −∂H

∂q.

In terms of symmetrical inertia matrix M(q), the Lagrangian and Hamil-tonian can be expressed as

L =1

2qT M(q)q − V (q), H =

1

2qT M(q)q + V (q). (1.21)

That is, the Lagrangian is kinetic energy minus potential energy, the Hamil-tonian is kinetic energy plus potential energy, and hence the Hamiltonian isthe energy function. To see that non-holonomic systems are conservative un-der τ = 0, we use energy function H and take its time derivative along thesolution of (1.19), that is,

dH(q, p)

dt=

[

d

dt

(

∂L

∂q

)T]

q +

(

∂L

∂q

)T

q −(

∂L

∂q

)T

q −(

∂L

∂q

)T

q

= λT A(q)q,

which is zero according to (1.9).

1.2.5 Reduced-order Model

It follows from (1.21), (1.19) and (1.9) that the dynamic equations of a non-holonomic system are given by

M(q)q + N(q, q) = AT (q)λ + B(q)τ,A(q)q = 0,

(1.22)

16 1 Introduction

where q ∈ ℜn, λ ∈ ℜk, τ ∈ ℜm, N(q, q) = C(q, q)q + fg(q), fg(q) = ∂V (q)/∂qis the vector containing gravity terms, and C(q, q) ∈ ℜn×n is the matrixcontaining centrifugal forces (of the terms q2

i ) and Coriolis forces (of termsqiqj). Matrix C(q, q) can be expressed in tensor as

C(q, q) = M(q) − 1

2qT

(

∂

∂qM(q)

)

,

or element-by-element as

Clj(q, q) =1

2

n∑

i=1

[

∂mlj(q)

∂qi+

∂mli(q)

∂qj− ∂mij(q)

∂ql

]

qi, (1.23)

where mij(q) are the elements of M(q). Embedded in (1.23) is the inherent

property that M(q) − 2C(q, q) is skew symmetrical, which is explored exten-sively in robotics texts [130, 195, 242]. There are (2n+ k) equations in (1.22),and there are (2n + k) variables in q, q, and λ.

Since the state is of dimension 2n and there are also k constraints, a totalof (2n − k) reduced-order dynamic equations are preferred for analysis anddesign. To this end, we need to pre-multiply GT (q) on both sides of the firstequation in (1.22) and have

GT (q)M(q)q + GT (q)N(q, q) = GT (q)B(q)τ,

in which λ is eliminated by invoking (1.11). On the other hand, differentiating(1.12) yields

q = G(q)u + G(q)u.

Combining the above two equations and recalling (1.12) render the followingreduced dynamic model of non-holonomic systems:

q = G(q)uM ′(q)u + N ′(q, u) = GT (q)B(q)τ,

(1.24)

where

M ′(q) = GT (q)M(q)G(q), N ′(q, u) = GT (q)M(q)G(q)u+GT (q)N(q, G(q)u).

Equation 1.24 contains (2n − k) differential equations for exactly (2n − k)variables. If m = n − k and if GT (q)B(q) is invertible, the second equationin (1.24) is identical to those of holonomic fully actuated robots, and it isfeedback linearizable (see Section 2.6.2 for details), and various controls caneasily be designed for τ to make u track any desired trajectory ud(t) [130, 195,242]. As a result, the focus of controlling non-holonomic systems is to designkinematic control u for Kinematic Model 1.12. Once control ud is designedfor Kinematic Model 1.12, control τ can be found using the backsteppingprocedure (see Section 2.6.1 for details).


Upon solving for q and q from (1.24), λ can also be determined. It followsfrom the second equation in (1.22) that

A(q)q + A(q)q = 0.

Pre-multiplying first M−1(q) and then A(q) on both sides of the first equationin (1.22), we know from the above equation that

λ = [A(q)M−1(q)AT (q)]−1

A(q)M−1(q) [N(q, q) − B(q)τ ] − A(q)q

.

1.2.6 Underactuated Systems

Generally, non-holonomic constraints with drift arise naturally from dynam-ics of underactuated mechanical systems. For example, the reduced dynamicmodel of (1.24) is underactuated if the dimension of torque-level input is lessthan the dimension of kinematic input, or simply m < n−k. To illustrate thepoint, consider the simple case that, in (1.24),

n − k = 2, m = 1, GT (q)B(q) =[

0 1]T

, M ′(q)=

[

m′11(q) m′

12(q)m′

21(q) m′22(q)

]

,

and

N ′(q, u)=

[

n′1(q, u)

n′2(q, u)

]

.

That is, the reduced-order dynamic equations are

m′11(q)u1 + m′

12(q)u2 + n′1(q, u) = 0,

m′21(q)u1 + m′

22(q)u2 + n′2(q, u) = τ.

(1.25)

If the underactuated mechanical system is properly designed, controllabilityof Dynamic Sub-system 1.25 and System 1.24 can be ensured (and, if needed,also checked using the conditions presented in Section 2.5). Solving for ui from(1.25) yields

u1 =−m′

12(q)τ − m′22(q)n

′1(q, u) + m′

12(q)n′2(q, u)

m′11(q)m

′22(q) − m′

21(q)m′12(q)

, (1.26)

u2 =m′

11(q)τ + m′21(q)n

′1(q, u) − m′

11(q)n′2(q, u)

m′11(q)m

′22(q) − m′

21(q)m′12(q)

. (1.27)

If u1 were the primary variable to be controlled and ud1 were the desired

trajectory for u1, we would design τ according to (1.26), for instance,

τ =1

m′12(q)

−m′22(q)n

′1(q, u) + m′

12(q)n′2(q, u)

+[u1 − ud1 − ud

1][m′11(q)m

′22(q) − m′

21(q)m′12(q)]

, (1.28)

18 1 Introduction

under whichd

dt[u1 − ud

1] = −[u1 − ud1].

Substituting (1.28) into (1.27) yields a constraint which is in the form of (1.8)and in general non-holonomic and with drift. As will be shown in Section 2.6.2,the above process is input-output feedback linearization (also called partialfeedback linearization in [241]), and the resulting non-holonomic constraintis the internal dynamics. It will be shown in Chapter 2 that the internaldynamics being input-to-state stable is critical for control design and stabilityof the overall system.

If a mechanical system does not have Pfaffian constraints but is under-actuated (i.e., m < n), non-holonomic constraints arises directly from itsEuler-Lagrange Equation 1.16. For example, if n = 2 and m = 1 with

B(q) =[

0 1]T

, Dynamic Equation 1.16 becomes

m11(q)q1 + m12(q)q2 + n1(q, q) = 0,m21(q)q1 + m22(q)q2 + n2(q, q) = τ.

(1.29)

Although differential equations of underactuated systems such as those in(1.29) are of second-order (and one of them is a non-holonomic constraintreferred to as non-holonomic acceleration constraint [181]), there is little dif-ference from those of first-order. For example, (1.25) and (1.29) become es-sentially identical after introducing the state variable u = q, and hence theaforementioned process can be applied to (1.29) and to the class of underac-tuated mechanical systems. More discussions on the class of underactuatedmechanical systems can be found in [216].

1.3 Vehicle Models

In this section, vehicle models and other examples are reviewed. Pfaffian con-straints normally arise from two types of systems: (i) in contact with a certainsurface, a body rolls without side slipping; (ii) conservation of angular mo-mentum of rotational components in a multi-body system. Ground vehiclesbelong to the first type, and aerial and space vehicles are of the second type.

1.3.1 Differential-drive Vehicle

Among all the vehicles, the differential-drive vehicle shown in Fig. 1.9 is thesimplest. If there are more than one pair of wheels, it is assumed for simplic-ity that all the wheels on the same side are of the same size and have thesame angular velocity. Let the center of the vehicle be the guidepoint whose

generalized coordinate is q =[

x y θ]T

, where (x, y) is the 2-D position of the

1.3 Vehicle Models 19

X

Y

x

y

Fig. 1.9. A differential-drive vehicle

vehicle, and θ is the heading angle of its body. For 2-D motion, the Lagrangianof the vehicle is merely the kinetic energy, that is,

L =1

2m(x2 + y2) +

1

2Jθ2, (1.30)

where m is the mass of the vehicle, and J is the inertia of the vehicle withrespect to the vertical axis passing through the guidepoint.

Under the assumption that the vehicle rolls without side slipping, thevehicle does not experience any motion along the direction of the wheel axle,which is described by the following non-holonomic constraint:

x sin θ − y cos θ = 0,

or in matrix form,

0 =[

sin θ − cos θ 0]

⎡

⎣

xy

θ

⎤

⎦

= A(q)q.

As stated in (1.11), the null space of matrix A(q) is the span of column vectorsin matrix

G(q) =

⎡

⎣

cos θ 0sin θ 0

0 1

⎤

⎦ .

Therefore, by (1.12), kinematic model of the differential-drive vehicle is⎡

⎣

xy

θ

⎤

⎦ =

⎡

⎣

cos θsin θ

0

⎤

⎦u1 +

⎡

⎣

001

⎤

⎦u2, (1.31)

where u1 is the driving velocity, and u2 is the steering velocity. Kinematiccontrol variables u1 and u2 are related to physical kinematic variables as

20 1 Introduction

u1 =ρ

2(ωr + ωl), u2 =

ρ

2(ωr − ωl),

where ωl is the angular velocity of left-side wheels, ωr is the angular velocityof right-side wheels, and ρ is the radius of all the wheels.

It follows from (1.30) and (1.19) that the vehicle’s dynamic model is

Mq = AT (q)λ + B(q)τ,

that is,⎡

⎣

m 0 00 m 00 0 J

⎤

⎦

⎡

⎣

xy

θ

⎤

⎦ =

⎡

⎣

sin θ− cos θ

0

⎤

⎦λ +

⎡

⎣

cos θ 0sin θ 0

0 1

⎤

⎦

[

τ1

τ2

]

, (1.32)

where λ is the Lagrange multiplier, τ1 is the driving force, and τ2 is the steeringtorque. It is obvious that

GT (q)B(q) = I2×2, GT (q)MG(q) = 0, GT (q)MG(q) =

[

m 00 I

]

.

Thus, the reduced-order dynamic model in the form of (1.24) becomes

mu1 = τ1, Ju2 = τ2. (1.33)

In summary, the model of the differential-drive vehicle consists of the twocascaded equations of (1.31) and (1.33).

1.3.2 A Car-like Vehicle

A car-like vehicle is shown in Fig. 1.10; its front wheels steer the vehicle whileits rear wheels have a fixed orientation with respect to the body. As shown inthe figure, l is the distance between the midpoints of two wheel axles, and thecenter of the back axle is the guidepoint. The generalized coordinate of theguidepoint is represented by q = [x y θ φ]T , where (x, y) are the 2-D Cartesiancoordinates, θ is the orientation of the body with respect to the x-axis, andφ is the steering angle.

During its normal operation, the vehicle’s wheels roll but do not slip side-ways, which translates into the following motion constraints:

vfx sin(θ + φ) − vf

y cos(θ + φ) = 0,vb

x sin θ − vby cos θ = 0,

(1.34)

where (vfx , vf

y ) and (vbx, vb

y) are the pairs of x-axis and y-axis velocities for thefront and back wheels, respectively. As evident from Fig. 1.10, the coordinatesof the midpoints of the front and back axles are (x + l cos θ, y + l sin θ) and(x, y), respectively. Hence, we have

vbx = x, vb

y = y, vfx = x − lθ sin θ, vf

y = y + lθ cos θ.


X

l

2

1

x

y

Y

Fig. 1.10. A car-like robot

Substituting the above expressions into the equations in (1.34) yields thefollowing non-holonomic constraints in matrix form:

0 =

[

sin(θ + φ) − cos(θ + φ) −l cosφ 0sin θ − cos θ 0 0

]

q= A(q)q.

Accordingly, we can find the corresponding matrix G(q) (as stated for (1.11)and (1.12)) and conclude that the kinematic model of a car-like vehicle is

⎡

⎢

⎢

⎣

xy

θ

φ

⎤

⎥

⎥

⎦

=

⎡

⎢

⎢

⎢

⎣

cos θ 0sin θ 0

1

ltan φ 0

0 1

⎤

⎥

⎥

⎥

⎦

[

u1

u2

]

, (1.35)

where u1 ≥ 0 (or u1 ≤ 0) and u2 are kinematic control inputs which need tobe related back to physical kinematic inputs. It follows from (1.35) that

√

x2 + y2 = u1, φ = u2,

by which the physical meanings of u1 and u2 are the linear velocity andsteering rate of the body, respectively.

For a front-steering and back-driving vehicle, it follows from the physicalmeanings that

u1 = ρω1, u2 = ω2,

where ρ is the radius of the back wheels, ω1 is the angular velocity of the backwheels, and ω2 is the steering rate of the front wheels. Substituting the aboveequation into (1.35) yields the kinematic control model:

22 1 Introduction

⎡

⎢

⎢

⎣

xy

θ

φ

⎤

⎥

⎥

⎦

=

⎡

⎢

⎢

⎢

⎣

ρ cos θ 0ρ sin θ 0ρ

ltan φ 0

0 1

⎤

⎥

⎥

⎥

⎦

[

ω1

ω2

]

, (1.36)

Kinematic Model 1.36 has singularity at the hyperplane of φ = ±π/2, whichcan be avoided (mathematically and in practice) by limiting the range of φ tothe interval (−π/2, π/2).

For a front-steering and front-driving vehicle, we know from their physicalmeanings that

u1 = ρω1 cosφ, u2 = ω2,

where ρ is the radius of the front wheels, ω1 is the angular velocity of the frontwheels, and ω2 is the steering rate of the front wheels. Thus, the correspondingkinematic control model is

⎡

⎢

⎢

⎣

xy

θ

φ

⎤

⎥

⎥

⎦

=

⎡

⎢

⎢

⎣

ρ cos θ cosφρ sin θ cosφ

ρl sinφ

0

⎤

⎥

⎥

⎦

ω1 +

⎡

⎢

⎢

⎣

0001

⎤

⎥

⎥

⎦

ω2,

which is free of any singularity.To derive the dynamic model for car-like vehicles, we know from the La-

grangian being the kinetic energy that

L =1

2mx2 +

1

2my2 +

1

2Jbθ

2 +1

2Jsφ

2,

where m is the mass of the vehicle, Jb is the body’s total rotational inertiaaround the vertical axis, and Js is the inertial of the front-steering mechanism.It follows from (1.19) that the dynamic equation is

Mq = AT (q)λ + B(q)τ,

where

M =

⎡

⎢

⎢

⎣

m 0 0 00 m 0 00 0 Jb 00 0 0 Js

⎤

⎥

⎥

⎦

, B(q) =

⎡

⎢

⎢

⎣

cos θ 0sin θ 0

0 00 1

⎤

⎥

⎥

⎦

,

λ ∈ ℜ2 is the Lagrange multiplier, τ = [τ1, τ2]T , τ1 is the torque acting on the

driving wheels, and τ2 is the steering torque. Following the procedure from(1.22) to (1.24), we obtain the following reduced-order dynamic model:

(

m +Jb

l2tan2 φ

)

u1 =Jb

l2u1u2 tan φ sec2 φ + τ1, Jsu2 = τ2. (1.37)

Note that reduced-order dynamic equations of (1.37) and (1.33) are similaras both have the same form as the second equation in (1.24). In comparison,kinematic models of different types of vehicles are usually distinct. Accord-ingly, we will focus upon kinematic models in the subsequent discussions ofvehicle modeling.


X

Y

x

y 1

22

1

Fig. 1.11. A fire truck

1.3.3 Tractor-trailer Systems

Consider first the fire truck shown in Fig. 1.11; it consists of a tractor-trailerpair that is connected at the middle axle while the first and third axles areallowed to be steered [34]. Let us choose the guidepoint to be the midpoint ofthe second axle (the rear axle of the tractor) and define the generalized coordi-

nate as q =[

x y φ1 θ1 φ2 θ2

]T, where (x, y) is the position of the guidepoint,

φ1 is the steering angle of the front wheels, θ1 is the orientation of the trac-tor, φ2 is the steering angle of the third axle, and θ2 is the orientation of thetrailer. It follows that

⎧

⎨

⎩

xf = x + lf cos θ1, yf = y + lf sin θ1,xm = x, ym = y,xb = x − lb cos θ2, yb = y − lb sin θ2,

(1.38)

where lf is the distance between the midpoints of the front and middle axles,lb is the distance between the midpoints of the middle and rear axles, and(xf , yf), (xm, ym) (xb, yb) are the midpoints of the front, middle and backaxles, respectively. If the vehicle operates without experiencing side slippageof the wheels, the corresponding non-slip constraints are:

⎧

⎨

⎩

xf sin(θ1 + φ1) − yf cos(θ1 + φ1) = 0,xm sin θ1 − ym cos θ1 = 0,xb sin(θ2 + φ2) − yb cos(θ2 + φ2) = 0.

(1.39)

Combining (1.38) and (1.39) yields the non-holonomic constraints in matrixform as A(q)q = 0, where

A(q) =

⎡

⎣

sin(θ1 + φ1) − cos(θ1 + φ1) 0 −lf cosφ1 0 0sin θ1 − cos θ1 0 0 0 0

sin(θ2 + φ2) − cos(θ2 + φ2) 0 0 0 lb cosφ2

⎤

⎦ .

24 1 Introduction

X

Y

x

y

0

n

1d

nd

Fig. 1.12. An n-trailer system

Following the step from (1.11) to (1.12), we obtain the following kinematicmodel for the fire truck:

q =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎣

cos θ1

sin θ1

01lf

tan φ1

0− 1

lbsec φ2 sin(φ2 − θ1 + θ2)

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎦

u1 +

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎣

001000

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎦

u2 +

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎣

000010

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎦

u3, (1.40)

where u1 ≥ 0 is the linear body velocity of the tractor, u2 is the steering rateof the tractor’s front axle, and u3 is the steering rate of the trailer’s axle.

The process of deriving Kinematic Equation 1.40 can be applied to othertractor-trailer systems. For instance, consider the n-trailer system studied in[237] and shown in Fig. 1.12. The system consists of a differential-drive tractor(also referred to as “trailer 0”) pulling a chain of n trailers each of which ishinged to the midpoint of the preceding trailer’s wheel axle. It can be shownanalogously that kinematic model of the n-trailer system is given by

x = vn cos θn

y = vn sin θn

θn = 1dn

vn−1 sin(θn−1 − θn)...

θi = 1di

vi−1 sin(θi−1 − θi), i = n − 1, · · · , 2...

θ1 = 1d1

u1 sin(θ0 − θ1)

θ0 = u2,


X

Y

r

1

2

Fig. 1.13. A planar space robot

where (x, y) is the guidepoint located at the axle midpoint of the last trailer,θj is the orientation angle of the jth trailer (for j = 0, · · · , n), dj is thedistance between the midpoints of the jth and (j − 1)th trailers’ axles, vi isthe tangential velocity of trailer i as defined by

vi =i∏

k=1

cos(θk−1 − θk)u1, i = 1, · · · , n,

u1 is the tangential velocity of the tractor, and u2 is the steering angularvelocity of the tractor.

1.3.4 A Planar Space Robot

Consider planar and frictionless motion of a space robot which, as shown inFig. 1.13, consists of a main body and two small point-mass objects at theends of the rigid and weightless revolute arms. If the center location of themain body is denoted by (x, y), then the positions of the small objects are at(x1, y1) and (x2, y2) respectively, where

x1 = x − r cos θ − l cos(θ − ψ1), y1 = y − r sin θ − l sin(θ − ψ1),x2 = x + r cos θ + l cos(θ − ψ2), y2 = y + r sin θ + l sin(θ − ψ2),

θ is the orientation of the main body, ψ1 and ψ2 are the angles of the armswith respect to the main body, and l is the length of the arms, and r is thedistance from the revolute joints to the center of the main body.

It follows that the Lagrangian (i.e., kinematic energy) of the system isgiven by

L =1

2m0(x

2 + y2) +1

2Jθ2 +

1

2m(x2

1 + y21) +

1

2m(x2

2 + y22)

=1

2[ x y ψ1 ψ2 θ ]M [ x y ψ1 ψ2 θ ]T ,

26 1 Introduction

where m0 is the mass of the main body, J is the inertia of the main body, mis the mass of the two small objects, and inertia matrix M = [Mij ] ∈ ℜ5×5 issymmetric and has the following entries:

M11 = M22 = m0 + 2m, M12 = 0,M13 = −ml sin(θ − ψ1), M14 = ml sin(θ − ψ2), M15 = 0,M23 = ml cos(θ − ψ1), M24 = −ml cos(θ − ψ2), M25 = 0,M33 = ml2, M34 = 0, M35 = −ml2 − mlr cosψ1,M44 = ml2, M45 = −ml2 − mlr cosψ2,M55 = J + 2mr2 + 2ml2 + 2mlr cosψ1 + 2mlr cosψ2.

Then, the dynamic equation of the space robot is given by (1.16).If the robot is free floating in the sense that it is not subject to any external

force/torque and that x(t0) = y(t0) = 0, x(t) = y(t) = 0 for all t > t0, andhence the Lagrangian becomes independent of θ. It follows from (1.16) that

∂L

∂θ= M35ψ1 + M45ψ2 + M55θ

must be a constant representing conservation of the angular momentum. Ifthe robot has zero initial angular momentum, its conservation is in essencedescribed by the non-holonomic constraint that

q =

⎡

⎢

⎣

10

−M35

M55

⎤

⎥

⎦u1 +

⎡

⎢

⎣

01

−M45

M55

⎤

⎥

⎦u2,

where q = [ψ1, ψ2, θ]T , u1 = ψ1, and u2 = ψ2.

For orbital and attitude maneuvers of satellites and space vehicles, momen-tum precession and adjustment must be taken into account. Details on under-lying principles of spacecraft dynamics and control can be found in [107, 272].

1.3.5 Newton’s Model of Rigid-body Motion

In general, equations of 3-D motion for a rigid-body can be derived usingNewton’s principle as follows. First, establish the fixed frame Xf , Yf , Zf(or earth inertia frame), where Zf is the vertical axis, and axes Xf , Yf , Zfare perpendicular and satisfy the right-hand rule. The coordinates in the fixedframe are position components of xf , yf , zf and Euler angles of θ, φ, ψ,and their rates are defined by xf , yf , zf and θ, φ, ψ, respectively.Second, establish an appropriate body frame Xb, Yb, Zb which satisfies theright-hand rule. The coordinates in the body frame are position componentsof xb, yb, zb, while the linear velocities are denoted by u, v, w (orequivalently xb, yb, zb) and angular velocities are denoted by p, q, r (orωx, ωy, ωz). There is no need to determine orientation angles in the body


frame as they are specified by Euler angles (with respect to the fixed frame,by convention).

Euler angles of roll φ, pitch θ, and yaw ψ are defined by the roll-pitch-yawconvention, that is, orientation of the fixed frame is brought to that of the bodyframe through the following right-handed rotations in the given sequence: (i)rotate Xf , Yf , Zf about the Zf axis through yaw angle ψ to generate frameX1, Y1, Z1; (ii) rotate X1, Y1, Z1 about the Y1 axis through pitch angleθ to generate frame X2, Y2, Z2; (iii) rotate X2, Y2, Z2 about the X2 axisthrough roll angle ψ to body frame Xb, Yb, Zb. That is, the 3-D rotationmatrix from the fixed frame to the body frame is

Rfb =

⎡

⎣

1 0 00 cosφ sin φ0 − sinφ cosφ

⎤

⎦×

⎡

⎣

cos θ 0 − sin θ0 1 0

sin θ 0 cos θ

⎤

⎦×

⎡

⎣

cosψ sin ψ 0− sinψ cosψ 0

0 0 1

⎤

⎦ .

Conversely, the 3-D rotation matrix from the body frame back to the bodyframe is defined by

R = Rz(ψ)Ry(θ)Rx(φ),

that is,

R =

⎡

⎣

cos θ cosψ sin θ sin φ cosψ − cosφ sin ψ sin θ cosφ cosψ + sin φ sin ψcos θ sin ψ sin θ sin φ sin ψ + cosφ cosψ sin θ cosφ sin ψ − sinφ cos ψ− sin θ sin φ cos θ cosφ cos θ

⎤

⎦ ,

(1.41)where Rz(ψ), Ry(θ), Rx(φ) are the elementary rotation matrices about z, y, xaxes, respectively, and they are given by

Rz(ψ) =

⎡

⎣

cosψ − sinψ 0sin ψ cosψ 0

0 0 1

⎤

⎦ , Ry(θ) =

⎡

⎣

cos θ 0 sin θ0 1 0

− sin θ 0 cos θ

⎤

⎦ ,

and

Rx(φ) =

⎡

⎣

1 0 00 cosφ − sinφ0 sin φ cosφ

⎤

⎦ .

It follows that linear velocities in the body frame are transformed to thosein the inertia frame according to

[

xf yf zf

]T= R

[

u v w]T

. (1.42)

On the other hand, it follows that RT R = I and hence

RT R + RT R = 0.

Defining matrix

ω(t)= RT R, (1.43)

28 1 Introduction

we know that ω(t) is skew-symmetric and that, by direction computation

ω(t) =

⎡

⎣

0 −r qr 0 −p−q p 0

⎤

⎦ . (1.44)

Pre-multiplying both sides of (1.43) by R yields

R = Rω(t) (1.45)

It is straightforward to verify that (1.45) is equivalent to the following trans-formation from angular velocities in the body frame to those in the inertiaframe:

⎡

⎣

φ

θ

ψ

⎤

⎦ =

⎡

⎣

1 sin φ tan θ cosφ tan θ0 cosφ − sin φ0 sin φ sec θ cosφ sec θ

⎤

⎦

⎡

⎣

pqr

⎤

⎦ . (1.46)

Equations 1.42 and 1.46 constitute the kinematic model of rigid-body motion.Note that, given any rotation matrix R in (1.41), Euler angles can always

be solved, but not globally. For instance, at θ = −π/2, matrix R reduces to

R =

⎡

⎣

0 − sin(φ + ψ) − cos(φ + ψ)0 cos(φ + ψ) − sin(φ + ψ)1 0 0

⎤

⎦ ,

and hence there are an infinite number of solutions φ and ψ for such a matrixR. The lack of existence of global and smooth solutions to the inverse problemof determining the Euler angles from the rotation is the singularity issue ofthe rotation space. The singularity can be overcome by using four-parameterquaternions which generalize complex numbers and provide a global parame-terization of the rotation space.

In the derivations of Lagrange-Euler equation in Section 1.2.3, it is as-sumed implicitly that the configuration space (i.e., both position and rotationspaces) can be parameterized by generalized coordinate q ∈ ℜn. It is due tothe singularity issue of Euler angles that Lagrange-Euler Equation 1.16 can-not be used to determine global dynamics of 3-D rigid-body motion. In whatfollows, a global characterization of the dynamics of one rigid-body subject toexternal forces and torques is derived using the Newton-Euler method. TheNewton-Euler method can also be applied to a multi-body mechanical sys-tem, but it needs to be executed recursively by accounting for all interactiveforces/torques within the system [80].

Given angular velocity vector ω =[

p q r]T

in the body frame, the angularvelocity vector ωf in the fixed frame is given by ωf = Rω, and the angularmomentum in the fixed frame is given by Jf (t)ωf , where J is the inertia ofthe rigid-body, and Jf (t) = RJRT is the instantaneous inertia with respectto the fixed frame. It follows from Newton’s law that


d

dt[Jf (t)ωf ] = τf ,

where τ ∈ ℜ3 is the external torque vector with respect to the body frame, andτf = Rτ is external torque vector with respect to the fixed frame. Invokingthe property of RRT = RT R = I, differentiating the left hand side of theabove equation, and utilizing (1.45) yield

τf =d

dt[RJω]

= RJω + RJω

= RJω + RωJω,

or equivalentlyJω + ωJω = τ. (1.47)

On the other hand, given linear velocity vector Vb =[

u v w]T

in the bodyframe, the linear velocity Vf in the fixed frame is given by Vf = RVb, and thelinear momentum in the fixed frame is given by mVf . It follows from Newton’slaw that

d

dt[mVf ] = Ff ,

where F ∈ ℜ3 is the external force vector with respect to the body frame, andFf = RF is external force vector with respect to the fixed frame. Differenti-ating the left hand side of the above equation and substituting (1.45) into theexpression yield

mVb + ωmVb = F. (1.48)

Equations 1.47 and 1.48 constitute the dynamic model of 3-D rigid-body mo-tion and are called Newton-Euler equations.

In the subsequent subsections, Kinematic Equations 1.42 and 1.46 andDynamic Equations 1.47 and 1.48 are used to model several types of 3-D or2-D vehicles.

1.3.6 Underwater Vehicle and Surface Vessel

In addition to rigid-body motion equations of (1.47) and (1.48), modeling ofunderwater vehicles and surface vessels may involve their operational modesand wave climate modeling. In what follows, two simplified modes of operationare considered, and the corresponding models are presented.

Cruising of an Underwater Vehicle

Consider an underwater vehicle cruising at a constant linear speed. Withoutloss of any generality, choose the body frame xb, yb, zb such that Vb =[vx 0 0]T ∈ ℜ3 is the velocity in the body frame, and let [x y z]T ∈ ℜ3 be the

30 1 Introduction

X

YBxBy

Fig. 1.14. A surface vessel

position of the center of mass of the vehicle in the inertia frame. It follows from(1.42) and (1.46) that kinematic equations of the vehicle reduce to [57, 171]

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎣

xyz

φ

θ

ψ

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎦

=

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎣

cos θ cosψ 0 0 0sin ψ cos θ 0 0 0− sin θ 0 0 0

0 1 sinφ tan θ cosφ tan θ0 0 cosφ − sinφ0 0 sin φ sec θ cosφ sec θ

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎦

⎡

⎢

⎢

⎣

vx

ωx

ωy

ωz

⎤

⎥

⎥

⎦

= G(q)

⎡

⎢

⎢

⎣

vx

ωx

ωy

ωz

⎤

⎥

⎥

⎦

,

where θ, φ, ψ are the Euler angles, and ωx, ωy, ωz are the angular velocitiesalong x, y and z axes in the body frame, respectively. The above model is non-holonomic due to zero linear velocity along the y and z axes in the body frame.It follows from the expression of G(q) that the two non-holonomic constraintscan be expressed as A(q)q = 0, where matrix A(q) is given by

[

cosψ sin θ sin φ − sin ψ cosφ sin ψ sin θ sin φ + cosψ cosφ cos θ sin φ 0 0 0sin ψ sin θ cosφ − sin ψ sin φ sin ψ sin θ cosφ − cosψ sin φ cos θ cosφ 0 0 0

]

.

A Surface Vessel

Consider the planar motion of an underactuated surface vessel shown in Fig.1.14. It is controlled by two independent propellers which generate the forcein surge and the torque in yaw. It follows from the relationship between thebody and inertia frames that vessel’s kinematic model is [215, 271]:

⎡

⎣

xy

ψ

⎤

⎦ =

⎡

⎣

cosψ − sinψ 0sinψ cosψ 0

0 0 1

⎤

⎦

⎡

⎣

vx

vy

ωz

⎤

⎦ , (1.49)

where (x, y) denote the position of the center of mass in the inertia frame(X−Y ), ψ is the yaw orientation, and (vx, vy) and ωz are the linear velocities


and angular velocity in the body frame. By ignoring wind and wave forces, onecan consolidate (1.47) and (1.48) and obtain the following simplified dynamicmodel [68, 215]:

⎧

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎩

vx =m22

m11vyωz − d11

m11vx +

1

m11τ1,

vy = −m11

m22vxωz −

d22

m22vy,

ωz =m11 − m22

m33vxvy − d33

m33ωz +

1

m33τ2,

(1.50)

where positive constants mii are determined by vessel’s inertia and its addedmass effects, positive constants dii are due to hydrodynamic damping, andτ1 and τ2 are the external force and torque generated by the two propellers.Model 1.50 is non-holonomic due to the non-integrable constraint of

m22vy = −m11vxωz − d22vy.

In the inertia frame, the above constraint can be expressed as

m22(x sin ψ−y cosψ)+(m22−m11)ψ(x cosψ+y sin ψ)+d22(x sin ψ−y cosψ) = 0.

1.3.7 Aerial Vehicles

Depending on the type and operational mode of an aerial vehicle, its dynamicmodel can be modeled by properly consolidating or elaborating KinematicEquations 1.42 and 1.46 and Dynamic Equations 1.47 and 1.48.

Fixed-wing Aircraft

In addition to Dynamic Equations 1.47 and 1.48, dynamic modeling of a fixed-wing aircraft typically involves establishment of the so-called flow/stabilityframe (whose coordinates are angle of attack α and angle of sideslip β). In thestability frame, aerodynamic coefficients can be calculated, and aerodynamicforces (lift, drag and side forces) and aerodynamic moments (in pitch, roll andyaw) can be determined with respect to the body frame. Should aerodynamiccoefficients depend upon α or β or both, the resulting equations obtained bysubstituting the expressions of aerodynamic and thrust forces/torques into(1.47) and (1.48) would have to be rearranged as accelerations would appearin both sides of the equations. This process of detailed modeling is quiteinvolved, and readers are referred to [174, 245, 265].

32 1 Introduction

D

L

T

V

Fig. 1.15. An unmanned aerial vehicle

For the fixed-wing aerial vehicle shown in Fig. 1.15, its simplified dynamicmodel is given by [265]

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

x = V cos γ cosφ,y = V cos γ sinφ,

h = V sin γ,

V =T cosα − D

m− g sin γ,

γ =T sin α + L

mVcos δ − g

Vcos γ,

φ =T sin α + L

mV

sin δ

cos γ,

(1.51)

where x is the down-range displacement, y is the cross-range displacement, h isthe altitude, V is the ground speed and is assumed to be equal to the airspeed,T is the aircraft engine thrust, α is the angle of attack, D is the aerodynamicdrag, m is the aircraft mass, g is the gravity constant, γ is the flight pathangle, L is the lift force, δ is the bank angle, and φ is the heading angle. Inthe case that altitude variations are not considered, Model 1.51 reduces to thefollowing planar aerial vehicle model:

⎧

⎨

⎩

x = u1 cosφy = u1 sinφ

φ = u2,


where

u1= V cos γ, and u2

=

(T sin α + L) sin δ

mV cos γ.

Note that the above simplified planar model is identical to Model 1.31 of adifferential-drive ground vehicle.

By using tilt-rotors or by utilizing directed jet thrust, fixed-wing aircraftcan lift off vertically. These aircraft are called vertical take-off and landing(VTOL) aircraft. If only the planar motion is considered, the dynamic equa-tions can be simplified to [223]

⎧

⎨

⎩

x = −u1 sin θ + εu2 cos θy = u1 cos θ + εu2 sin θ − g

θ = u2,

where (x, y, θ) denote the position and orientation of the center of mass of theaircraft, g is the gravitational constant, control inputs u1 and u2 are the thrust(directed downwards) and rolling moment of the jets, respectively, and ε > 0is the small coefficient representing the coupling between the rolling momentand lateral acceleration of the aircraft. In this case, the VTOL aircraft is anunderactuated system with three degrees of freedom but two control inputs.

Helicopter

Parallel to the derivations of Dynamic Equations 1.47 and 1.48, one can showin a straightforward manner as done in [223] that the helicopter model is givenby

⎧

⎨

⎩

pf = vf ,mvf = RF,Jω = −ωJω + τ,

where pf ∈ ℜ3 and vf ∈ ℜ3 denote the position and velocity of the center ofmass of the helicopter in the inertia frame, respectively; ω ∈ ℜ3, F ∈ ℜ3, andτ ∈ ℜ3 are the angular velocity, the force and the torque in the body frame,respectively; m and J are the mass and inertia in the body frame, and R andω are defined in (1.41) and (1.44), respectively.

Autonomous Munitions

Smart munitions range from ballistic missiles to self-guided bullets. For thesemunitions, basic equations of motion remain in the form of (1.47) and (1.48).Details on missile modeling, guidance and control can be found in [230].

34 1 Introduction

1.3.8 Other Models

Operation of a vehicle system may involve other entities in addition to physicalvehicles themselves. Those entities, real or virtual, can be described by othermodels such as the following l-integrator model:

zj = zj+1, j = 1, · · · , l − 1; zl = v, ψ = z1, (1.52)

where zj ∈ ℜm are the state sub-vectors, ψ ∈ ℜm is the output, and v ∈ ℜm

is the input. For example, if l = 1, Model 1.52 reduces to the single integratormodel of ψ = v which has been used to characterize dynamics of genericagents. For l = 2, Model 1.52 becomes the double integrator model which isoften used as the model of life forms.

1.4 Control of Heterogeneous Vehicles

Consider a team of q heterogeneous vehicles. Without loss of any generality,the model of the ith vehicle can be expressed as

φi = Φi(φi, vi), ψi = hi(φi), (1.53)

where i ∈ 1, · · · , q, φi(t) ∈ ℜni is the state, and ψi(t) ∈ ℜm is the outputin the configuration space, and vi(t) ∈ ℜm is the control input. As shown inSection 1.3, Model 1.53 often contains non-holonomic constraints. The teamcould interact with remote and hands-off operator(s) represented by virtualvehicle(s). In order to make the vehicles be cooperative, pliable and robust,control must be designed to achieve cooperative behavior(s) while meeting aset of constraints. Figure 1.16 shows a hierarchical control structure for anautonomous vehicle system.

While the hierarchy in Fig. 1.16 can have many variations, the key is itsmulti-level control and autonomy:

Vehicle-level autonomy: navigation and control algorithms are designed andimplemented (as the bottom level in Fig. 1.16) such that each of thevehicles is pliable to its constraints, is robust to environmental changes,and is also capable of best following any given command signal.

Team-level autonomy: cooperative control algorithm is designed and imple-mented (as the second level in Fig. 1.16) to account for an intermittentsensing and communication network and to achieve cooperative behav-ior(s). Through the network and by the means of a virtual vehicle, anoperator can adjust the status of cooperative behavior of the vehicles.

Mission-level autonomy and intelligence: high-level tactical decisions are en-abled through human-machine interaction and automated by a multi-objective decision-making model with online learning capabilities.

1.4 Control of Heterogeneous Vehicles 35

Mission and Global Planner (offline)

Multi-Objective Decision Making

Non-Regret and Bayesian Learning

Tactical Strategies

Teaming, Resource Allocation,

Coverage, and Waypoint Selection

Networked Control

Cooperation, Formation and Other

Behavior

Real-time Trajectory

planning

Closed-loop Vehicle

Tracking Control

Fig. 1.16. Hierarchical control structure for autonomous vehicle system

To achieve motion autonomy for each vehicle, the navigation and con-trol problems need to be solved. The navigation problem is to find a desiredtrajectory ψd

i and its open-loop steering control vdi . Typically, the trajectory

ψdi should satisfy the boundary conditions and motion constraints (such as

non-holonomic constraints in the model as well as geometrical constraints im-posed by the environment), and it needs to be updated. To find ψd

i and vdi in

real-time, an online trajectory planning algorithm (such as the one in Section3.2) can be used. Then, the control problem is to find a closed-loop feedbackcontrol of form

vi = vi(ψdi , φi, v

di ) (1.54)

such that the desired trajectory is followed asymptotically as qi → qdi . This

will be pursued in Section 3.3.1. Alternatively, the navigation and controlproblems can be combined, in which case qd

i is merely a desired (moving)target location and the control needs to be improved to be

36 1 Introduction

vi = v′i(ψdi , φi, ri), (1.55)

where ri is the reactive control capable of making the vehicle conform to thechanges of the environment (whenever possible). Comparison of Controls 1.54and 1.55 will be made in Section 3.4.2.

To achieve team autonomy, a cooperative control should be designed interms of the available feedback information as

ui = ui(t, si1(t)ψ1, · · · , siq(t)ψq), (1.56)

where sij(t) are binary time functions, sii ≡ 1; sij(t) = 1 if ψj(t) (or itsequivalence) is known to the ith vehicle at time t, and sij = 0 otherwise. Sinceinformation flow over the network changes over time, Cooperative Control 1.56must react accordingly but ensure group cooperative behavior(s), which is themajor issue in analysis and synthesis. Besides observing the group behaviorψg, each vehicle can also exhibit an individual behavior ψd

i within the group.Then, cooperative control can be implemented on top of vehicle-level controlby embedding ui into vi and by modifying (1.55) as

vi = v′′i (ψdi , φi, ri + ui). (1.57)

Control 1.57 is to ensure ψi → ψdi + ψg, which is the subject of Chapters 5

and 6. In contrast, it is often too difficult to integrate cooperative control intoFeedback Control 1.54, since changing the desired trajectory induces largetransient responses and since Control 1.54 is a purely tracking control and isnot pliable or robust by itself.

Although both cooperative control and navigation control are implementedon the same vehicle, the hierarchy is physically necessary: the navigation con-trol is a self-feedback control required to make any specific vehicle follow anymotion command it received, and the cooperative control is needed to syn-thesize a motion command based on the information available from all thesensors/receivers on a vehicle. From the point view of analysis and design, thehierarchy is also necessary since, by separating the two, canonical forms canbe developed so that heterogeneous vehicles can be handled systematically.

Finally, successful operation of autonomous vehicles requires that certaintactical decisions be made real-time and automatically by each of the au-tonomous vehicles. Often, a decision must be made according to a number offactors which may or may not agree with each other. In [92], a decision-makingmodel based on no-regret is mathematically formulated to describe qualita-tively the possible courses of action, to measure the outcomes, to quantifythe success likelihood, and to enable the optimal multi-objective decision-making. Using the Bayesian formula, the decision-making model is capable ofonline learning through which the weightings are updated to improve decision-making over time. In addition to deriving tactical decisions, the decision-making model can also be used to choose adequate way-points for the trajec-tory planning algorithm, to determine fidelity of obstacle/velocity detection

1.5 Notes and Summary 37

from noisy sensor feedbacks, to adjust the formation of a multi-vehicle team,etc. In these applications, the decision-making model increases the level of au-tonomy and improves the system performance. Nonetheless, decision-makingmodels and high-level autonomy/learning are beyond the scope of this bookand hence not discussed further.

1.5 Notes and Summary

Cooperative control of autonomous vehicles is the primary application con-sidered in the book. Most vehicles and underactuated mechanical systemsare non-holonomic. Holonomic systems are the systems subject to constraintsthat limit their possible configurations, and non-holonomic systems are thosewith non-integrable constraints on their velocities. The terminology of holo-nomic and non-holonomic systems can be traced back to [89], and the wordholonomic (or holonomous) in Greek contains the words meaning integral. In-depth analysis of non-holonomic systems as well as other interesting examples(such as a unicycle with rotor, a skate on an inclined plane, a Chaplygin sleigh,a roller racer, a car towing several trailers, a rattleback, a satellite or space-craft with momentum wheels, Toda lattice, and hopping robot) can be foundin texts [23, 79, 145, 168, 169, 238]. In this chapter, analysis of holonomicand non-holonomic systems are reviewed, and common vehicles are modeled.In Chapter 3, a canonical form is developed for kinematic models of non-holonomic systems, and the kinematic control problem of a non-holonomicsystem is solved. By physical nature, the kinematic and dynamic models of avehicle are a set of cascaded equations. It will be shown in Chapter 2 that,if the vehicle can be kinematically controlled, its dynamic control can easilydesigned using the backstepping design, a conceptually intuitive, mathemati-cally simple and physically rooted technique.

Cooperative control of dynamical systems is the main subject of this book.Even for the simplest entities with no dynamics, analysis and synthesis ofcooperative behaviors are quite involved because the interactions among theentities are time-varying and may not be known a priori. This is illustratedby the sample systems in Section 1.1.2. To investigate cooperative systems,useful results on analysis and design of control systems are reviewed firstin Chapter 2. Later in Chapters 4, 5 and 6, these results are extended tothe systems equipped with an intermittent sensing/communication network,and tools and techniques of analyzing and designing linear and non-linearcooperative systems are developed. Combined with the design in Chapter 3,cooperative systems can be synthesized and constructed for heterogeneousentities that may have high-order dynamics and may be non-linear and non-holonomic.

2

Preliminaries on Systems Theory

In this chapter, basic concepts and analysis tools in systems theory are sum-marized. We begin with matrix algebra and matrix norms, the standard in-struments for qualitatively and quantitatively analyzing linear time-invariantsystems and their properties. While analysis of time-varying and non-linearsystems often requires advanced tools and particular results, the simple math-ematical concept of a contraction mapping can be used to describe such qual-itative characteristics as stability and convergence. Accordingly, the contrac-tion mapping theorem is introduced, and it, together with the Barbalat lemmaand the comparison theorem, will facilitate the development of analysis toolsfor cooperative control.

Lyapunov direct method is the universal approach for analyzing generaldynamical systems and their stability. Search for a successful Lyapunov func-tion is the key, and relevant results on linear systems, non-linear systems, andswitching systems are outlined. Standard results on controllability as well ascontrol design methodologies are also reviewed.

2.1 Matrix Algebra

Let ℜ and C be the set of real and complex numbers, respectively. Then, ℜn

represents the set of n−tuples for which all components belong to ℜ, and ℜn×m

is the set of n-by-m matrices over ℜ. Let ℜ+= [0, +∞) and ℜ+/0

= (0, +∞),

and let ℜn+ denote the set of all n-tuples whose components belong to ℜ+.

Definition 2.1. For any matrix A ∈ ℜn×n, its eigenvalues λi are the roots ofthe characteristic equation of |λI − A| = 0, its eigenvectors are the non-zerovectors si satisfying the equation Asi = λisi. The set of all its eigenvaluesλ ∈ C is called spectrum of A and is denoted by σ(A), and its spectral radius

is defined by ρ(A)= max|λ| : λ ∈ σ(A).

40 2 Preliminaries on Systems Theory

Set X is said to be a linear vector space if x, y, z ∈ X implies x+y = y+x ∈X , x+(y+z) = (x+y)+z ∈ X , −x ∈ X with x+(−x) = 0 ∈ X , αx ∈ X , andα(x+y) = αx+αy, where α ∈ ℜ is arbitrary. Clearly, ℜn×m is a linear vectorspace. Operations of addition, subtraction, scalar multiplication, and matrixmultiplication are standard in matrix algebra. The so-called Kronecker productis defined as D⊗E = [dijE]. Among the properties of Kronecker product are:if matrix dimensions are compatible,

A ⊗ (B + C) = A ⊗ B + A ⊗ C, (A ⊗ B)T = (AT ⊗ BT ),

(A ⊗ B)−1 = (A−1 ⊗ B−1), and (A ⊗ B)(C ⊗ D) = ((AC) ⊗ (BD)).

Definition 2.2. A set of vectors vi ∈ X for i = 1, · · · , l is said to be linearlyindependent if the equation

α1v1 + · · · + αlvl = 0

has only the trivial solution α1 = · · · = αl = 0. If there is a non-trivialsolution αi to the above equation, say αk = 0, the set of vector is said to belinear dependent, in which case one of the vectors can be expressed as a linearcombination of the rest, i.e.,

vk =

n∑

j=1,j =k

−αj

αkvj .

Set X ⊂ ℜn is said to be of rank p if X is composed of exactly p linearlyindependent vectors.

Matrix S ∈ ℜn×n is invertible and its inverse S−1 exists as SS−1 =S−1S = I if and only if its rows (columns) are linearly independent. Formatrix A ∈ ℜn×n, there are always exactly n eigenvalues, either complex orreal, but there may not be n linearly independent eigenvectors. A square ma-trix whose number of linearly independent eigenvectors is less than its order issaid to be defective. A sufficient condition under which there are n linearly in-dependent eigenvectors is that there are n distinct eigenvalues. Matrix productS−1AS is called a similarity transformation on matrix A, and eigenvalues areinvariant under such a transformation. Matrix A is said to be diagonalizable asS−1AS = Λ if and only if matrix A has n linearly independent eigenvectors si,where S =

[

s1 s2 · · · sn

]

and Λ = diagλ1, · · · , λn. Transformation matrixS is said to be a unitary matrix if S−1 = ST . It is known that any symmetricmatrix A can be diagonalized under a similarity transformation of unitarymatrix S. A very special unitary matrix is the permutation matrix which isobtained by permuting the rows of an n×n identity matrix according to somenew ordering of the numbers 1 to n. Therefore, for any permutation matrixP , every row and column contains precisely a single 1 with 0s everywhere else,its determinant is always ±1, and P−1 = PT as a permutation is reversed bypermutating according to the new (i.e. transposed) order.

2.1 Matrix Algebra 41

Diagonalization fails only if (but not necessarily if) there are repeatedeigenvalues. For an nth-order matrix whose eigenvalue λi repeats k times, thenull space of matrix (A − λiI) determines linearly independent eigenvectorssi associated with λi. Specifically, the linearly independent eigenvectors ofmatrix (A − λiI) span its null space, and their number is the dimension ofthe null space and can assume any integer value between 1 and k. Obviously,defectiveness of matrix A occurs when the dimension of the null space is lessthan k. If matrix A ∈ ℜn×n is defective, its set of eigenvectors is incompletein the sense that there is no similarity transformation (or an invertible eigen-vector matrix) that diagonalizes A. Hence, the best one can do is to choosea similarity transformation matrix S such that S−1AS is as nearly diagonalas possible. A standardized matrix defined to be the closest to diagonal isthe so-called Jordan canonical form. In (2.1) below, order ni is determined by(A − λiI)nis′i = 0 but (A − λiI)ni−1s′i = 0. The corresponding transforma-tion matrix S consists of the eigenvector(s) si = (A − λiI)ni−1s′i as well asgeneralized eigenvectors s′i up to (A − λiI)ni−2s′i.

Definition 2.3. Matrix J is said to be in the Jordan canonical form if itis block diagonal where the diagonal blocks, the so-called Jordan blocks Ji,contain identical eigenvalues on the diagonal, 1 on the super-diagonal, andzero everywhere else. That is,

J =

⎡

⎢

⎣

J1

. . .

Jl

⎤

⎥

⎦, and Ji =

⎡

⎢

⎢

⎢

⎢

⎣

λi 1. . .

. . .

. . . 1λi

⎤

⎥

⎥

⎥

⎥

⎦

∈ ℜni×ni , (2.1)

where λi may be identical to λj for some i = j (that is, repeated eigenvaluemay appear in different Jordan blocks, but eigenvalues are all the same in anygiven Jordan block), and n1 + · · ·+ nl = n. The number of Jordan blocks withthe same eigenvalue λi is called algebraical multiplicity of λi. The order ni ofJordan block Ji is called geometrical multiplicity.

A generalization of distance or length in linear vector space X , called normand denoted by ‖ · ‖, can be defined as follows: for any x, y ∈ X and α ∈ ℜ,‖x‖ ≥ 0, ‖x‖ = 0 if and only if x = 0, ‖αx‖ = |α|·‖x‖, and ‖x+y‖ ≤ ‖x‖+‖y‖.That is, a norm is a positive definite function (see Section 2.3.1) that satisfiesthe triangular inequality and is linear with respect to a real, positive constantmultiplier. In ℜn, the vector p-norm (p ≥ 1) and ∞-norm are defined by

‖x‖p =

[

n∑

i=1

|xi|p]

1p

and ‖x‖∞ = max1≤i≤n

|xi|,

respectively. The most commonly used are 1-norm (p = 1), Euclidean norm(p = 2) or 2-norm, and ∞-norm. All the vector norms are compatible withthemselves as, for all x, y ∈ ℜn,


‖xT y‖p ≤ ‖x‖p‖y‖p,

and they are also equivalent as, for any p, q > 1, there exist constants c1 andc2 such that

c1‖x‖p ≤ ‖x‖q ≤ c2‖x‖p.

The p-norm can also be defined in an infinite dimensional linear vector space.For an infinite sequence of scalars xi : i ∈ ℵ = x1, x2, · · ·, p-norm (p ≥ 1)and ∞-norm are defined by

‖x‖p =

[ ∞∑

i=1

|xi|p]

1p

and ‖x‖∞ = supi

|xi|,

respectively. A sequence xi : i ∈ ℵ is said to belong to lp-space if ‖x‖p < ∞and to l∞-space if ‖x‖∞ < ∞.

In linear vector space X ⊂ ℜn, matrix A ∈ ℜn×n in linear algebraicequation of y = Ax represents a linear transformation, and its induced matrixnorms are defined as

‖A‖q= max

‖x‖q =0

‖y‖q

‖x‖q= max

‖x‖q=1‖y‖q = max

‖x‖q=1‖Ax‖q,

where q ≥ 1 is a positive real number including infinity. It follows that ρ(A) ≤‖A‖ for any induced matrix norm. According to the above definition, explicitsolutions can be found for several induced matrix norms, in particular,

‖A‖1 = max1≤j≤n

n∑

i=1

|aij |, ‖A‖2 =√

λmax(AT A), ‖A‖∞ = ‖AT ‖1,

where aij denotes the (i, j)th element of A, and λmax(E) represents the oper-ation of finding the maximum eigenvalue of E.

If a vector or a matrix is time-varying, the aforementioned vector normor induced matrix norm can be applied pointwise at any instant of time, andhence the resulting norm value is time-varying. In this case, a functional normdefined below can be applied on top of the pointwise norm and over time: iff : ℜ+ → ℜ is a Lebesgue measurable time function, then p-norm (1 ≤ p < ∞)and ∞-norm of f(·) are

‖f‖p=

(∫ ∞

t0

|f(t)|pdt

)1p

, and ‖f‖∞ = sup

t0≤t≤∞|f(t)|,

respectively. Function f(t) is said to belong to Lp-space if ‖f(t)‖p < ∞ andto L∞-space if ‖f(t)‖∞ < ∞. Many results on stability analysis and in thischapter can be interpreted using the Lp-space or L∞-space. For instance,any continuous and unbounded signal does not belong to either Lp-space orL∞-space; and all uniformly bounded functions belong to L∞-space. In fact,boundedness is always required to establish a stability result.

2.2 Useful Theorems and Lemma 43

In a normed linear vector space, sequence xk : k ∈ ℵ is said to be aCauchy sequence if, as k, m → ∞, ‖xk − xm‖ → 0. It is known and alsoeasy to show that a convergent sequence must be a Cauchy sequence butnot every Cauchy sequence is convergent. Banach space is a complete normedlinear vector space in which every Cauchy sequence converges therein. Amongthe well known Banach spaces are ℜn with norm ‖ · ‖p with 1 ≤ p ≤ ∞, andC[a, b] (all continuous-time functions over interval [a, b]) with functional normmaxt∈[a,b] ‖ · ‖p.

2.2 Useful Theorems and Lemma

In this section, several mathematical theorems and lemma in systems andcontrol theory are reviewed since they are simple yet very useful.

2.2.1 Contraction Mapping Theorem

The following theorem, the so-called contraction mapping theorem, is a specialfixed-point theorem. It has the nice features that both existence and unique-ness of solution are ensured and that mapping T has its gain in norm lessthan 1 and hence is contractional. Its proof is also elementary as, by a succes-sive application of mapping T , a convergent power series is obtained. Withoutλ < 1, Inequality 2.2 is referred to as the Lipschitz condition.

Theorem 2.4. Let S be a closed sub-set of a Banach space X with norm ‖ · ‖and let T be a mapping that maps S into S. If there exists constant 0 ≤ λ < 1such that, for all x, y ∈ S,

‖T (x) − T (y)‖ ≤ λ‖x − y‖, (2.2)

then solution x∗ to equation x = T (x) exists and is a unique fixed-point in S.

Proof: Consider sequence xk+1 = T (xk), k ∈ ℵ. It follows that

‖xk+l − xk‖ ≤l∑

j=1

‖xk+j − xk+j−1‖

=

l−1∑

j=0

‖T (xk+j) − T (xk+j−1)‖

≤l−1∑

j=0

λ‖xk+j − xk+j−1‖

≤l−1∑

j=0

λk+j−1‖x2 − x1‖

≤ λk−1

1 − λ‖x2 − x1‖,


by which the sequence is a Cauchy sequence. By definition of X , xk → x∗ forsome x∗ and hence x∗ is a fixed-point since

‖T (x∗) − x∗‖ ≤ ‖T (x∗) − T (xk)‖ + ‖xk+1 − x∗‖≤ λ‖x∗ − xk‖ + ‖xk+1 − x∗‖ → 0 as k → ∞.

Uniqueness is obvious from the fact that, if x = T (x) and y = T (y),

‖x − y‖ = ‖T (x) − T (y)‖ ≤ λ‖x − y‖,

which holds only if x = y since λ < 1.

Computationally, fixed-point x∗ is the limit of convergent sequence xk+1 =T (xk), k ∈ ℵ. Theorem 2.4 has important applications in control theory,starting with existence and uniqueness of a solution to differential equations[108, 192]. It will be shown in Section 2.3.2 that the Lyapunov direct methodcan be viewed as a special case of the contraction mapping theorem. Indeed,the concept of contraction mapping plays an important role in any conver-gence analysis.

2.2.2 Barbalat Lemma

The following lemma, known as the Barbalat lemma [141], is useful in conver-gence analysis.

Definition 2.5. A function w : ℜ+ → ℜ is said to be uniformly continuous if,for any ǫ > 0, there exists δ > 0 such that |t−s| < δ implies |w(t)−w(s)| < ǫ.

Lemma 2.6. Let w : ℜ+ → ℜ be a uniformly continuous function. Then, if

limt→∞∫ t

t0w(τ)dτ exists and is finite, limt→∞ w(t) = 0.

Proof: To prove by contradiction, assume that either w(t) does not have alimit or limt→∞ w(t) = c = 0. In the case that limt→∞ w(t) = c = 0, there

exists some finite constant α0 such that limt→∞∫ t

t0w(τ)dτ = α0+ct ≤ ∞, and

hence there is a contradiction. If w(t) does not have a limit as t → ∞, thereexists an infinite time sub-sequence ti : i ∈ ℵ such that limi→∞ ti = +∞and |w(ti)| ≥ ǫw > 0. Since w(t) is uniformly continuous, there exists interval[ti − δti, ti + δti] within which |w(t)| ≥ 0.5|w(ti)|. Therefore, we have

∫ ti+δti

t0

w(τ)dτ ≥∫ ti−δti

t0

w(τ)dτ + w(ti)δti, if w(ti) > 0,

and∫ ti+δti

t0

w(τ)dτ ≤∫ ti−δti

t0

w(τ)dτ + w(ti)δti, if w(ti) < 0.

Since ti is an infinite sequence and both |w(ti)| and δti are uniformly pos-itive, we know by taking the limit of ti → ∞ on both sides of the above


inequalities that∫ t

t0w(τ)dτ cannot have a finite limit, which contradicts the

stated condition.

The following example illustrates the necessity of w(t) being uniformlycontinuous for Lemma 2.6, and it also shows that imposing w(t) ≥ 0 does notadd anything. In addition, note that limt→∞ w(t) = 0 (e.g., w(t) = 1/(1 + t))

may not imply limt→∞∫ t

t0w(τ)dτ < ∞ either.

Example 2.7. Consider scalar time function: for n ∈ ℵ,

w(t) =

⎧

⎨

⎩

2n+1(t − n) if t ∈ [n, n + 2−n−1]2n+1(n + 2−n − t) if t ∈ [n + 2−n−1, n + 2−n]0 everywhere else

,

which is a triangle-wave sequence of constant height. It follows that w(t) iscontinuous and that

limt→∞

∫ t

0

w(τ)dτ =1

2+

1

22+

1

23+ · · · = 1 < ∞.

However, w(t) does not have a limit as t → ∞ as it approaches the sequenceof discrete-time impulse functions in the limit.

If w(t) is differentiable, w(t) is uniformly continuous if w(t) is uniformlybounded. This together with Lemma 2.6 has been widely used adaptive control[232], non-linear analysis [108], and robustness analysis [192], and it will beused in Chapter 6 to analyze non-linear cooperative systems.

2.2.3 Comparison Theorem

The following theorem, known as the comparison theorem, can be used to findexplicitly either a lower bound or an upper bound on a non-linear differentialequation which itself may not have an analytical solution.

Theorem 2.8. Consider the scalar differential equation

r = β(r, t), r(t0) = r0 (2.3)

where β(r, t) is continuous in t and locally Lipschitz in r for all t ≥ t0 andr ∈ Ω ⊂ ℜ. Let [t0, T ) be the maximal interval of existence of the solution r(t)such that r(t) ∈ Ω, where T could be infinity. Suppose that, for t ∈ [t0, T ),

v ≤ β(v, t), v(t0) ≤ r0, v(t) ∈ Ω. (2.4)

Then, v(t) ≤ r(t) for all t ∈ [t0, T ).


Proof: To prove by contradiction, assume that t1 ≥ t0 and δt1 > 0 exist suchthat

v(t1) = r(t1), and r(t) < v(t) ∀t ∈ (t1, t1 + δt1]. (2.5)

It follows that, for 0 < h < δt1,

r(t + h) − r(t1)

h<

v(t + h) − v(t1)

h,

which in turn impliesr(t1) < v(t1).

Combined with (2.3) and (2.4), the above inequality together with the equalityin (2.5) leads to the contradiction

β(t1, r(t1)) < β(t1, v(t1)).

Thus, either t1 or δt1 > 0 does not exist, and the proof is completed.

Extension of Theorem 2.8 to the multi-variable case is non-trivial andrequires the so-called quasi-monotone property defined below. Should sucha property hold, Theorem 2.10 can be applied. Nonetheless, the condition isusually too restrictive to be satisfied, as evident from the fact that, if F (x, t) =Ax and is mixed monotone, all the entries in sub-matrices A11 and A22 mustbe non-negative while entries of A12 and A22 are all non-positive, where

A =

[

A11 A12

A21 A22

]

, and A11 ∈ ℜk×k.

Definition 2.9. Function F (x, t) : ℜ+ ×ℜn → ℜn is said to possess a mixedquasi-monotone property with respect to some fixed integer k ∈ 0, 1, · · · , nif the following conditions hold:

(a) For all i ∈ 1, · · · , k, function Fi(x, t) is non-decreasing in xj for j =1, · · · , k and j = i, and it is non-increasing in xl where k < l ≤ n.

(b) For all i ∈ k + 1, · · · , n, function Fi(x, t) is non-increasing in xj forj = 1, · · · , k, and it is non-decreasing xl where k < l ≤ n and l = i.

In the special cases of k = 0 and k = n, function F (x, t) is said to have quasi-monotone non-decreasing and quasi-monotone non-increasing properties, re-spectively. Furthermore, function F (x, t) is said to possess mixed monotone(or monotone non-decreasing or monotone non-increasing) property if j = iand l = i are not demanded in (a) and (b).


Theorem 2.10. Consider the following two sets of differential inequalities:for v, w ∈ ℜn and for some k ∈ 0, 1, · · · , n

vi ≤ Fi(v, t), i ∈ 1, · · · , kvj > Fj(v, t), j ∈ k + 1, · · · , n

wi > Fi(w, t), i ∈ 1, · · · , kwj ≤ Fj(w, t), j ∈ k + 1, · · · , n .

(2.6)or

vi < Fi(v, t), i ∈ 1, · · · , kvj ≥ Fj(v, t), j ∈ k + 1, · · · , n

wi ≥ Fi(w, t), i ∈ 1, · · · , kwj < Fj(w, t), j ∈ k + 1, · · · , n .

(2.7)Suppose that F (x, t) is continuous in t and locally Lipschitz in x for all t ≥ t0and x ∈ Ω ⊂ ℜn, that initial conditions satisfy the inequalities of

vi(t0) < wi(t0), i ∈ 1, · · · , kvj(t0) > wj(t0), j ∈ k + 1, · · · , n , (2.8)

that F (x, t) has the mixed quasi-monotone property with respect to k, and that[t0, T ) be the maximal interval of existence of solutions v(t) and w(t) such thatv(t), w(t) ∈ Ω, where T could be infinity. Then, over the interval of t ∈ [t0, T ),inequalities vi(t) < wi(t) and vj(t) > wj(t) hold for all i ∈ 1, · · · , k and forall j ∈ k + 1, · · · , n.

Proof: Define

zi(t) = vi(t) − wi(t), i ∈ 1, · · · , kzj(t) = wj(t) − vj(t), j ∈ k + 1, · · · , n .

It follows from (2.8) that zi(t0) < 0 for all i. To prove by contradiction,assume that t1 ≥ t0 be the first time instant such that, for some l and forsome δt1 > 0,

zl(t1) = 0, and zl(t) > 0 ∀t ∈ (t1, t1 + δt1].

Hence, we know that, for all α ∈ 1, · · · , n,

zα(t1) ≤ 0, (2.9)

and that, for 0 < h < δt1,

zl(t + h) − zl(t1)

h=

zl(t + h)

h> 0,

which in turn implieszl(t1) > 0.

Combined with (2.6) or (2.7), the above inequality becomes

0 < zl(t1) < Fl(v(t1), t) − Fl(w(t1), t), if l ≤ k0 < zl(t1) < Fl(w(t1), t) − Fl(v(t1), t), if l > k

,


and, since F (x, t) possesses the mixed quasi-monotone property with respectto k, the above set of inequality contradicts with (2.9) or simply

vi(t1) ≤ wi(t1), i ∈ 1, · · · , kvj(t1) ≥ wj(t1), j ∈ k + 1, · · · , n .

Hence, t1 and δt1 > 0 cannot co-exist, and the proof is completed.

In Section 2.3.2, Theorem 2.8 is used to facilitate stability analysis of non-linear systems in terms of a Lyapunov function and to determine an explicitupper bound on state trajectory. In such applications as stability of large-scale interconnected systems, dynamic coupling among different sub-systemsmay have certain properties that render the vector inequalities of (2.6) and(2.7) in terms of a vector of Lyapunov functions vk and, if so, Theorem 2.10can be applied [118, 119, 156, 268]. Later in Section 6.2.1, another comparisontheorem is developed for cooperative systems, and it is different from Theorem2.10 because it does not require the mixed quasi-monotone property.

2.3 Lyapunov Stability Analysis

Dynamical systems can be described in general by the following vector differ-ential equation:

x = F ′(x, u, t), y = H(x, t), (2.10)

where x ∈ ℜn is the state, y ∈ ℜl is the output, and u ∈ ℜm is the input.Functions F ′(·) and H(·) provide detailed dynamics of the system. System2.10 is said to be affine if its differential equation can be written as

x = f(x) + g(x)u. (2.11)

The control problem is to choose a feedback law u = u(x, t) as the input suchthat the controlled system has desired the properties including stability, per-formance and robustness. Upon choosing the control, the closed-loop systemcorresponding to (2.10) becomes

x = F ′(x, u(x, t), t)= F (x, t), y = H(x, t). (2.12)

The analysis problem is to study qualitative properties of System 2.12 forany given set of initial condition x(t0). Obviously, control design must be anintegrated part of stability analysis, and quite often control u = u(x, t) is cho-sen through stability analysis. Analysis becomes somewhat easier if functionsF (x, t) and H(x, t) in (2.12) are independent of time. In this case, the systemin the form of

x = f(x), y = h(x), (2.13)

is called autonomous. In comparison, a system in the form of (2.12) is said tobe non-autonomous.

2.3 Lyapunov Stability Analysis 49

As the first step of analysis, stationary points of the system, called equilib-rium points, should be found by solving the algebraic equation: for all t ≥ t0,

F (0, t) = 0.

A system may have a unique equilibrium point (e.g., all linear systems ofx = Ax with invertible matrix A), or finite equilibrium points (e.g., certainnon-linear systems), or an infinite number of equilibrium points (e.g., cooper-ative systems to be introduced later). In standard stability analysis, x = 0 isassumed since, if not, a constant translational transformation can be appliedso that any specific equilibrium point of interest is moved to be the origin inthe transformed state space. As will be shown in Chapter 5, a cooperativesystem typically has an infinite set of equilibrium points, its analysis needs tobe done with respect to the whole set and hence should be handled differently.Should the system have periodic or chaotic solutions, stability analysis couldalso be done with respect to the solutions [2, 101]. In this section, the analysisis done only with respect to the equilibrium point of x = 0.

Qualitative properties of System 2.12 can be measured using the follow-ing Lyapunov stability concepts [141]. The various stability concepts providedifferent characteristics on how close the solution of System 2.12 will remainaround or approach the origin if the solution starts in some neighborhood ofthe origin.

Definition 2.11. If x = 0 is the equilibrium point for System 2.12, then x = 0is

(a) Lyapunov stable (or stable) if, for every pair of ǫ > 0 and t0 > 0, thereexists a constant δ = δ(ǫ, t0) such that ‖x(t0)‖ ≤ δ implies ‖x(t)‖ ≤ ǫ fort ≥ t0.

(b) Unstable if it is not Lyapunov stable.(c) Uniformly stable if it is Lyapunov stable and if δ(ǫ, t0) = δ(ǫ).(d) Asymptotically stable if it is Lyapunov stable and if there exists a constant

δ′(t0) such that ‖x(t0)‖ ≤ δ′ implies limt→∞ ‖x(t)‖ = 0.(e) Uniformly asymptotically stable if it is uniformly stable and if δ′(t0) = δ′.(f) Exponentially stable if there are constants δ, α, β > 0 such that ‖x(t0)‖ ≤

δ implies‖x(t)‖ ≤ α‖x(t0)‖e−β(t−t0).

Lyapunov stability states that, if the initial condition of the state is ar-bitrarily close to the origin, the state can remain within any small hyperballcentered at the origin. The relevant quantity δ represents an estimate on theradius of stability regions. If δ is infinite, the stability results are global; other-wise, the stability results are local. Asymptotical stability requires addition-ally that the solution converges to the origin, and quantity δ′ is an estimateon the radius of convergence region. Complementary to the above Lyapunovstability concepts are the concepts of uniform boundedness, uniform ultimate


boundedness, asymptotic convergence, and input-to-state stability (ISS) [235].

Definition 2.12. System 2.12 is said to be

(a) Uniformly bounded if, for any c′ > 0 and for any initial condition‖x(t0)‖ ≤ c′, there exists a constant c such that ‖x(t)‖ ≤ c for t ≥ t0.

(b) Uniformly ultimately bounded with respect to ǫ > 0 if it is uniformlybounded and if there exists a finite time period τ such that ‖x(t)‖ ≤ ǫ fort ≥ t0 + τ .

(c) Asymptotically convergent if it is uniformly bounded and if x(t) → 0 ast → ∞.

System 2.10 is input-to-state stable if, under u = 0, it is uniformly asymp-totically stable at x = 0 and if, under any bounded input u, the state is alsouniformly bounded.

For a control system, uniform boundedness is the minimum requirement. Itis obvious that Lyapunov stability implies uniform boundedness, that asymp-totic stability implies asymptotic convergence, and that asymptotic conver-gence implies uniform ultimate boundedness. But, their reverse statementsare not true in general. Concepts (a), (b), and (c) in Definition 2.12 are oftenused in robustness analysis [192] because they do not require x = 0 be anequilibrium point.

Several of the above concepts are illustrated by the first-order systems inthe following example.

Example 2.13. (1) The first-order, time-varying, linear system of x = a(t)xhas the following solution:

x(t) = x(t0)e

∫

t

t0a(τ)dτ

.

Using this solution, one can verify the following stability results:

(1a) System x = −x/(1 + t)2 is stable but not asymptotically stable.(1b) System x = (6t sin t − 2t)x is stable but not uniformly stable.(1c) System x = −x/(1+ t) is uniformly stable and asymptotically stable, but

not uniformly asymptotically stable.

(2) System x = −x3 has the solution of x(t) = 1/√

2t + 1/x2(0). Hence, thesystem is asymptotically stable but not exponentially stable.(3) System x = −x + sign(x)e−t has the solution x(t) = e−(t−t0)x(t0) +sign(x(t0))(e

−t0 − e−t), where sign(·) is the standard sign function withsign(0) = 0. Thus, the system is asymptotically convergent, but it is notstable.(4) System x = −x + xu is exponentially stable with u = 0, but it is notinput-to-state stable.


In the above example, stability is determined by analytically solving dif-ferential equations. For time-invariant linear systems, a closed-form solutionis found and used in Section 2.4 to determine stability conditions. For time-varying linear systems or for non-linear systems, there is in general no closed-form solution. In these cases, stability should be studied using the Lyapunovdirect method.

2.3.1 Lyapunov Direct Method

The Lyapunov direct method can be utilized to conclude various stabilityresults without the explicit knowledge of system trajectories. It is based onthe simple mathematical fact that, if a scalar function is both bounded frombelow and decreasing, the function has a limit as time t approaches infinity.For stability analysis of System 2.12, we introduce the following definition.

Definition 2.14. A time function γ(s) is said to be strictly monotone increas-ing (or strictly monotone decreasing) if γ(s1) < γ(s2) (or γ(s1) > γ(s2)) forany s1 < s2.

Definition 2.15. A scalar function V (x, t) is said to be

(a) Positive definite (p.d.) if V (0, t) = 0 and if V (x, t) ≥ γ1(‖x‖) for somescalar, strictly monotone increasing function γ1(·) with γ1(0) = 0. A pos-itive definite function V (x, t) is said to be radially unbounded if its asso-ciated lower bounding function γ1(·) has the property that γ1(r) → ∞ asr → +∞.

(b) Positive semi-definite (p.s.d.) if V (x, t) ≥ 0 for all t and x.(c) Negative definite or negative semi-definite (n.d. or n.s.d.) if −V (x, t) is

positive definite or positive semi-definite, respectively.(d) Decrescent if V (x, t) ≤ γ2(‖x‖) for some scalar, strictly monotone in-

creasing function γ2(·) with γ2(0) = 0.

Clearly, V (x, t) being p.d. ensures that scalar function V (x, t) is boundedfrom below. To study stability of System 2.12, its time derivative along anyof system trajectories can be evaluated by

V (x, t) =∂V

∂t+

(

∂V

∂x

)T

x =∂V

∂t+

(

∂V

∂x

)T

F (x, t).

If V is n.d. (or n.s.d.), V (x, t) keeps decreasing (or is non-increasing), asymp-totic stability (or Lyapunov stability) can be concluded as stated in the fol-lowing theorem, and V (x, t) is referred to as a Lyapunov function.

Theorem 2.16. System 2.12 is

(a) Locally Lyapunov stable if, in a neighborhood around the origin, V (x, t) isp.d. and V (x, t) is n.s.d.


(b) Locally uniformly stable as ‖x(t)‖ ≤ γ−11 γ2(‖x(t0)‖) if, for x ∈ x ∈

ℜn : ‖x‖ < η with η ≥ γ2(‖x(t0)‖), V (x, t) is p.d. and decrescent andV (x, t) is negative semi-definite.

(c) Uniformly bounded and uniformly ultimately bounded with respect to ǫ ifV (x, t) is p.d. and decrescent and if, for x ∈ x ∈ ℜn : ‖x‖ ≥ γ−1

1 γ2(ǫ),V (x, t) is negative semi-definite.

(d) Locally uniformly asymptotically stable in the region of x ∈ ℜn : ‖x‖ <γ−12 (γ1(η)) if, for ‖x‖ < η, V (x, t) is p.d. and decrescent and if V (x, t)

is n.d.(e) Globally uniformly asymptotically stable if V (x, t) is p.d., radially un-

bounded and decrescent and if V (x, t) is n.d. everywhere.(f) Exponentially stable if γ1‖x‖2 ≤ V (x, t) ≤ γ2‖x‖2 and V (x, t) ≤ −γ3‖x‖2

for positive constants γi, i = 1, 2, 3.(g) Unstable if, in every small neighborhood around the origin, V is n.d. and

V (x, t) assumes a strictly negative value for some x therein.

System 2.10 is ISS if V (x, t) is p.d., radially unbounded and decrescent and

if V (x, t) ≤ −γ3(‖x‖) + γβ3 (‖x‖)γ4(‖u‖), where 0 ≤ β < 1 is a constant,

and γi(·) with i = 3, 4 are scalar strictly monotone increasing functions withγi(0) = 0.

A useful observation is that, if V (x, t) ≤ 0 for ‖x‖ ≤ η with η ≥ γ2(‖x(t0)‖)and if ‖x(t0)‖ < γ−1

2 (γ1(η)), V (x(t0), t0) ≤ 0 and, for any sufficiently smallδt > 0, V (x(t), t) ≤ V (x(t0), t0) ≤ γ2(‖x(t0)‖) and hence V (x, t + δt) ≤ 0.By applying the observation and repeating the argument inductively, all thestatements in Theorem 2.16 except for (vi) can be proven. Proof of (vi) ofTheorem 2.16 will be pursued in the next subsection. Counterexamples areprovided in [108] to show that the condition of radial unboundedness is neededto conclude global stability and that the decrescent condition is required forboth asymptotic stability and uniform stability.

The key of applying Lyapunov direct method is to find Lyapunov func-tion V (x, t). In searching of Lyapunov function, we should use the backwardprocess of first finding V symbolically in terms of ∂V/∂x and then selecting∂V/∂x so that V is found and V has one of the properties listed in Theorem2.16. The process is illustrated by the following example.

Example 2.17. Consider the second-order system:

x1 = x2, x2 = −x31 − x2.

Let V (x) be the Lyapunov function to be found. It follows that

V =∂V

∂x1x2 +

∂V

∂x2(−x3

1 − x2).

In order to conclude Lyapunov stability, we need to show that V is at leastn.s.d. To this end, we need to eliminate such sign-indefinite terms as the crossproduct terms of x1 and x2. One such choice is that


∂V

∂x1x2 =

∂V

∂x2x3

1, (2.14)

which holds if V = x41 + 2x2

2 is chosen. Consequently, we know that

V = −4x22

is n.s.d. and hence the system is globally (uniformly) Lyapunov stable.Next, let us determine whether Lyapunov function V exists to make V n.d.

It follows from the expression for V that V may become n.d. only if ∂V /∂x2

contains such a term as x1. Accordingly, we introduce an additional term of2ax1 into the previous expression for ∂V/∂x2, that is,

∂V

∂x2= 2ax1 + 4x2

for some constant a > 0. Hence, it follows that

V (x) = 2ax1x2 + 2x22 + h(x1),

where function h(x1) > 0 is desired. To ensure (2.14), h(x1) must contain x41.

On the other hand, V (x) should be positive definite, and this can be madepossible by including bx2

1 in h(x1) for some constant b > 0. In summary, thesimplest form of V (x) should be

V (x) = 2ax1x2 + 2x22 + x4

1 + bx21,

which is positive definite if a < 2√

2b. Consequently, we have

V = (2ax2 + 4x31 + 2bx1)x2 + (2ax1 + 4x2)(−x3

1 − x2)

= −2ax41 − (4 − 2a)x2

2 + (2b − 2a)x1x2,

which is n.d. under many choices of a and b (e.g., a = b = 1). Since V is p.d.and V is n.d., the system is shown to be asymptotically stable.

While the system is stable, many choices of V (x) (such as V (x) = x21 +x2

2)would yield a sign-indefinite expression of V .

The above example shows that the conditions in Theorem 2.16 are suffi-cient, Lyapunov function is not unique and, unless the backward process yieldsa Lyapunov function, stability analysis is inconclusive. Nonetheless, existenceof a Lyapunov function is guaranteed for all stable systems. This result isstated as the following theorem, referred to Lyapunov converse theorem, andits proof based on qualitative properties of system trajectory can be found in[108].

Theorem 2.18. Consider System 2.12. Then,

(a) If the system is stable, there is a p.d. function V (x, t) whose time derivativealong the system trajectory is n.s.d.


(b) If the system is (globally) uniformly asymptotically stable, there is a p.d.decrescent (and radially unbounded) function V (x, t) such that V is n.d.

(c) If the system is exponentially stable, there are Lyapunov function V (x, t)and positive constants γi such that γ1‖x‖2 ≤ V (x, t) ≤ γ2‖x‖2, V (x, t) ≤−γ3‖x‖2, and ‖∂V/∂x‖ ≤ γ4‖x‖.

It is the converse theorem that guarantees existence of the Lyapunov func-tions and makes Lyapunov direct method a universal approach for non-linearsystems. Nonetheless, the theorem provides a promise rather than a recipefor finding a Lyapunov function. As a result, Lyapunov function has to befound for specific classes of non-linear systems. In the subsequent subsections,system properties are explored to search for Lyapunov functions.

2.3.2 Explanations and Enhancements

A Lyapunov function can be sought by exploiting either physical, mathe-matical, or structural properties of system dynamics. In what follows, physi-cal and mathematical features of the Lyapunov direct method are explained,while structural properties of dynamics will be explored in Section 2.6.1 toconstruct the Lyapunov function. In Section 2.6.4, the Lyapunov function isrelated to a performance function in an optimal control design.

Interpretation as an Energy Function

In essence, stability describes whether and how the system trajectory movestoward its equilibrium point, and hence the motion can be analyzed using aphysical measure such as an energy function or its generalization. Intuitively,if the total energy of a system keeps dissipating over time, the system willeventually lose all of its initial energy and consequently settle down to anequilibrium point. Indeed, Lyapunov function is a measure of generalized en-ergy, and the Lyapunov direct method formalizes the dissipative argument.As an illustration, consider an one-dimensional rigid-body motion for whichkinetic and potential energies are assumed to be

K =1

2m(x)x2, P = P(x),

respectively, where m(x) is a positive number or function. It follows from(1.16) that the dynamic equation of motion is

m(x)x +1

2

dm(x)

dxx2 +

dP(x)

dx= τ. (2.15)

Obviously, the energy function of the system is

E = K + P =1

2m(x)x2 + P(x). (2.16)


It follows that the time derivative of E along trajectories of System 2.15 is

E = xτ.

If the net external force is zero, E = 0, the total energy of systems is conser-vative, and hence System 2.15 with τ = 0 is called a conservative system. Onthe other hand, if τ = −kx for some k > 0, the external input is a dynamicfriction force, and System 2.15 is dissipative as

E = −kx2,

which is n.s.d. Depending upon the property of potential energy P(x), equi-librium point(s) can be found, and asymptotic stability could be concludedby following Example 2.17 and finding an appropriate Lyapunov function. Inother words, an energy function such as the one in (2.16) can be used as thestarting point to search for the Lyapunov function.

Interpretation as a Contraction Mapping

Suppose that, for a given system x = F (x, t), Lyapunov function V (x) isfound such that V (x) is positive definite and

V (x) =

[

∂V (x)

∂x

]T

F (x, t)

is negative definite. It follows that, for any infinite time sequence tk : k ∈ ℵ,V (x(t)) ≤ V (x(ti)) for all t ∈ [ti, ti+1) and

V (x(ti+1)) = V (x(ti)) +

∫ ti+1

ti

V (x)dτ ≤ −λV (x(ti)),

for some λ ∈ [0, 1). It follows from Theorem 2.4 that Lyapunov functionV (x(ti)) itself is a contraction mapping from which asymptotic stability canbe concluded.

Enhancement by Comparison Theorem

The comparison theorem, Theorem 2.8, can be used to facilitate a generalizedLyapunov argument if the Lyapunov function and its time derivative rendera solvable inequality. The following lemma is such a result in which V is notnegative definite in the neighborhood around the origin.

Lemma 2.19. Let V be a (generalized) Lyapunov function for System 2.12such that

γ1(‖x(t)‖) ≤ V (x, t) ≤ γ2(‖x‖),and

V (x, t) ≤ −λγ2(‖x‖) + ǫϕ(t),

where γi(·) are strictly monotone increasing functions with γi(0) = 0, λ, ǫ > 0are constants, and 0 ≤ ϕ(t) ≤ 1. Then, the system is


(a) Uniformly ultimately bounded with respect to γ−11 (ǫ/λ).

(b) Asymptotically convergent if ϕ(t) converges to zero.(c) Exponentially convergent if ϕ(t) = e−βt for some β > 0.

Proof: It follows that V (x, t) ≤ −λV (x, t) + ǫϕ(t) and hence, by Theorem2.8, V (x, t) ≤ w(t), where

w = −λw + ǫϕ(t), w(t0) = V (x(t0), t0).

Solving the scalar differential equation yields

V (x, t) ≤ e−λ(t−t0)V (x(t0), t0) + ǫ

∫ t

t0

e−λ(t−s)ϕ(s)ds,

from which the statements become obvious.

If ǫ = 0 in the statement, Lemma 2.19 reduces to some of the stabilityresults in Theorem 2.16.

Enhancement by Barbalat Lemma

As a useful tool in Lyapunov stability analysis, the Barbalat lemma, Lemma2.6, can be used to conclude convergence for the cases that V is merely n.s.d.with respect to state x and may also contain an L1-space time function. Thefollowing lemma illustrates such a result.

Lemma 2.20. Consider System 2.12 in which function F (x, t) is locally uni-formly bounded with respect to x and uniformly bounded with respect to t. LetV be its Lyapunov function such that

γ1(‖x(t)‖) ≤ V (x, t) ≤ γ2(‖x‖),

andV (x, t) ≤ −γ3(‖z(t)‖) + ϕ(t),

where γi(·) are strictly monotone increasing functions with γi(0) = 0, γ3(·)is locally Lipschitz, z(t) is a sub-vector of x(t), and ϕ(t) belongs to L1-space.Then, the system is uniformly bounded and the sub-state z(t) is asymptoticallyconvergent.

Proof: It follows from the expression of V that

V (x(t), t) +

∫ t

t0

γ3(‖z(τ)‖)dτ ≤ V (x0, t0) +

∫ t

t0

ϕ(s)ds

≤ V (x0, t0) +

∫ ∞

t0

ϕ(s)ds

< ∞.


The above inequality implies that V (x(t), t) and hence x(t) are uniformlybounded. Recalling properties of F (x, t), we know from (2.12) that x is uni-formly bounded and thus x(t) as well as z(t) is uniformly continuous. On theother hand, the above inequality also shows that γ3(‖z(τ)‖) belongs to L1-space. Thus, by Lemma 2.6, z(t) is asymptotically convergent.

For the autonomous system in (2.13), Lemma 2.20 reduces to the famousLaSalle’s invariant set theorem, given below. Set Ω is said to be invariant fora dynamic system if, by starting within Ω, its trajectory remains there for allfuture time.

Theorem 2.21. Suppose that V (x) is p.s.d. for ‖x‖ < η and, along any tra-jectory of System 2.13, V is n.s.d. Then, state x(t) converges either to a

periodic trajectory or an equilibrium point in set Ω= x ∈ ℜn : V (x) =

0, ‖x‖ < η. System 2.13 is asymptotically stable if set Ω contains no peri-odic trajectory but only equilibrium point x ≡ 0. Moreover, asymptotic stabilitybecomes global if η = ∞ and V (x) is radially unbounded.

It must be emphasized that Theorem 2.21 only holds for autonomous sys-tems. It holds even without V (x) being positive semi-definite as long as, forany constant l > 0, the set defined by V (x) < l is closed and bounded.

Directional Derivative

Consider the autonomous system in (2.13). The corresponding Lyapunov func-tion is time-invariant as V = V (x), and its time derivative along trajectoriesof System 2.13 is

V =

(

∂V

∂x

)T

f(x), or V = (xV )T f(x),

which is the dot product of system dynamics and gradient of Lyapunov func-tion. Hence, as the projection of the gradient along the direction of motion,V is called directional derivative.

To simplify the notations in the subsequent discussions, the so-called Liederivative is introduced. Lie derivative of scalar function ξ(x) with respect tovector function f(x), denoted by Lfξ, is a scalar function defined by

Lfξ = (Tx ξ)f.

High-order Lie derivatives can be defined recursively as, for i = 1, · · ·,L0

fξ = ξ, Lifξ = Lf (Li−1

f ξ) = [Tx (Li−1

f ξ)]f, and LgLfξ = [Tx (Lfξ)]g,

where g(x) is another vector field of the same dimension. It is obvious that, ifyj = hj(x) is a scalar output of System 2.13, the ith-order time derivative of

this output is simply y(i)j = Li

fhj and that, if V (x) is the Lyapunov function

for System 2.13, V = LfV .


2.3.3 Control Lyapunov Function

Consider the special case that dynamics of Control System 2.10 do not ex-plicitly depend on time, that is,

x = F ′(x, u), (2.17)

where F ′(0, 0) = 0. For System 2.17, existence of both a stabilizing controland its corresponding Lyapunov function is captured by the following conceptof control Lyapunov function.

Definition 2.22. A smooth and positive definite function V (x) is said to bea control Lyapunov function for System 2.17 if, for any x = 0,

infu∈ℜm

LF ′(x,u)V (x) < 0. (2.18)

Control Lyapunov function V (·) is said to satisfy the small control propertyif, for every ǫ > 0, there exists δ > 0 such that Inequality 2.18 holds for anyx with 0 < ‖x‖ < δ and for some u(x) with ‖u(x)‖ < ǫ.

Clearly, in light of (2.18), the time derivative of the control Lyapunovfunction along the system trajectory can always be made negative definite byproperly choosing a feedback control u(x), which is sufficient for concludingat least local asymptotic stability. The converse Lyapunov theorem, Theorem2.18, also ensures the existence of a control Lyapunov function if System 2.17is asymptotically stabilized. The following lemma due to [8, 165] provides asufficient condition for constructing control and concluding stability.

Lemma 2.23. Suppose that System 2.17 has a control Lyapunov functionV (x). If the mapping u → LF ′(x,u)V (x) is convex for all x = 0, then thesystem is globally asymptotically stable under a feedback control u(x) whichis continuous for all x = 0. In addition, if V (x) satisfies the small controlproperty, the control u(x) is continuous everywhere.

Given a control Lyapunov function, construction of a stabilizing controlfor System 2.17 is generally non-trivial. In addition, the convex conditionrequired by the above lemma may not be valid in general. For the affine non-linear control system in (2.11), existence of control Lyapunov function V (x)is equivalent to the requirement that Lg(x)V (x) = 0 implies Lf(x)V (x) <0. Should the control Lyapunov function be known for Affine System 2.11,a universal feedback controller is available in terms of the following Sontagformula [234]:

u(x) =

−LfV +√

(Lf V )2+‖LgV ‖2α(x)

‖LgV ‖2 (LgV )T if LgV = 0

0 if LgV = 0, (2.19)

where α(x) ≥ 0 is a scalar function. It is straightforward to show that u(x) isstabilizing as inequality LfV + [LgV ]u(x) < 0 holds everywhere, that u(x) is


continuous except at those points satisfying LgV = 0, and that u(x) becomescontinuous everywhere under the choice of α(x) = ξ(‖LgV ‖), where ξ(·) is ascalar function satisfying ξ(0) = 0 and ξ(a) > 0 for a > 0.

The problem of finding a control Lyapunov function for System 2.17can be studied by imposing certain conditions on function F (x, u), eitheranalytical properties or special structures. In the first approach, one typi-cally assumes that a positive definite function V0(x) (as a weak version ofLyapunov function) is already known to yield Lf(x)V0(x) ≤ 0 and that,for controllability (which will be discussed in Section 2.5), vector fieldsf, adfgk, ad2

fgk, · · · satisfy certain rank conditions, where f(x) = F ′(x, 0)and gk(x) = ∂F ′(x, u)/∂uk|u=0. Based on the knowledge of V0(x) as well asf and gk, a control Lyapunov function V (x) can be constructed [152]. In thesecond approach, a special structural property of the system is utilized toexplicitly search for control Lyapunov function, which will be carried out inSection 2.6.

2.3.4 Lyapunov Analysis of Switching Systems

Among non-autonomous systems of form (2.12), there are switching systemswhose dynamics experience instantaneous changes at certain time instantsand are described by

x = F (x, s(t)), s(t) ∈ I, (2.20)

where s(·) is the switching function (or selection function), and I is the cor-responding value set (or index set). In the discussion of this subsection, it isassumed that set I be a finite sub-set of ℵ and that, uniformly with respectto all s(t) ∈ I, function F (x, s(t)) be bounded for each x and also be locallyLipschitz in x.

In principle, analysis of Switching System 2.20 is not much different fromthat of Non-autonomous System 2.12. Specifically, Lyapunov direct methodcan readily be applied to System 2.20, and the successful Lyapunov function isusually time dependent as V (x, s(t)). The dependence of V on discontinuousfunction s(t) introduces the technical difficulty that time derivative V containssingular points. Consequently, non-smooth analysis such as semi-continuity,set valued map, generalized solution, and differential inclusions [11, 65, 110,219, 220] need be used in stability analysis.

One way to avoid the complication of non-smooth analysis is to find acommon Lyapunov function, that is, Lyapunov function V (x) that is pos-itive definite and whose time derivative LF (x,s(t))V (x) is negative definiteno matter what fixed value s(t) assumes in set I. It is obvious that, if a(radially-unbounded) common Lyapunov function exists, System 2.20 is (glob-ally) asymptotically stable and so are the family of autonomous systems

xs = F (xs, s)= Fs(xs), s ∈ I. (2.21)


However, stability of all the autonomous systems in (2.21) is not necessary orsufficient for concluding stability of Switching System 2.20. It will be shown inSection 2.4.3 that a system switching between two stable linear autonomoussystems may be unstable. Similarly, a system switching between two unstablelinear autonomous systems may be stable. In other words, an asymptoticallystable switching system in the form of (2.20) may not have a common Lya-punov function, nor do its induced family of autonomous systems in (2.21)have to be asymptotically stable. On the other hand, if the switching is arbi-trary in the sense that switching function s(t) is allowed to assume any valuein index set I at any time, stability of Switching System 2.20 implies stabilityof all the autonomous systems in (2.21). Indeed, under the assumption of ar-bitrary switching, the corresponding Lyapunov converse theorem is available[49, 142, 155] to ensure the existence of a common Lyapunov function for anasymptotically stable switching system of (2.20) and for all the autonomoussystems in (2.21). In short, stability analysis under arbitrary switching canand should be done in terms of a common Lyapunov function.

In most cases, switching occurs according to a sequence of time instantsthat are either fixed or unknown a priori and, if any, the unknown switchingtime instants are not arbitrary because they are determined by uncertainexogenous dynamics. In these cases, System 2.20 is piecewise-autonomous as,for a finite family of autonomous functions Fs(x) : s ∈ I, and for somesequence of time instances ti : i ∈ ℵ,

x = F (x, s(ti))= Fs(ti)(x), ∀t ∈ [ti, ti+1). (2.22)

System 2.22 may not have a common Lyapunov function, nor may its inducedfamily of autonomous systems in (2.21). The following theorem provides thecondition under which stability can be concluded using a family of Lyapunovfunctions defined for Autonomous Systems 2.21 and invoked over consecutivetime intervals.

Theorem 2.24. Consider the piecewise-autonomous system in (2.22). Sup-pose that the corresponding autonomous systems in (2.21) are all globallyasymptotically stable and hence have radially unbounded positive definite Lya-punov functions Vs(x) and that, along any trajectory of System 2.22, the fol-lowing inequality holds: for every pair of switching times (ti, tj) satisfyingti < tj, s(ti) = s(tj) ∈ I, and s(tk) = s(tj) for any tk of ti < tk < tj,

Vs(tj)(x(tj)) − Vs(ti)(x(ti)) ≤ −Ws(ti)(x(ti)), (2.23)

where Ws(x) with s ∈ I are a family of positive definite continuous functions.Then, Switched System 2.22 is globally asymptotically stable.

Proof: We first show global Lyapunov stability. Suppose ‖x(t0)‖ < δ forsome δ > 0. Since Vs(x) is radially unbounded and positive definite, thereexist scalar strictly monotone increasing functions γs1(·) and γs2(·) such that


γs1(‖x‖) ≤ Vs(x) ≤ γs2(‖x‖). Therefore, it follows from (2.23) that, for anytj ,

Vs(tj)(x(tj)) ≤ maxs∈I

γs2(δ),

which yields

‖x(tj)‖ ≤ maxσ∈I

γ−1σ1 max

s∈Iγs2(δ)

= ǫ.

The above inequality together with asymptotic stability of autonomous sys-tems in (2.21) imply that System 2.22 is globally Lyapunov stable.

Asymptotic stability is obvious if time sequence ti : i ∈ ℵ is finite. If thesequence is infinite, let tσj

: σj ∈ ℵ, j ∈ ℵ denote its infinite sub-sequencecontaining all the entries of s(ti) = s(tσj

) = σ ∈ I. It follows from (2.23) thatLyapunov sequence Vσ(x(tσj

)) is monotone decreasing and hence has a limitc and that

0 = c − c = limj→∞

[Vσ(x(tσj)) − Vσ(x(tσj+1 )] ≤ − lim

j→∞Wσ(x(tσj

)) ≤ 0.

Since Ws(·) is positive definite and σ ∈ I is arbitrary, x(ti) converges tozero. The proof is completed by recalling asymptotic stability of autonomoussystems in (2.21).

In stability analysis of switching systems using multiple Lyapunov func-tions over time, Theorem 2.24 is representative among the available results[28, 133, 185]. Extensions such as that in [91] can be made so that some Lya-punov functions are allowed to increase during their active time intervals aslong as these increases are bounded by positive definite functions properly in-corporated into Inequality 2.23. Similarly, some of the systems in (2.21) maynot have to be asymptotically stable provided that an inequality in the form of(2.23) holds over time. As will be illustrated in Section 2.4.3 for linear switch-ing systems, Inequality 2.23 implies monotone decreasing over sequences ofconsecutive intervals, while System 2.22 can have transient increases such asovershoots during some of the intervals. To satisfy Inequality 2.23, either thesolution to (2.22) or a quantitative bound on the solution should be found,which is the main difficulty in an application of Theorem 2.24. In general,Inequality 2.23 can always be ensured if the so-called dwell time, the lengthof the intervals for System 2.22 to stay as one of the systems in (2.21), is longif the corresponding system in (2.21) is asymptotically stable and sufficientlyshort if otherwise. Further discussions on dwell time can be found in Section2.4.3 and in [133].

It is worth noting that, while multiple Lyapunov functions are used inTheorem 2.24 to establish stability, only one of the Lyapunov functions isactively used at every instant of time to capture the instantaneous systembehavior and their cumulative effect determines the stability outcome. Thisis different from Comparison Theorem 2.10 by which a vector of Lyapunovfunctions is used simultaneously to determine qualitative behavior of a system.


2.4 Stability Analysis of Linear Systems

State space model of a linear dynamical system is

x = A(t)x + B(t)u, y = C(t)x + D(t)u, (2.24)

where x ∈ ℜn is the state, u ∈ ℜm is the input, and y ∈ ℜp is the output.Matrices A, B, C, D are called system matrix, input matrix, output matrix,and direct coupling matrix, respectively. System 2.24 is called time-invariantif these matrices are all constant. In what follows, analysis tools of linearsystems are reviewed.

2.4.1 Eigenvalue Analysis of Linear Time-invariant Systems

Eigenvalue analysis is both fundamental and the easiest to understandingstability of the linear time-invariant system:

x = Ax + Bu. (2.25)

Defining matrix exponential function eAt as

eAt =

∞∑

j=0

1

j!Ajtj, (2.26)

we know that eAt satisfies the property of

d

dteAt = AeAt = eAtA,

that the solution to x = Ax is x(t) = eA(t−t0)x(t0) and hence eA(t−t0) is calledstate transition matrix, and that the solution to (2.25) is

x(t) = eAtx(0) +

∫ t

0

eA(t−τ)Bu(τ)dτ. (2.27)

The application of a similarity transformation, reviewed in Section 2.1,is one of the time-domain approaches for solving state space equations andrevealing stability properties. Consider System 2.25 and assume that, undersimilarity transformation z = S−1x, z = S−1ASz = Jz where J is the Jordancanonical form in (2.1). It follows from the structure of J and the Taylor seriesexpansion in (2.26) that

eAt = S

⎡

⎢

⎣

eJ1t

. . .

eJlt

⎤

⎥

⎦S−1,

in which

2.4 Stability Analysis of Linear Systems 63

eJit =

⎡

⎢

⎢

⎢

⎢

⎣

eλit teλit · · · tni−1

(ni−1)!eλit

0. . .

. . ....

......

. . . teλit

0 0 · · · eλit

⎤

⎥

⎥

⎥

⎥

⎦

. (2.28)

Therefore, the following necessary and sufficient conditions [102, 176] can beconcluded from the solutions of (2.27) and (2.28):

(a) System 2.25 with u = 0 is Lyapunov stable if and only if its eigenvaluesare not in the right open half plane and those on the imaginary axis areof geometrical multiplicity one.

(b) System 2.25 with u = 0 is asymptotically stable if and only if matrix Ais Hurwitz (i.e., its eigenvalues are all in the left open half plane).

(c) System 2.25 is input-to-state stable if and only if it is asymptoticallystable.

(d) System 2.25 with u = 0 is exponentially stable if and only if it is asymp-totically stable.

2.4.2 Stability of Linear Time-varying Systems

Consider first the continuous-time linear time-varying system

x = A(t)x, x ∈ ℜn. (2.29)

Its solution can be expressed as

x(t) = Φ(t, t0)x(t0), (2.30)

where state transition matrix Φ(·) has the properties that Φ(t0, t0) = I,Φ(t, s) = Φ−1(s, t), and ∂Φ(t, t0)/∂t = A(t)Φ(t, t0). It follows that the abovesystem is asymptotically stable if and only if

limt→∞

Φ(t, t0) = 0.

However, an analytical solution of Φ(t, t0) is generally difficult to find ex-cept for simple systems. The following simple example shows that eigenvalueanalysis does not generally reveal stability of linear time-varying systems.

Example 2.25. Consider System 2.29 with

A(t) =

[

−1 +√

2 cos2 t 1 −√

2 sin t cos t

−1 −√

2 sin t cos t −1 +√

2 sin2 t

]

.

Matrix A(t) is continuous and uniformly bounded. By direct computation,one can verify that the corresponding state transition matrix is


Φ(t, 0) =

[

e(√

2−1)t e−t sin t

−e(√

2−1)t sin t e−t cos t

]

.

Hence, the system is unstable. Nonetheless, eigenvalues of matrix A(t) are at[−(2 −

√2) ±

√2j]/2, both of which are time-invariant and in the left open

half plan.

Piecewise-constant systems are a special class of time-varying systems.Consider a continuous-time switching system which is in the form of (2.29)and whose system matrix A(t) switches between two constant matrices as, forsome τ > 0 and for k ∈ ℵ,

A(t) =

A1 if t ∈ [2kτ, 2kτ + τ)A2 if t ∈ [2kτ + τ, 2kτ + 2τ)

. (2.31)

Since the system is piecewise-constant, the state transition matrix is knownto be

Φ(t, 0) =

eA1(t−2kτ)Hk for [2kτ, 2kτ + τ)eA2(t−2kτ−τ)eA1τHk for [2kτ + τ, 2kτ + 2τ)

, H = eA2τeA1τ ,

whose convergence depends upon the property of matrix exponential Hk.Indeed, for System 2.29 satisfying (2.31), we can define the so-called averagesystem as

xa = Aaxa, x ∈ ℜn. (2.32)

where xa(2kτ) = x(2kτ) for all k ∈ ℵ, e2Aaτ = H , and matrix Aa can beexpressed in terms of Baker-Campbell-Hausdorff formula [213] as

Aa = A1 + A2 +1

2adA1(A2) +

1

12adA1(adA1A2) + adA2(adA1A2)

+ · · · , (2.33)

and adAB = AB − BA is the so-called Lie bracket (see Section 2.5 for moredetails). Note that Aa in (2.33) is an infinite convergent sequence in which thegeneric expression of ith term is not available. If adAB = 0, the system givenby (2.31) is said to have commuting matrices Ai and, by invoking (2.32) and(2.33), it has the following solution:

x(t) = eA1(τ+τ+···)eA2(τ+τ+···)x(0),

which is exponentially stable if both A1 and A2 are Hurwitz. By induction, alinear system whose system matrix switches among a finite set of commutingHurwitz matrices is exponentially stable. As a relaxation, the set of matricesare said to have nilpotent Lie algebra if all Lie brackets of sufficiently high-order are zero, and it is shown in [84] that a linear system whose system matrixswitches among a finite set of Hurwitz matrices of nilpotent Lie algebra is alsoexponentially stable.


Similarly, consider a discrete-time linear time-varying system

xk+1 = Akxk, xk ∈ ℜn. (2.34)

It is obvious that the above system is asymptotically stable if and only if

limk→∞

AkAk−1 · · ·A2A1= lim

k→∞

k∏

j=1

Aj = 0. (2.35)

In the case that Ak arbitrarily switches among a finite number matricesD1, · · · , Dl, The sequence convergence of (2.35) can be embedded into sta-bility analysis of the switching system:

zk+1 = Akzk, (2.36)

where Ak switches between two constant matrices D1 and D2,

D1 = diagD1, · · · , Dl, D2 = T ⊗ I,

and T ∈ ℜl×l is any of the cyclic permutation matrices (see Sections 4.1.1 and4.1.3 for more details). Only in the trivial case of n = 1 do we know that thestandard time-invariant stability condition, |Di| < 1, is both necessary andsufficient for stability of System 2.36. The following example shows that, evenin the simple case of n = 2, the resulting stability condition on System 2.36 isnot general or consistent enough to become satisfactory. That is, there is nosimple stability test for switching systems in general.

Example 2.26. Consider the pair of matrices: for constants α, β > 0,

D1 =√

β

[

1 α0 1

]

, D2 = DT1 .

It follows that, if the system matrix switches among D1 and D2, productAkAk−1 has four distinct choices: D2

1, D22, D1D2, and D2D1. Obviously, both

D21 and D2

2 are Schur (i.e., asymptotically stable) if and only if β < 1. On theother hand, matrix

D1D2 = β

[

1 + α2 αα 1

]

is Schur if and only if β < 2/[2 + α2 +√

(2 + α2)2 − 4], in which β has toapproach zero as α becomes sufficiently large.

Such a matrix pair of D1, D2 may arise from System 2.29. For instance,consider the case of (2.31) with c > 0:

A1 =

[

−1 c0 −1

]

, A2 = AT1 , and eA1τ = e−τ

[

1 cτ0 1

]

.

Naturally, System 2.34 can exhibit similar properties.


The above discussions demonstrate that eigenvalue analysis does not gen-erally apply to time-varying systems and that stability depends explicitly uponchanges of the dynamics. In the case that all the changes are known a priori,eigenvalue analysis could be applied using the equivalent time-invariant sys-tem of (2.32) (if found). In general, stability of time-varying systems shouldbe analyzed using such tools as the Lyapunov direct method.

2.4.3 Lyapunov Analysis of Linear Systems

The Lyapunov direct method can be used to handle linear and non-linearsystems in a unified and systematic manner. Even for linear time-invariantsystems, it provides different perspectives and insights than linear tools suchas impulse response and eigenvalues. For Linear System 2.29, the Lyapunovfunction can always be chosen to be a quadratic function of form

V (x, t) = xT P (t)x,

where P (t) is a symmetric matrix and is called Lyapunov function matrix. Itstime derivative along trajectories of System 2.29 is also quadratic as

V (x, t) = −xT Q(t)x,

where matrices P (t) and Q(t) are related by the so-called differential Lya-punov equation

P (t) = −AT (t)P (t) − P (t)A(t) − Q(t). (2.37)

In the case that the system is time-invariant, Q(t) and hence P (t) can beselected to be constant, and (2.37) becomes algebraic.

To conclude asymptotic stability, we need to determine whether the twoquadratic functions xT P (t)x and xT Q(t)x are positive definite. To this end,the following Rayleigh-Ritz inequality should be used. For any symmetricmatrix H :

λmin(H)‖x‖2 ≤ xT Hx ≤ λmax(H)‖x‖2,

where λmin(H) and λmax(H) are the minimum and maximum eigenvalues ofH , respectively. Thus, stability of linear systems can be determined by check-ing whether matrices P and Q are positive definite (p.d.). Positive definitenessof a symmetric matrix can be checked using one of the following tests:

(a) Eigenvalue test: a symmetric matrix P is p.d. (or p.s.d.) if and only if allits eigenvalues are positive (or non-negative).

(b) Principal minor test: a symmetric matrix P is p.d. (or p.s.d.) if and onlyif all its leading principal minors are positive (or non-negative).

(c) Factorization test: a symmetric matrix P is p.s.d. (or p.d.) if and only ifP = WWT for some (invertible) matrix W .


(d) Gershgorin test: a symmetric matrix P is p.d. or p.s.d. if, for all i ∈1, · · · , n,

pii >

n∑

j=1,j =i

|pij | or pii ≥n∑

j=1,j =i

|pij |,

respectively.

For time-varying matrices P (t) and Q(t) , the above tests can be applied butshould be strengthened to be uniform with respect to t.

As discussed in Section 2.3.1, Lyapunov function V (x, t) and its matrixP (t) should be solved using the backward process. The following theoremprovides such a result.

Lemma 2.27. Consider System 2.29 with uniformly bounded matrix A(t).Then, it is uniformly asymptotically stable and exponentially stable if andonly if, for every uniformly bounded and p.d. matrix Q(t), solution P (t) to(2.37) is positive definite and uniformly bounded.

Proof: Sufficiency follows directly from a Lyapunov argument with Lyapunovfunction V (x, t) = xT P (t)x. To show necessity, choose

P (t) =

∫ ∞

t

ΦT (τ, t)Q(τ)Φ(τ, t)dτ, (2.38)

where Q be p.d. and uniformly bounded, and Φ(·, ·) is the state transitionmatrix in (2.30). It follows from the factorization test that P (t) is positivedefinite. Since matrix A(t) is uniformly bounded, ΦT (t, τ) is exponentiallyconvergent and uniformly bounded if and only if the system is exponentiallystable. Thus, P (t) defined above exists and is also uniformly bounded if andonly if the system is exponentially stable. In addition, it follows that

P (t) =

∫ ∞

t

∂ΦT (τ, t)

∂tQ(τ)Φ(τ, t)dτ +

∫ ∞

t

ΦT (τ, t)Q(τ)∂Φ(τ, t)

∂tdτ − Q(t)

= −AT (t)P (t) − P (t)A(t) − Q(t),

which is (2.37). This completes the proof.

To solve for Lyapunov function P (t) from Lyapunov Equation 2.37, A(t)needs to be known. Since finding P (t) is computationally similar to solvingfor state transition matrix Φ(·, ·), finding a Lyapunov function may be quitedifficult for some linear time-varying systems. For linear time-invariant sys-tems, Lyapunov function matrix P is constant as the solution to either thealgebraic Lyapunov equation or the integral given below:

AT P + PA = −Q, P =

∫ ∞

0

eAT τQeAτdτ, (2.39)

where Q is any positive definite matrix.


The Lyapunov direct method can be applied to analyze stability of lin-ear piecewise-constant systems. If System 2.20 is linear and has arbitraryswitching, a common Lyapunov function exists, but may not be quadraticas indicated by the counterexample in [49]. Nonetheless, it is shown in [158]that the common Lyapunov function can be chosen to be of the followinghomogeneous form of degree 2: for some vectors ci ∈ ℜn,

V (x) = max1≤i≤k

Vi(x), Vi(x) = xT cicTi x.

Using the notations in (2.22), a linear piecewise-constant system can be ex-pressed as

x = As(ti)x, s(ti) ∈ I. (2.40)

Assuming that As be Hurwitz for all s ∈ I, we have Vs(x) = xT Psx where Ps

is the solution to algebraic Lyapunov equation

ATs Ps + PsAs = −I.

Suppose that ti < ti+1 < tj and that s(ti) = s(tj) = p and s(ti+1) = q = p. Itfollows that, for t ∈ [ti, ti+1),

Vp = −xT x ≤ −σpVp,

and henceVp(x(ti+1)) ≤ e−σpτi+1Vp(x(ti)),

where τi+1 = ti+1 − ti is the dwell time, and σp = 1/λmax(Pp) is the timeconstant. Similarly, it follows that, over the interval [ti+1, ti+2)

Vq(x(ti+2)) ≤ e−σqτi+2Vq(x(ti+1)).

Therefore, we have

Vp(x(ti+2)) − Vp(x(ti))

≤ λmax(Pp)‖x(ti+2)‖2 − Vp(x(ti))

≤ λmax(Pp)

λmin(Pq)Vq(x(ti+2)) − Vp(x(ti))

≤ λmax(Pp)

λmin(Pq)e−σqτi+2Vq(x(ti+1)) − Vp(x(ti))

≤ λmax(Pp)

λmin(Pp)

λmax(Pq)

λmin(Pq)e−σqτi+2Vp(x(ti+1)) − Vp(x(ti))

≤ −[

1 − λmax(Pp)

λmin(Pp)

λmax(Pq)

λmin(Pq)e−σqτi+2e−σpτi+1

]

Vp(x(ti)),

which is always negative definite if either time constants σs are sufficientlysmall or dwell times τk are sufficiently long or both, while the ratios of

2.5 Controllability 69

λmax(Ps)/λmin(Ps) remain bounded. By induction and by applying Theo-rem 2.24, we know that asymptotic stability is maintained for a linear systemwhose dynamics switch relatively slowly among a finite number of relativelyfast dynamics of exponentially stable autonomous systems. This result illus-trates robustness of asymptotic stability with respect to switching, but it isquite conservative (since the above inequality provides the worst estimate ontransient overshoots of the switching system). Similar analysis can also bedone for Discrete-time System 2.36.

In the special case that matrices As are all commuting Hurwitz matrices,it has been shown in Section 2.4.2 that System 2.40 is always exponentiallystable for all values of time constants and dwell times. Furthermore, System2.40 has a quadratic common Lyapunov function in this case. For instance,consider the case that I = 1, 2 and let P1 and P2 be the solutions toLyapunov equations

AT1 P1 + P1A1 = −I, AT

2 P2 + P2A2 = −P1.

Then, P2 is the common Lyapunov function matrix since eA1τ1 and eA2τ2

commute and, by (2.39),

P2 =

∫ ∞

0

eAT2 τ2P1e

A2τ2dτ2 =

∫ ∞

0

eAT1 τ1W2e

A1τ1dτ1,

where

W2 =

∫ ∞

0

eAT2 τ2eA2τ2dτ2

is positive definite. Given any finite index set I, the above observation leadsto a recursive process of finding the quadratic common Lyapunov function[173].

2.5 Controllability

Roughly speaking, a control system is locally controllable if a steering lawfor its control can be found to move the state to any specified point in aneighborhood of its initial condition. On the other hand, Affine System 2.11can be rewritten as

x = f(x) +

m∑

i=1

gi(x)ui =

m∑

i=0

gi(x)ui, (2.41)

where x ∈ ℜn, u0 = 1, g0(x)= f(x), and vector fields gi(x) are assumed to be

analytic and linearly independent. Thus, controllability should be determinedby investigating time evolution of the state trajectory of (2.41) under allpossible control actions.


To simplify the notation, we introduce the so-called Lie bracket: for anypair of f(x), g(x) ∈ ℜn, [f, g] or adfg (where ad stands for “adjoint”) is avector function of the two vector fields and is defined by

[f, g] = (Tx g)f − (T

x f)g.

High-order Lie brackets can be defined recursively as, for i = 1, · · ·,

ad0fg = g, and adi

fg = [f, adi−1f g].

An important property associated with the Lie derivative and Lie bracket isthe so-called Jacobi identity: for any f(x), g(x) ∈ ℜn and ξ(x) ∈ ℜ,

Ladfgξ = LfLgξ − LgLfξ, (2.42)

and it can easily be verified by definition. The following are the conceptsassociated with Lie bracket.

Definition 2.28. A set of linearly independent vector fields ξl(x) : l =1, · · · , k is said to be involutive or closed under Lie bracket if, for all i, j ∈1, · · · , k, the Lie bracket [ξi, ξj ] can be expressed as a linear combination ofξ1 up to ξk.

Definition 2.29. Given smooth linearly independent vector fields ξl(x) : l =1, · · · , k, the tangent-space distribution is defined as

∆(x) = spanξ1(x), ξ2(x), · · · , ξk(x).

∆(x) is regular if the dimension of ∆(x) does not vary with x. ∆(x) is invo-lutive if [f, g] ∈ ∆ for all f, g ∈ ∆. Involutive closure ∆ of ∆(x), also calledLie algebra, is the smallest distribution that contains ∆ and is closed underLie bracket.

While Lie brackets of gi(x) do not have an explicit physical meaning, thespan of their values consists of all the incremental movements achievable forSystem 2.41 under modulated inputs. For instance, consider the followingtwo-input system:

x = g1(x)u1 + g2(x)u2, x(0) = x0.

Should piecewise-constant inputs be used, there are only two linearly inde-pendent values for the input vector at any fixed instant of time, and they canbe used to construct any steering control input over time. For instance, thefollowing is a piecewise-constant steering control input:

[

u1(t) u2(t)]T

=

⎧

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎩

[

1 0]T

t ∈ (0, ε][

0 1]T

t ∈ (ε, 2ε][

−1 0]T

t ∈ (2ε, 3ε][

0 −1]T

t ∈ (3ε, 4ε]

,

2.5 Controllability 71

g1 g

2

net motion

1g 1g

2g

2g

Fig. 2.1. Incremental motion in terms of Lie bracket

where ε > 0 is some sufficiently small constant. One can calculate the second-order Taylor series expansions of x(iε) recursively for i = 1, 2, 3, 4 and showthat

[g1, g2](x0) = limh→0

x(4ε) − x0

ε2.

As shown by Fig. 2.1, Lie bracket [g1, g2](x0) is the net motion generatedunder the steering control. In general, controllability of Affine System 2.11can be determined in general by a rank condition on its Lie algebra, as statedby the following theorem [40, 114].

Definition 2.30. A system is controllable if, for any two points x0, xf ∈ℜn, there exist a finite time T and a control u(x, t) such that the systemsolution satisfies x(t0) = x0 and x(t0 + T ) = xf . The system is small-timelocally controllable at x1 if u(x, t) can be found such that x(t0 + δt) = x1 forsufficiently small δt > 0 and that x(t) with x(t0) = x0 stays near x1 at alltimes.

Theorem 2.31. Affine System 2.41 is small-time locally controllable at x ifthe involutive closure of

∆(x) = f(x), g1(x), · · · , gm(x)

is of dimension n, that is, the rank of the controllability Lie algebra is n.

If an affine system is not controllable, controllability decomposition canbe applied to separate controllable dynamics from uncontrollable dynamics.For non-affine systems, conditions could be obtained using local linearizationwith respect to control u. More details on non-linear controllability can befound in texts [12, 96, 175, 223]. For Linear System 2.25, the controllabilityLie algebra reduces to


g, adfg, · · · , adn−1f g = B, AB, · · · , An−1B,

which is the linear controllability matrix C =[

B AB A2B · · · An−1B]

. And,System 2.25 is said to be in the controllable canonical form if the matrix pair(A, B) is given by

Ac=

⎡

⎢

⎢

⎢

⎢

⎢

⎣

0 1 0 · · · 00 0 1 · · · 0...

......

. . ....

0 0 0 · · · 10 0 0 · · · 0

⎤

⎥

⎥

⎥

⎥

⎥

⎦

, Bc=

⎡

⎢

⎢

⎢

⎢

⎢

⎣

00...01

⎤

⎥

⎥

⎥

⎥

⎥

⎦

. (2.43)

Consider the linear time-varying system:

x = A(t)x + B(t)u, , (2.44)

where matrices A(t) and B(t) are uniformly bounded and of proper dimension.Their controllability can directly be studied using its solution

x(t) = Φ(t, t0)x(t0) +

∫ t

t0

Φ(t, τ)B(τ)u(τ)dτ

= Φ(t, t0)

[

x(t0) +

∫ t

t0

Φ(t0, τ)B(τ)u(τ)dτ

]

.

It is straightforward to verify that, if the matrix

Wc(t0, tf) =

∫ tf

t0

Φ(t0, τ)B(τ)BT (τ)ΦT (t0, τ)dτ (2.45)

is invertible, the state can be moved from any x0 to any xf under control

u(t) = −BT (t)ΦT (t0, t)W−1c (t0, tf )[x0 − Φ(t0, tf )xf ].

Matrix Wc(t0, tf ) is the so-called controllability Gramian, and it is easy toshow [39] that Wc(t0, tf ) is invertible and hence System 2.44 is controllableif and only if the n rows of n-by-m matrix function Φ(t0, t)B(t) are linearindependent over time interval [t0, tf ] for some tf > t0. To develop a con-trollability condition without requiring the solution of state transition matrixΦ(t0, t), it follows that

∂k

∂tkΦ(t0, t)B(t) = Φ(t0, t)Ek(t), k = 0, 1, · · · , (n − 1),

where E0(t) = B(t) and Ek+1(t) = −A(t)Ek(t) + Ek(t) for k = 0, 1, · · · , (n −1). Hence, controllability of System 2.44 is equivalent to the Kalman rankcondition of

rank[

E0(t) E1(t) · · · En−1(t)]

= n,

2.6 Non-linear Design Approaches 73

Comparing (2.45) and (2.38), we know that, under the choice of posi-tive semi-definite matrix Q(t) = B(t)BT (t), Lyapunov Equation 2.37 stillyields positive definite Lyapunov function P (t) provided that System 2.44is controllable and that its uncontrolled dynamics in (2.29) are asymptoti-cally stable. To design a Lyapunov-based control and to ensure global anduniformly asymptotic stability, it is necessary that the system solution is uni-formly bounded and that Lyapunov function matrix P (t) exists and is alsouniformly bounded, which leads to the following definition and theorem [103].Later in Section 2.6.3, Control 2.46 below is shown to be optimal as well.

Definition 2.32. System 2.44 is said to be uniformly completely controllableif the following two inequalities hold for all t:

0 < αc1(δ)I ≤ Wc(t, t + δ) ≤ αc2(δ)I, ‖Φ(t, t + δ)‖ ≤ αc3(δ),

where Wc(t0, tf ) is defined by (2.45), δ > 0 is a fixed constant, and αci(·) arefixed positively-valued functions.

Theorem 2.33. Consider System 2.44 under control

u = −R−1(t)BT (t)P (t)x, (2.46)

where matrices Q(t) and R(t) are chosen to be positive definite and uniformlybounded, and matrix P (t) is the solution to the differential Riccati equation

P + [PA + AT P − PBR−1BT P + Q] = 0 (2.47)

with terminal condition of P (∞) being positive definite. Then, if System 2.44is uniformly completely controllable, Lyapunov function V (x, t) = xT P (t)x ispositive definite and decrescent, and Control 2.46 is asymptotically stabilizing.

2.6 Non-linear Design Approaches

In this section, three popular design methods are outlined for non-linear sys-tems; they are backstepping design, feedback linearization, and optimal con-trol.

2.6.1 Recursive Design

In this section, we focus upon the following class of feedback systems:⎧

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎩

x1 = f1(x1, t) + x2,x2 = f2(x1, x2, t) + x3,...xn = fn(x1, · · · , xn, t) + u,

(2.48)


where xi ∈ ℜl, x1 is the output, and u is the input. Letting xn up to x1

be the outputs of a chain of pure integrators, we can graphically connectthe inputs to the integrators by using dynamic equations in (2.48). In theresulting block diagram, there is a feedforward chain of integrators, and allother connections are feedback. In particular, the ith integrator has xi+1 andxi as its input and output, respectively. Should fi(x1, · · · , xi, t) = fi(xi, t),System 2.48 would become a cascaded chain of first-order non-linear sub-systems. As such, System 2.48 is also referred to as a cascaded system. Severalclasses of physical systems, especially electromechanical systems [130, 195,242], have by nature this structural property on their dynamics, and theircontrollability is guaranteed.

The aforementioned structural property of cascaded dynamics provides anatural and intuitive way for finding both Lyapunov function and a stabilizingcontrol. The systematic procedure, called backstepping or backward recursivedesign [72, 116, 192], is a step-by-step design process in which the first-ordersub-systems are handled one-by-one and backwards from the output x1 backto the input u. Specifically, let us consider the case of n = 2 and begin withthe first sub-system in (2.48), that is,

x1 = f1(x1, t) + x2.

If x2 were a control variable free to be selected, a choice of x2 would easily befound to stabilize the first sub-system. Because x2 is a state variable ratherthan a control, we rewrite the first sub-system as

x1 = f1(x1, t) + xd2(x1, t) + [x2 − xd

2(x1, t)],

and choose the fictitious control xd2(x1, t) (such as the most obvious choice

of xd2(x1, t) = −f1(x1, t) − x1) to stabilize asymptotically fictitious system

x1 = f1(x1, t) + xd2(x1, t) by ensuring the inequality

2xT1 [f1(x1, t) + xd

2(x1, t)] ≤ −‖x1‖2.

Thus, choosing Lyapunov sub-function V1(x1) = ‖x1‖2, we have that, alongthe trajectory of (2.48),

V1 ≤ −2‖x1‖2 + 2xT1 z2,

where z2 = x2 − xd2(x1, t) is a transformed state variable of x2. As shown in

Section 2.3.2, the above inequality of V1 implies that, if z2 is asymptoticallyconvergent, so is x1. To ensure asymptotic convergence of z2, we know fromits definition that

z2 = u − ∂xd2

∂t−(

∂xd2

∂x1

)T

[f1(x1, t) + x2].

As before, the above dynamic equation of z2 is of first-order and hence controlu can be easily found (for instance, the choice rendering z2 = −z2) such that,with V2(x1, x2) = ‖z2‖2,


V2 ≤ −2‖z2‖2.

Therefore, we now have found Lyapunov function V = V1 + α2V2 whose timederivative is

V = −2‖x1‖2 + 2xT1 z2 − 2α2‖z2‖2,

which is negative definite for any choice of α2 > 1/4. Hence, under the con-trol u selected, both x1 and z2 and consequently both x1 and x2 are globallyasymptotically stable. It is straightforward to see that, by induction, a stabiliz-ing control and the corresponding Lyapunov function can be found recursivelyfor System 2.48 of any finite-order n.

2.6.2 Feedback Linearization

The feedback linearization approach provides the conditions under which apair of state and control transformations exist such that a non-linear systemis mapped (either locally or globally) into the linear controllable canonicalform. Specifically, the objective of feedback linearization is to map System2.11 into the form

z = Acz + Bcv,

w = φ(z, w), (2.49)

where the pair Ac, Bc is that in (2.43), z(x) ∈ ℜr is the state of the feedbacklinearized sub-system, r is the so-called relative degree of the system, w isthe state of so-called internal dynamics, [zT wT ] = [zT (x) wT (x)] is thestate transformation, and v = v(u, x) is the control transformation. Then,standard linear control results can be applied through the transformationsto the original non-linear system provided that the internal dynamics areminimum phase (i.e., the zero dynamics of w = φ(0, w) are asymptoticallystable).

By (2.49), System 2.11 with m = 1 is feedback linearizable if functionh(x) ∈ ℜ exists such that

z1 = h(x); zi = zi+1, i = 1, · · · , r − 1; zr = v; w = φ(z, w) (2.50)

for some vector functions w(x) and φ(·) and for transformed control v(u, x).It follows from dynamics of System 2.11 that zi = Lfzi + (Lgzi)u. Therefore,equations of (2.50) are equivalent to

⎧

⎨

⎩

Lgwj = 0, j = 1, · · · , n − r;

LgLi−1f h = 0, i = 1, · · · , r − 1;

LgLrfh = 0,

(2.51)

while the control transformation and linearized state variables are defined by

v = Lfzr + (Lgzr)u; z1 = h(x), zi+1 = Lfzi i = 1, · · · , r − 1. (2.52)


By Jacobi Identity 2.42, Lgh = LgLfh = 0 if and only if Lgh = Ladfgh = 0.By induction, we can rewrite the partial differential equations in (2.51) as

⎧

⎨

⎩

Lgwj = 0, j = 1, · · · , n − r;Ladi

fgh = 0, i = 1, · · · , r − 1;

LgLrfh = 0.

(2.53)

In (2.53), there are (n − 1) partial differential equations that are all homo-geneous and of first-order. Solutions to these equations can be found underrank-based conditions, and they are provided by the following theorem oftenreferred to as the Frobenius theorem [96]. For control design and stabilityanalysis, the transformation from x to [zT , wT ]T needs to be diffeomorphic(i.e., have a unique inverse), which is also ensured by Theorem 2.34 since, byimplicit function theorem [87], the transformation is diffeomorphic if its Ja-cobian matrix is invertible and since the Jacobian matrix consists of qj(x),the gradients of the solutions.

Theorem 2.34. Consider k(n−k) first-order homogeneous partial differentialequations:

Lξiqj = 0, i = 1, · · · , k; j = 1, · · · , n − k, (2.54)

where ξ1(x), ξ2(x), · · · , ξk(x) is a set of linearly independent vectors in ℜn,and qj(x) are the functions to be determined. Then, the solutions qj(x) to(2.54) exist if and only if distribution ξ1(x), ξ2(x), · · · , ξk(x) is involutive.Moreover, under the involutivity condition, the gradients of the solutions,qj(x), are linearly independent.

Applying Theorem 2.34 to (2.53), we know that System 2.11 is feedbacklinearizable with relative degree r = n (i.e., full state feedback linearizable)if the set of vector fields g, adfg, · · · , adn−2

f g is involutive and if matrix

[g adfg · · · adn−1f g] is of rank n. Upon verifying the conditions, the first state

variable z1 = h(x) can be found such that Ladifgh = 0 for i = 1, · · · , n− 1 but

LgLnf h = 0, the rest of state variables and the control mapping are given by

(2.52). The following theorem deals with the case of m ≥ 1 and r = n, and asimilar result can be applied for the general case of r < n [96].

Theorem 2.35. System 2.11 is feedback linearizable (i.e., can be mapped intothe linear controllable canonical form) under a diffeomorphic state transfor-mation z = T (x) and a control mapping u = α(x) + β(x)v if and onlyif the nested distributions defined by D0 = span

g1 · · · gm

and Di =

Di−1 + span

adifg1 · · · adi

fgm

with i = 1, · · · , n − 1 have the propertiesthat Dl are all involutive and of constant rank for 0 ≤ l ≤ n − 2 and thatrank Dn−1 = n.

As shown in Section 2.5, matrix [g adfg · · · adn−1f g] having rank n

is ensured by non-linear controllability. The involutivity condition on set


g adfg · · · adn−2f g ensures the existence and diffeomorphism of state and

control transformations but does not have a clear intuitive explanation. Notethat the involutivity condition is always met for linear systems (since the setconsists of constant vectors only).

2.6.3 Optimal Control

For simplicity, consider Affine System 2.11 and its optimal control problemover the infinite horizon. That is, our goal is to find control u such that thefollowing performance index

J(x(t0)) =

∫ ∞

t0

L(x(t), u(t))dt (2.55)

is minimized, subject to Dynamic Equation 2.11 and terminal condition ofx(∞) = 0. There are two approaches to solve the optimal control problem:the Euler-Lagrange method based on Pontryagin minimum principle, and theprinciple of optimality in dynamic programming.

By using a Lagrange multiplier, the Euler-Lagrange method converts anoptimization problem with equality constraints into one without any con-straint. That is, System Equation 2.11 is adjoined into performance index Jas

J (t0) =

∫ ∞

t0

L(x, u) + λT [f(x) + g(x)u− x]dt =

∫ ∞

t0

[H(x, u, λ, t)−λT x]dt,

where λ ∈ ℜn is the Lagrange multiplier, and

H(x, u, λ, t) = L(x, u) + λT [f(x) + g(x)u] (2.56)

is the Hamiltonian. Based on calculus of variations [187], J is optimized locallyby u if δ1J = 0, where δ1J is the first-order variation of J due to variationδu in u and its resulting state variation δx in x. Integrating in part the termcontaining x in J and then finding the expression of its variation, one canshow that δ1J = 0 holds under the following equations:

λ = −∂H

∂x= −∂L

∂x−(

∂f

∂x

)T

λ −n∑

i=1

nu∑

j=1

λi∂gij

∂xuj , λ(∞) = 0, (2.57)

0 =∂H

∂u=

∂L

∂u+ gT λ, or H(x, λ, u) = min

uH(x, λ, u). (2.58)

If instantaneous cost L(·) is a p.d. function, optimal control can be solvedfrom (2.58). Equation 2.58 is referred to as the Pontryagin minimum prin-ciple. Equation 2.57 is the so-called costate equation; it should be solvedsimultaneously with State Equation 2.11. Equations 2.11 and 2.57 representa two-point boundary-value problem, and they are necessary for optimality


but generally not sufficient. To ensure that the value of J is a local minimum,its second-order variation δ2J must be positive, that is, the following Hessianmatrix of Hamiltonian H should be positive definite:

HH=

⎡

⎢

⎢

⎣

∂2H

∂x2

∂2H

∂x∂u

∂2H

∂u∂x

∂2H

∂u2

⎤

⎥

⎥

⎦

.

For the linear time-varying system in (2.44), it is straightforward to verifythat λ = P (t)x and J (t0) = V (x(t0), t0) and that Control 2.46 is optimal.

Alternatively, an optimal control can be derived by generalizing Perfor-mance Index 2.55 into

J∗(x(t)) = infu

∫ ∞

t

L(x(τ), u(τ))dτ,

which can be solved using dynamic programming. The principle of optimalitystates that, if u∗ is the optimal control under Performance Index 2.55, controlu∗ is also optimal with respect to the above measure for the same system butwith initial condition x(t) = x∗(t). Applying the principle of optimality toJ∗(x(t + δt)) and invoking Taylor series expansion yield the following partialdifferential equation:

minu

H(x, u, λ)∣

∣

∣

λ = xJ∗(x)= −t J∗(x) = 0. (2.59)

which is so-called Hamilton-Jacobi-Bellman (HJB) equation [10, 16, 33, 97].For Affine System 2.11 and under the choice of L(x, u) = α(x) + ‖u‖2, the

HJB equation reduces to

LfJ∗ − 1

4‖LgJ

∗‖2 + α(x) = 0, (2.60)

where α(x) ≥ 0 is any scalar function. If the above partial differential equa-tion has a continuously differentiable and positive definite solution J∗(x), theoptimal control is

u∗(x) = −1

2(LgJ

∗)T ,

under which the closed-loop system is asymptotically stable. However, solvingHJB Equation 2.60 in general is quite difficult even for Affine System 2.11.

2.6.4 Inverse Optimality and Lyapunov Function

Solving HJB Equation 2.60 is equivalent to finding control Lyapunov functionV (x). If J∗ is known, the Lyapunov function can be set as V (x) = J∗(x), andthe optimal control is stabilizing since


LfJ∗ + (LgJ∗)u∗ = −1

4‖LgJ

∗‖2 − α(x) ≤ 0.

On the other hand, given a control Lyapunov function V (x), a class of stabi-lizing controls for System 2.11 can be found, and performance under any ofthese controls can be evaluated. For instance, consider Control 2.19 in Section2.3.3. Although optimal value function J∗(x) is generally different from V (x),we can assume that, for some scalar function k(x), xJ∗ = 2k(x)x V . Thatis, in light of stability outcome, J∗ has the same level curves as those of V .Substituting the relationship into (2.60) yields

2k(x)LfV − k2(x)‖LgV ‖2 + α(x) = 0,

which is a quadratic equation in k(x). Choosing the solution with positivesquare root, we know that

k(x) =LfV +

√

(LfV )2 + ‖LgV ‖2α(x)

‖LgV ‖2

and that optimal control u∗(x) reduces to Control 2.19. That is, for any choiceof α(x), Control 2.19 is optimal with respect to J∗, which is called inverseoptimality [72, 234].

In the special case of linear time-invariant systems, the relationship be-tween the Lyapunov-based control design and inverse optimality is morestraightforward. Should System 2.25 be controllable, a stabilizing control ofgeneral form u = −Kx exists such that A − BK is Hurwitz. For any pair ofp.d. matrices Q ∈ ℜn×n and R ∈ ℜm×m, matrix sum (Q + KT RK) is p.d.and hence, by Lemma 2.27, solution P to Lyapunov equation

P (A − BK) + (A − BK)T P = −(Q + KT RK)

is p.d. Thus, it follows from System 2.25 under control u = −Kx that

xT (t)Px(t) = −∫ ∞

t

dxT Px

dτdτ =

∫ ∞

t

(xT Qx + uT Ru)dτ.

Thus, every stabilizing control is inversely optimal with respect to a quadraticperformance index, and the optimal performance value over [t,∞) is the cor-responding quadratic Lyapunov function.


The Lyapunov direct method [141] is a universal approach for both stabil-ity analysis and control synthesis of general systems, and finding a Lyapunovfunction is the key [108, 264]. As shown in Sections 2.3.2 and 2.6, the Lya-punov function [141] or the pair of control Lyapunov function [235, 236] and


the corresponding controller can be searched for by analytically exploitingsystem properties, that is, a control Lyapunov function [8] can be constructedfor systems of special forms [261], for systems satisfying the Jurdejevic-Quinnconditions [61, 152], for feedback linearizable systems [94, 96, 99, 175], andfor all the systems to which recursive approaches are applicable [116, 192].Numerically, a Lyapunov function can also be found by searching for its dualof density function [210] or the corresponding sum-of-square numerical rep-resentation in terms of a polynomial basis [190]. Alternative approaches arethat a numerical solution to the HJB equation yields the Lyapunov functionas an optimal value function [113] and that set-oriented partition of the statespace renders discretization and approximation to which graph theoretical al-gorithms are applicable [81]. For linear control systems, the problem reducesto a set of linear matrix inequalities (LMI) which are convex and can be solvedusing semi-definite programming tools [27], while linear stochastic systems canbe handled using Perron-Frobenius operator [121].

Under controllability and through the search of control Lyapunov func-tion, a stabilizing control can be designed for dynamic systems. In Chapter3, several vehicle-level controls are designed for the class of non-holonomicsystems that include various vehicles as special cases. To achieve cooperativebehaviors, a team of vehicles needs to be controlled through a shared sens-ing/communication network. In Chapter 4, a mathematical representation ofthe network is introduced, and a matrix-theoretical approach is developed byextending the results on piecewise-constant linear systems in Section 2.4.2.In Chapter 5, cooperative stability is defined, and the matrix-theoretical ap-proach is used to study cooperative controllability over a network, to designa linear cooperative control, and to search for the corresponding control Lya-punov function. Due to the changes in the network, a family of control Lya-punov functions would exist over consecutive time intervals in a way similarto those in Theorem 2.24. Since the networked changes are uncertain, the con-trol Lyapunov functions cannot be determined and their changes over timecannot be assessed. Nonetheless, all the control Lyapunov functions are al-ways quadratic and have the same square components, and these componentscan be used individually and together as a vector of Lyapunov function com-ponents to study stability. This observation leads to the Lyapunov functioncomponent-based methodology which is developed in Chapter 6 to extendboth Theorems 2.10 and 2.24 and to analyze and synthesize non-linear coop-erative systems.

3

Control of Non-holonomic Systems

Analysis and control of non-holonomic systems are addressed in this chapter.In order to proceed with the analysis and control design in a more system-atic way, the so-called chained form is introduced as the canonical form fornon-holonomic systems. As examples, kinematic equations of several vehi-cles introduced in Chapter 1 are transformed into the chained form. Then,a chained system is shown to be non-linearly controllable but not uniformlycompletely controllable in general, and it is partially and dynamically feed-back linearizable, but can only be stabilized under a discontinuous and/ortime-varying control. Based on these properties, open-loop steering controlsare synthesized to yield a trajectory for the constrained system to move eithercontinually or from one configuration to another. To ensure that a continualand constrained trajectory is followed, feedback tracking controls can be de-signed and implemented with the steering control. To move the system to aspecific configuration, stabilizing controls can be used. For a vehicle system,the formation control problem is investigated. To make the vehicle systemcomply with environmental changes, either a real-time optimized path plan-ning algorithm or a multi-objective reactive control can be deployed.

3.1 Canonical Form and Its Properties

In this section, the chained form is introduced as the canonical form for thekinematic constraints of non-holonomic systems. It is shown that, throughstate and control transformations, the kinematic sub-system of a non-holonomicsystem can be mapped into the chained form or one of its extensions. Vehiclemodels introduced in Chapter 1 are used as examples to determine the stateand control transformations. Properties of the chained form are studied forthe purpose of systematic control designs in the subsequent sections. In par-ticular, controllability, feedback linearizability, existence of smooth control,and uniform complete controllability are detailed for chained form systems.

82 3 Control of Non-holonomic Systems

3.1.1 Chained Form

The n-variable single-generator m-input chained form is defined by [169, 266]:⎧

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎩

x1 = u1,x21 = u1x22, · · · , x2(n2−1) = u1x2n2 , x2n2 = u2,...xm1 = u1xm2, · · · , xm(nm−1) = u1xmnm

, xmnm= um,

(3.1)

where x = [x1, x21, · · · , x2n2 , · · · , xm1, · · · , xmnm]T ∈ ℜn is the state, u =

[u1, · · · , um]T ∈ ℜm is the control, and y = [x1, x21, · · · , xm1]T ∈ ℜm is the

output. If m = 2, Chained Form 3.1 reduces to

x1 = u1, x2 = u1x3, · · · , xn−1 = u1xn, xn = u2. (3.2)

Since analysis of and control design for (3.1) and (3.2) are essentially identical,we will focus mostly upon the two-input chained form. Through reorderingthe state variables, (3.2) renders its alternative expressions:

z1 = u1, z2 = u2, z3 = z2u1, · · · , zn = zn−1u1, (3.3)

where z1 = x1, and zj = xn−j+2 for j = 2, · · · , n.Given a mechanical system subject to non-holonomic constraints, it is

often possible to convert its constraints into the chained form either locallyor globally by using a coordinate transformation and a control mapping. Thetransformation process into the chained form and the conditions are parallel tothose of feedback linearization studied in Section 2.6.2. For example, considera two-input kinematic system in the form of (1.12), that is,

q = g1(q)v1 + g2(q)v2, (3.4)

where q ∈ ℜn with n > 2, vector fields g1 and g2 are linearly independent,and v = [v1, v2]

T ∈ ℜ2 is the vector of original control inputs. Should System3.4 be mapped into Chained Form 3.2, we know from direct differentiationand Jacobi Identity 2.42 that x1 = h1(q) and x2 = h2(q) exist and satisfy thefollowing conditions:

⎧

⎪

⎪

⎨

⎪

⎪

⎩

(

∂h1

∂q

)T

∆1 = 0,

(

∂h1

∂q

)T

g1 = 1,(

∂h2

∂q

)T

∆2 = 0,

(

∂h2

∂q

)T

adn−2g1

g2 = 0,

(3.5)

where ∆j(q) with j = 0, 1, 2 are distributions defined by

∆0(q)= spang1, g2, adg1g2, · · · , adn−2

g1g2,

∆1(q)= spang2, adg1g2, · · · , adn−2

g1g2,

∆2(q)= spang2, adg1g2, · · · , adn−3

g1g2.

3.1 Canonical Form and Its Properties 83

Under Condition 3.5, System 3.4 can be mapped into Chained Form 3.2 understate transformation

x =[

h1 h2 Lg1h2 · · · Ln−2g1

h2

]T = Tx(q)

and control transformation

u =

[

1 0Ln−1

g1h2 Lg2L

n−2g1

h2

]

v= Tu(q)v.

The state transformation corresponding to Chained Form 3.3 is

z =[

h1 Ln−2g1

h2 · · · Lg1h2 h2

]T = Tz(q).

Applying Theorem 2.34 to the first-order homogeneous partial differentialequations in (3.5), we know that transformations Tx(q) and Tu(q) exist andare diffeomorphic for Systems 3.4 and 3.2 if and only if both ∆1 and ∆2 ⊂ ∆1

are involutive, ∆0(q) is of dimension of n and hence also involutive, and h1(q)is found to yield (∂h1/∂q)T g1 = 1. Similarly, the following theorem can beconcluded [266] as the sufficient conditions under which a driftless system canbe transformed into Chained Form 3.1.

Theorem 3.1. Consider the driftless non-holonomic system

q =

m∑

i=1

gi(q)vi, (3.6)

where q ∈ ℜn, vi ∈ ℜ, and vector fields of gi are smooth and linearly indepen-dent. Then, there exist state transformation x = Tx(q) and control mappingu = Tu(q)v to transform System 3.6 into Chained Form 3.1 if there existfunctions h1, · · · , hm of form

hi =[

hi1(q) hi2(q) · · · hin(q)]T

,

h11 = 1hj1 = 0

,

i = 1, · · · , mj = 2, · · · , m

such that the distributions

∆k= spanadi

h1h2, · · · , adi

h1hm : 0 ≤ i ≤ k, 0 ≤ k ≤ n − 1 (3.7)

have constant dimensions, are all involutive, and ∆n−1 is of dimension (n−1).

In what follows, several non-holonomic vehicle models derived in Chapter1 are converted into Chained Form 3.1 .

Differential-drive Vehicle

Consider Kinematic Model 1.31, and choose h1(q) = θ and h2(q) = x sin θ −y cos θ. It is straightforward to verify that Condition 3.5 holds. Therefore, thecoordinate transformation is


z1 = θ, z2 = x cos θ + y sin θ, z3 = x sin θ − y cos θ (3.8)

and the control mapping is

u1 = v2 + z3v1, u2 = v1. (3.9)

Under the pair of diffeomorphic transformations, Kinematic Model 1.31 isconverted into the following chained form:

z1 = v1, z2 = v2, z3 = z2v1.

Car-like Vehicle

Consider Kinematic Model 1.35. Condition 3.5 holds under the choices ofh1(q) = x and h2(q) = y. It is straightforward to verify that KinematicEquation 1.35 is converted into the following chained form

z1 = v1, z2 = v2, z3 = z2v1, z4 = z3v1. (3.10)

under the state transformation

z1 = x, z2 =tan(φ)

l cos3(θ), z3 = tan(θ), z4 = y, (3.11)

and control mapping

u1 =v1

ρ cos(θ), u2 = − 3 sin(θ)

l cos2(θ)sin2(φ)v1 + l cos3(θ) cos2(φ)v2. (3.12)

Mappings 3.11 and 3.12 are diffeomorphic in the region where θ ∈ (−π/2, π/2).

Fire Truck

Consider Kinematic Model 1.40, and select the following vector fields:

h1 =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎣

1tan θ1

01lf

sec θ1 tanφ1

0− 1

lbsec θ1 secφ2 sin(φ2 − θ1 + θ2)

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎦

, h2 =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎣

001000

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎦

, h3 =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎣

000010

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎦

.

It is routine to check that, excluding all the singular hyperplanes of θ1 − θ2 =φ1 = φ2 = θ1 = π/2, the distributions of ∆k defined in Theorem 3.1 all haveconstant dimensions and are involutive for k = 0, 1, · · · , 5 and that ∆5 is ofdimension 5. Thus, Kinematic Equation 1.40 is transformed into the chainedform of


⎧

⎨

⎩

z1 = v1

z21 = v2, z22 = z21v1, z23 = z22v1,z31 = v3, z32 = z31v1.

Indeed, the corresponding state and control transformations are [34]

⎧

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎩

z1 = x1,

z21 =1

lftan φ1 sec3 θ1, z22 = tan θ1, z23 = y1,

z31 = − 1

lbsin(φ2 − θ1 + θ2) sec φ2 sec θ1, z32 = θ2,

and⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

v1 = u1 cos θ1,

v2 =3

l2ftan2 φ1 tan θ1 sec4 θ1v1 +

1

l2fsec2 φ1 sec3 θ1u2,

v3 =1

lf lbcos(φ2 + θ2) tanφ1 secφ2 sec3 θ1v1

+1

l2bcos(φ2 − θ1 + θ2) sin(φ2 − θ1 + θ2) sec2 φ2 sec2 θ1v1

− 1

lfcos(θ2 − θ1) sec2 φ2 sec θ1u3.

3.1.2 Controllability

Controllability of a dynamic system provides a definitive answer to the ques-tion whether the state can be driven to a specific point from any (nearby)initial condition and under an appropriate choice of control. As shown inSection 2.5, there are two basic approaches to check controllability. Given anon-linear system, we can determine linear controllability by first linearizingthe non-linear system at an equilibrium point (typically assumed to be theorigin) or along a given trajectory and then invoking the Kalman rank condi-tion or calculating the controllability Gramian for the resulting linear system(which is either time-invariant or time-varying). Alternatively, we can deter-mine non-linear controllability by simply applying Theorem 2.31 which, alsoknown as Chow’s theorem, is essentially a rank condition on the Lie brack-ets of vector fields of the system. Note that a controllable non-linear systemmay not have a controllable linearization, which is illustrated by the followingsimple example.

Example 3.2. Consider a differential-drive vehicle whose model is

x1 = u1 cosx3, x2 = u1 sin x3, x3 = u2, (3.13)

which yields


g1(x) =[

cosx3 sin x3 0]T

, g2(x) =[

0 0 1]T

. (3.14)

Its linearized system at the origin is

x = Ax + Bu, A = 0, B =

⎡

⎣

1 00 00 1

⎤

⎦ ,

which is not controllable. On the other hand, it follows from

adg1g2 =[

sinx3 − cosx3 0]

that rankg1, g2, adg1g2 = 3. That is, the system is (globally non-linearly)controllable.

Physical explanation of Example 3.2 is that, due to non-holonomic con-straints, the wheeled vehicle is not allowed to move sideways and hence is notlinearly controllable but it can accomplish parallel parking through a series ofmaneuvers and thus is non-linearly controllable. As pointed out in Section 2.5,motion can always be accomplished in the sub-space spanned by Lie bracketsof vector fields. As such, the basic conclusions in Example 3.2 hold in generalfor non-holonomic systems. To see this, consider Chained System 3.2 whosevector fields are

g1(x) =[

1 x3 · · · xn 0]T

, g2(x) =[

0 0 · · · 0 1]

. (3.15)

Every point in the state space is an equilibrium point of System 3.2, and itslinearized system (at any point) is

x = Ax + Bu, A = 0, B =

[

1 0 · · · 0 00 0 · · · 0 1

]T

,

which is not controllable. Direct computation yields

adg1g2 =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎣

00...0−10

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎦

, ad2g1

g2 =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎣

00...100

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎦

, · · · , adn−2g1

g2 =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎣

0(−1)n−2

0...00

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎦

, (3.16)

which implies that rankg1, g2, adg1g2, · · · , adn−2g1

g2 = n globally. Hence,Chained System 3.2 is (globally non-linearly) controllable. The following factscan be further argued [169]:

(a) The presence of holonomic constraints makes a constrained system notcontrollable due to the fact that the system cannot move to any pointviolating the holonomic constraints.

(b) Once the holonomic constraints are met through order reduction, thereduced-order system becomes controllable.

(c) A constrained system only with non-holonomic constraints is always con-trollable.


3.1.3 Feedback Linearization

Should a non-linear system be feedback linearizable, its control design becomessimple. In what follows, non-holonomic systems are shown not to be feedbacklinearizable, but the feedback linearization technique is still applicable undercertain circumstances.

Feedback Linearization

As discussed in Section 2.6.2, the non-linear system

x = f(x) + g(x)u (3.17)

is feedback linearizable if it can be transformed into the linear controllablecanonical form under a state transformation z = T (x) and a static controlmapping u = α(x) + β(x)v. If the relative degree is less than the systemdimension, the transformed system also contains non-linear internal dynamicsas

z = Acz + Bcv, w = φ(z, w), (3.18)

where[

zT wT]T

= T (x) is the state transformation.According to Theorem 2.35, Chained System 3.2 (which has no drift as

f = 0) is feedback linearizable if and only if g1(x), g2(x) is involutive. Itfollows from (3.15) and (3.16) that adg1g2 ∈ spang1, g2 and hence g1, g2is not involutive. Thus, Chained System 3.2 is not feedback linearizable. Infact, for any driftless non-holonomic system in the form of (3.6) and withn > m, vector fields are never involutive because rankg1, · · · , gm = m < nwhile controllability implies rankg1, g2, adg1g2, · · · , adn−2

g1g2 = n. Therefore,

it is concluded that any driftless non-holonomic system must not be feedbacklinearizable.

Partial Feedback Linearization over a Region

System 3.17 is said to be partially feedback linearizable if, under a partial statetransformation z = T ′(x) and a static control mapping u = α(x) + β(x)v, itcan be mapped into the following pseudo-canonical form [37]:

z = Acz + Bcv1, w = φ1(z, w) + φ2(z, w)v, (3.19)

where[

zT wT]T

= T (x) is the state transformation, and v =[

vT1 vT

2

]

withv1 ∈ ℜp and v2 ∈ ℜm−p. The key differences between feedback linearizationand partial feedback linearization can be seen by comparing (3.18) and (3.19);that is, in partial feedback linearization, the linearized sub-system does notcontain all the control inputs, and the non-linear internal dynamics are allowedto be driven by the transformed control v. Conditions on partial feedback


linearization and the largest feedback linearizable sub-system can be found in[144].

Non-linear dynamics of Chained System 3.2 are partially feedback lineariz-able not in any neighborhood around the origin, but in a region where certainsingularities can be avoided. Specifically, consider the region Ω = x ∈ ℜn :x1 = 0 and define the following coordinate transformation: for any x ∈ Ω,

ξ1 = x1, ξ2 =x2

xn−21

, · · · , ξi =xi

xn−i1

, · · · , ξn = xn. (3.20)

Applying Transformation 3.20 to (3.2) yields

ξ1 = u1, ξ2 =ξ3 − (n − 2)ξ2

ξ1u1, · · · , ξi =

ξi+1 − (n − i)ξi

ξ1u1, · · · , ξn = u2.

(3.21)Should x1(t0) = 0, x(t) ∈ Ω is ensured under control

u1 = −kx1, (3.22)

where constant k > 0 can be arbitrarily chosen. Substituting (3.22) into (3.21),we obtain the linearized sub-system

ξ2 = k(n − 2)ξ2 − kξ3, · · · , ξi = k(n − i)ξi − kξi+1, · · · , ξn = u2, (3.23)

which is time-invariant and controllable. It is straightforward to map LinearSystem 3.23 into the linear controllable canonical form. This partial feedbacklinearization over region Ω is referred to as the σ-process [9], and it will beused in Section 3.3.3 for a control design.

Dynamic Feedback Linearization

As an extension, System 3.17 is said to be dynamically feedback linearizableif it can be transformed into the linear controllable canonical form under adynamic controller of form

u = α1(x, ξ) + β1(x, ξ)v, ξ = α2(x, ξ) + β2(x, ξ)v,

and an augmented state transformation z = T (x, ξ), where ξ ∈ ℜl for someinteger l > 0. Typically, an m-input affine system in the form of (3.17) is feed-back linearizable if output z1j(x) with j = 1, · · · , m is given (or can be found)

such that u appears in z(rj)ij but not in z

(k)ij for k < rj and if r1 + · · ·+ rm = n.

In other words, a well-defined vector relative degree [r1 · · · rm]T is requiredfor feedback linearization. Typically, a system is dynamically feedback lin-earizable but not (statically) feedback linearizable because the vector relativedegree of the system does not exist but can be found by considering uj as

a part of the state variables and by introducing its time derivative u(lj)j as


the control variables. In other words, the partially feedback linearized systemof (3.19) could be used to derive conditions on dynamic feedback lineariza-tion. Both sufficient conditions [38] and necessary condition [233] have beenreported.

For many physical systems, dynamic feedback linearization can be doneby appropriately choosing the system output, as illustrated by the followingexamples.

Example 3.3. Consider Model 3.13 of a differential-drive vehicle. It followsfrom (3.14) that

adg1g2 =[

sin x3 − cosx3 0]T

and rank[

g1 g2 adg1g2

]

= 3,

which implies that D0 = spang1, g2 is not involutive and, by Theorem 2.35,Kinematic Equation 3.13 is not static feedback linearizable. On the otherhand, choosing x1, x2 as the output variables, differentiating them twice andsubstituting the equations of (3.13) into the result yield

[

x1

x2

]

=

[

cosx3 −u1 sin x3

sin x3 u1 cosx3

] [

u1

u2

]

=

[

v1

v2

]

,

which is in the linear canonical form with respect to the transformed controlv. Solving u1 and u2 in terms of v yields the dynamic control mapping:

u1 = ξ, ξ = v1 cosx3 + v2 sinx3, u2 =1

u1(v2 cos θ − v1 sin θ).

Note that the above transformation becomes singular if u1 = 0. In summary,Kinematic Model 3.13 is dynamically feedback linearizable provided that thevehicle keeps moving.

Example 3.4. Consider Model 1.35 of a car-like vehicle. It follows that

g1 =[

cos θ sin θ 1l tan φ 0

]T, g2 =

[

0 0 0 1]T

andadg1g2 =

[

0 0 − 1l cos2 φ 0

]T/∈ spang1, g2.

Thus, System 1.35 is not static feedback linearizable. On the other hand,choosing (x, y) as the output variables and differentiating them three timesyield

[

x(3)

y(3)

]

=

[

− 3l u1u1 sin θ tanφ − 1

l2 u31 tan2 φ cos θ

3l u1u1 cos θ tanφ − 1

l2 u31 tan2 φ sin θ

]

+

[

cos θ − sin θl cos2 φu2

1

sin θ cos θl cos2 φu2

1

]

[

u1

u2

]

=

[

v1

v2

]

,


which is in the linear canonical form. Consequently, Model 1.35 is dynamicfeedback linearizable under the dynamic feedback controller

u1 = ξ1,

ξ1 = ξ2,[

ξ2

u2

]

=

[

cos θ − sin θl cos2 φξ2

1

sin θ cos θl cos2 φξ2

1

]−1[

v1

v2

]

−[

− 3l ξ1ξ2 sin θ tan φ − 1

l2 ξ31 tan2 φ cos θ

3l ξ1ξ2 cos θ tan φ − 1

l2 ξ31 tan2 φ sin θ

]

,

provided that u1 = 0.

Control design can easily be done for non-linear systems that are feedbacklinearizable or dynamic feedback linearizable. It is worth mentioning thatcascaded feedback linearization [267] further enlarges the class of non-linearsystems that can be handled using the differential geometric approach.

3.1.4 Options of Control Design

Control designs can be classified into two categories: an open-loop control tonavigate the state to an equilibrium point or to make the system output tracka desired trajectory, and a closed-loop feedback control to ensure stabilityand compensate for disturbances. For a linear system, the open-loop controldesign is basically trivial (since an asymptotically stable linear system has aunique equilibrium) and, with controllability, a closed-loop stabilizing controlcan be chosen to be continuous and of either static state feedback or dynamicoutput feedback. For a non-holonomic system, trajectory planning is usuallyrequired for the system to comply with non-holonomic constraints and, dueto controllability, the open-loop control problem can always be solved as willbe shown in Section 3.2. However, controllability does not necessarily implythat a continuous static state feedback control always exists for non-linearsystems. The following theorem provides a necessary condition on existenceof such a continuous static state feedback control, and existence of a solutionto Algebraic Equation 3.24 is referred to as the Brockett condition [12, 30].

Theorem 3.5. Suppose that non-linear system

x = f(x, u)

is asymptotically stabilized under a continuous static state feedback law u =u(x) with u(0) = 0, where f(x, u) is a smooth function. Then, for any givenǫ > 0, there is a constant δ > 0 such that, for every point ξ ∈ ℜn satisfying‖ξ‖ < δ, algebraic equation

ξ = f(x, u) (3.24)

can be satisfied for some pair of x ∈ ℜn and u ∈ ℜm in the set of ‖x‖ < ǫ and‖u‖ < ǫ.


1x

2x

V(x)=Constant

V(x)<0.

Fig. 3.1. Two-dimensional explanation of the Brockett condition

Proof of Theorem 3.5 is based on the fact that, under a continuous andstabilizing control u(x), the system has a control Lyapunov function V (x)whose time derivative is negative definite as V < 0. Since x is continuous,since the level curves of V (x) = c in a neighborhood of the origin are allclosed, and since V < 0 holds everywhere along a level curve of V , x mustassume all the directions along the level curves and hence Algebraic Equation3.24 must be solvable for all ξ ∈ ℜn of small magnitude. Figure 3.1 providesthe graphical illustration that, given V < 0, the corresponding trajectoriespassing through a closed level curve all move inwards and that, since x iscontinuous, x assumes all the possible directions in 2-D along the level curve.The following example illustrates an application of Theorem 3.5.

Example 3.6. Consider the system

x = f(x, u) =

⎡

⎣

u1

u2

x2u1 − x1u2

⎤

⎦ ,

whose vector fields are

g1(x) =[

1 0 x2

]T, g2(x) =

[

0 1 −x1

]T.

It follows thatadg1g2 =

[

0 0 −1]T

and that rankg1, g2, adg1g2 = 3. Thus, the system is small-time control-lable. On the other hand, algebraic equation

f(x, u) =[

0 0 ǫ]T

does not have a solution for any ǫ = 0. By Theorem 3.5, the system cannotbe stabilized under a continuous and static control of u = u(x).


In general, consider the class of non-holonomic driftless systems:

x =

m∑

i=1

gi(x)ui= G(x)u,

where x ∈ ℜn, m < n, and vector fields gi(x) are linearly independent. Then,without loss of any generality, assume that the top m-by-m block of G(x) isof full rank. Then, the algebraic equation

G(x)u =[

01×m ǫ1 · · · ǫn−m

]T

has no solution unless ǫ1 = · · · = ǫn−m = 0. By Theorem 3.5, the class ofdriftless systems including Chained Form 3.2 cannot be stabilized under anycontinuous static feedback control. Similarly, a continuous time-independentdynamic feedback control would not be stabilizing either. This means that, fordriftless non-holonomic systems, a stabilizing feedback control must be eitherdiscontinuous or time-varying (as u = u(x, t)) or both. Such designs will bepursued in Section 3.3.

3.1.5 Uniform Complete Controllability

Given the fact that non-holonomic systems are non-linearly controllable butnot stabilizable under any continuous static feedback control and that theirpartial or dynamic feedback linearizations have singularity, we need to searchfor a way to design systematically an appropriate feedback control. As re-vealed in Section 2.3.3, a successful control design often boils down to find-ing a control Lyapunov function. And, it is shown in Section 2.5 that, whilethere is no direct connection between non-linear controllability and existenceof Lyapunov function, uniform complete controllability naturally renders aLyapunov function. Although driftless non-holonomic systems do not have acontrollable linearized system around the origin, their uniform complete con-trollability can be determined as follows.

Consider a non-holonomic system in the chained form of (3.2). It followsthat the dynamics can be expressed as

x = A(u1(t))x + Bu,

or equivalently,x1 = u1, z = A2(u1(t))z + B2u2, (3.25)

where

z =[

x2 x3 · · · xn

]T, A = diagA1, A2, B = diagB1, B2,

Ac ∈ ℜ(n−1)×(n−1) and Bc ∈ ℜ(n−1)×1 are those defined in (2.43),

A1 = 0, B1 = 1, A2(u1(t)) = u1(t)Ac, B2 = Bc. (3.26)


Clearly, System 3.25 consists of two cascaded sub-systems. The first sub-system of x1 is linear time-invariant and uniformly completely controllable.Hence, u1 can always be designed (and, as will be shown in Section 3.3.2, u1

should be chosen as a feedback control in terms of x rather than just x1).By analyzing and then utilizing the properties of u1(t), the second non-linearsub-system of z can be treated as a linear time-varying system. This two-stepprocess allows us to conclude the following result on uniform complete con-trollability of non-linear and non-holonomic systems. Once uniform completecontrollability is established, the corresponding control Lyapunov functionbecomes known, and a feedback control design can be carried out for u2.

Definition 3.7. A time function w(t) : [t0,∞) → R is said to be uniformlyright continuous if, for every ǫ > 0, there exists η > 0 such that t ≤ s ≤ t + ηimplies |w(s) − w(t)| < ǫ for all t ∈ [t0,∞).

Definition 3.8. A time function w(t) : [t0,∞) → R is said to be uniformlynon-vanishing if there exist constants δ > 0 and w > 0 such that, for anyvalue of t, |w(s)| ≥ w holds somewhere within the interval [t, t + δ]. Functionw(t) is called vanishing if limt→∞ w(t) = 0.

Lemma 3.9. Suppose that scalar function u1(t) is uniformly right continuous,uniformly bounded, and uniformly non-vanishing. Then, the sub-system of z in(3.25) (that is, the pair of u1(t)Ac, Bc) is uniformly completely controllable.

Proof: Since A(n−1)c = 0, the state transition matrix of system z = u1(t)Acz

can be calculated according to

Φ(t, τ) = eAcβ(t,τ) =

n−2∑

k=0

1

k!Ak

cβk(t, τ), (3.27)

where β(t, τ) =∫ t

τ u1(s)ds. The proof is done by developing appropriatebounds on Controllability Grammian 2.45 in terms of Φ(t, τ) in (3.27) andapplying Definition 2.32.

Since u1(t) is uniformly bounded as |u1d(t)| ≤ u1 for some constant u1 > 0,we know from (3.27) that

‖Φ(t, t + δ)‖ ≤ e‖Ac‖·|β(t,t+δ)| ≤ eu1‖Ac‖δ = α3(δ).

Consequently, we have that, for any unit vector ξ,

ξT Wc(t, t + δ)ξ ≤∫ t+δ

t

‖Φ(t, τ)‖2dτ

≤∫ t+δ

t

e2u1‖Ac‖(τ−t)dτ

=

∫ δ

0

e2u1‖Ac‖sds= α2(δ). (3.28)


For every δ > 0, there exists constant u1 > 0 such that |u1(s)| ≥ u1 holdsfor some s(t) ∈ [t, t + δ] and for all t. It follows from uniform right continuityand uniform boundedness that, for some sub-intervals [s(t), s(t) + σ(δ, u1)] ⊂[t, t + δ] where value σ(·) is independent of t, u1(τ) has a fixed sign and isuniformly bounded away from zero for all τ ∈ [s(t), s(t) + σ(δ, u1)]. Thus, itfollows from (3.27) that, for any unit vector ξ,

ξT Wc(t, t + δ)ξ ≥ ξT Wc(s(t), s(t) + σ(δ, u1))ξ

=

∫ σ(δ,u1)

0

∣

∣

∣

∣

ξT eAc

∫

φ

0u1(s(t)+τ)dτ

Bc

∣

∣

∣

∣

2

dφ. (3.29)

Defining variable substitution θ(φ) =∫ φ

0|u1(s(t)+ τ)|dτ , we know from u1(τ)

being of fixed sign in interval [s(t), s(t)+σ(δ, u1)] that function θ(φ) is strictlymonotonically increasing over [0, σ(δ, u1)] and uniformly for all t, that

θ(φ) =

∫ φ

0u1(s(t) + τ)dτ if u1(s(t)) > 0

−∫ φ

0 u1(s(t) + τ)dτ if u1(s(t)) < 0,

and that, since dθ/dφ = 0, function θ(φ) has a well defined inverse with

dφ =dθ

|u1(s(t) + φ)| ≥dθ

u1> 0.

Applying the change of variable to (3.29) yields

∫ σ(δ,u1)

0

∣

∣

∣

∣

ξT eAc

∫

φ

0u1(s(t)+τ)dτ

Bc

∣

∣

∣

∣

2

dφ

≥

⎧

⎪

⎪

⎨

⎪

⎪

⎩

1

u1

∫ σ(δ,u1)

0

∣

∣ξT eAcθBc

∣

∣

2dθ, if u1(s(t)) > 0

1

u1

∫ σ(δ,u1)

0

∣

∣ξT e−AcθBc

∣

∣

2dθ, if u1(s(t)) < 0

≥ 1

u1min

∫ σ(δ,u1)

0

∣

∣ξT eAcθBc

∣

∣

2dθ,

∫ σ(δ,u1)

0

∣

∣ξT e−AcθBc

∣

∣

2dθ

= α1(δ, u1, u1). (3.30)

In (3.30), the value of α1(·) is positive because both time-invariant pairs of±Ac, Bc are controllable. The proof is completed by combining (3.29) and(3.30).

Since u1(t) is the control for the first sub-system in (3.25), its propertydepends upon the control objective for x1. Should x1 be commanded to tracka non-vanishing time function, u1(t) is non-vanishing and hence uniform com-plete controllability is ensured for the design of u2. If stabilization of x1 is de-sired, u1(t) must be vanishing in which case the second sub-system in (3.25)


is no longer uniformly completely controllable. Nonetheless, an appropriatetransformation can be applied to the sub-system of x2 so that uniform com-plete controllability is recovered for the purpose of finding Lyapunov functionand carrying out the design of u2. In what follows, two examples adoptedfrom [208] are included to illustrate transformations used to recover uniformcomplete controllability in two different cases. In Example 3.10, u1(t) is van-ishing but does not belong to L1 space, and a time-unfolding transformationis applied. It also is shown in [208] that, if u1(t) is non-vanishing but notuniformly non-vanishing, a time-folding transformation can be applied. InExample 3.11, u1(t) is vanishing and belongs to L1 space, and a time-varyingstate transformation is prescribed.

Example 3.10. Consider

u1(t) =1

κ(t)w(t),

where w(t) is continuous and uniformly bounded, κ(t) > 0 for all t ≥ 0,limt→∞ κ(t) = +∞, but 1/κ(t) ∈ L1. Let us introduce the following transfor-mation of time and control:

τ =

∫ t

0

1

κ(s)ds, and u2(t) =

1

κ(t)u′

2(τ).

The transformation essentially unfolds the time, and it is diffeomorphic. Ap-plying the transformation to the sub-system of z in (3.25) yields

dz(τ)

dτ= w′(τ)Acz(τ) + Bcu

′2, (3.31)

where w′(τ) = w(t) with t being replaced by the inverse of the above timetransformation (which can be found once κ(t) is specified). As long as w′(τ)in (3.31) is uniformly non-vanishing, design of control u2 can be done throughthe design of u′

2 for System 3.31.

Example 3.11. Suppose that

u1(t) = e−tw(t),

where w(t) is continuous, uniformly non-vanishing, and uniformly bounded.Consider the time-dependent state transformation

z′ = diage(n−2)t, · · · , et, 1z. (3.32)

Applying the transformation to the second sub-system in (3.25) yields

z′ = diag(n − 2), · · · , 1, 0z′ + diage(n−2)t, · · · , et, 1[u1(t)Acz + Bcu2]

= diag(n − 2), · · · , 1, 0z′ + w(t)Acz′ + Bcu2

= [diag(n − 2), · · · , 1, 0 + w(t)Ac] z′ + Bcu2. (3.33)


If w(t) = 1, Transformed System 3.33 is time-invariant and controllable (henceuniformly completely controllable). Given any uniform non-vanishing functionw(t), the state transition matrix of System 3.33 can be found to check uniformcomplete controllability. Stability analysis is needed (and will be shown inSection 3.3.2) to guarantee that Transformation 3.32 is well defined.

3.1.6 Equivalence and Extension of Chained Form

Under the global state transformation

z1 = x1,zj1 = xjnj

, 2 ≤ j ≤ m,

zjk = (−1)kxj(nj−k+1) +k∑

l=1

(−1)l 1

(k − l)!(x11)

k−lxj(nj−l+1),

2 ≤ k ≤ nj , 2 ≤ j ≤ m,

Chained System 3.1 becomes the so-called power form defined by

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

z1 = u1,

z21 = u2, z22 = z1u2, z23 =z21

2!u2, · · · z2n2 =

zn2−11

(n2 − 1)!u2,

...

zm1 = um, zm2 = z1um, zm3 =z21

2!um, · · · zmnm

=znm−11

(nm − 1)!um.

(3.34)

Another equivalent model for non-holonomic systems is the so-called skew-symmetric chained form [222]. Specifically, consider a system in Chained Form3.3, that is,

x1 = u1; xi = xi+1u1, i = 2, · · · , n − 1; xn = u2. (3.35)

Define the following coordinate transformation:

z1 = x1, z2 = x2, z3 = x3; zj+3 = kjzj+1 + Lg1zj+2, 1 ≤ j ≤ n − 3;

where kj > 0 are positive constants (for 1 ≤ j ≤ n − 3), and g1 =[1, x3, x4, · · · , xn, 0]T . Then, under the transformation, Chained System 3.35is converted into the skew-symmetric chained form:

⎧

⎪

⎪

⎨

⎪

⎪

⎩

z1 = u1,z2 = u1z3,zj+3 = −kj+1u1zj+2 + u1zj+4, 0 ≤ j ≤ n − 4,zn = −kn−2u1zn−1 + w2,

(3.36)

where w2 = (kn−2zn−1 + Lg1zn)u1 + u2.


On the other hand, Chained Form 3.1 can be generalized by admittingboth draft terms and higher-order derivatives. Such a generalization of (3.2)leads to the so-called extended chained form:⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

x(k1)1 = u1,

x(k2)2 = α2

(

x1, · · · , x(k1−1)1 , u1

)

x3 + β2

(

x2, · · · , x(k2−1)2

)

,

...

x(kn−1)n−1 = αn−1

(

x1, · · · , x(kn−1−1)1 , u1

)

xn + βn−1

(

xn−1, · · · , x(kn−1−1)n−1

)

,

x(kn)n = u2 + βn

(

xn, · · · , x(kn−1)n

)

,

(3.37)where ki are positive integers, αi(·) are smooth functions (some or all ofwhich are vanishing with respect to their arguments), and βi(·) are smoothdrift functions. If αi(·) = u1 and βi(·) = 0, Model 3.37 reduces to that in[117]. An extended power form can be similarly defined [112]. As illustratedby the following example, the extended chained model should be used as thecanonical model for certain non-holonomic systems.

Example 3.12. Consider Model 1.49 and 1.50 of a surface vessel. By directioncomputation, the model is transformed into the following extended chainedform:

⎧

⎨

⎩

x = v1,

y = v1ξ + β(x, y, ψ, ψ),

ξ = v2,

where ξ = tanψ is the transformed state variable, hence ψ = arctan ξ andψ = ξ/(1 + ξ2),

β(x, y, ψ, ψ) =

[(

1 − m11

m22

)

xψ − d22

m22y

]

(sin ψ tan ψ + cosψ),

and transformed control inputs vi are defined by

v1 =

(

m22

m11yψ − d11

m11x +

1

m11τ1

)

cosψ +

(

m11

m22xψ +

d22

m22y

)

sin ψ

−(x sinψ + y cosψ)ψ,

and

v2 = sec2 ψ

(

m11 − m22

m33xy − d33

m33ψ +

1

m33τ2

)

+ 2(ψ)2 sec2 ψ tan ψ,

respectively.

The dynamic augmentation from (3.2) to (3.37) captures a larger classof non-holonomic kinematic constraints, and it can be further extended toinclude a torque-level dynamic model such as the second equation in (1.24).


From the control design point of view, these augmentations of introducing in-tegrators can be handled in principle by employing the backstepping approachdiscussed in Section 2.6.1. If drift functions βi(·) have similar vanishing prop-erties as αi(·), the aforementioned design methods such as dynamic feedbacklinearization and state transformation can be applied to Extended ChainedModel 3.37 in a similar fashion as Standard Chained Model 3.2.

Introduction of the chained forms enables us to have one canonical formfor different non-holonomic systems, to study their common properties, and todevelop systematic procedures of analysis and control design. Since there is awell defined transformation between the original system and its chained form,all the results such as properties and control designs can be mapped back tothe original systems. Indeed, the transformation typically retains some (out-put) variables of the original model. For instance, in Chained Form 3.10 of afront-steer back-drive vehicle, Cartesian coordinators of its guidepoint remainas the output (and state) variables, but steering angle and body angle nolonger appear explicitly. Nonetheless, it is always possible and often better toconduct analysis and control design on the original model in a specific appli-cation such that physical meanings of the original variables can be exploitedand that additional constraints (such as maximum steering rate, maximumsteering angle, etc.) can be taken into consideration.

3.2 Steering Control and Real-time Trajectory Planning

The basic issue of motion planning is to find a feasible trajectory (or the de-sired trajectory) which satisfies such motion requirements as boundary condi-tions, non-holonomic constraints, and other geometrical constraints imposedby the environment. Non-holonomic systems are small-time non-linearly con-trollable, and hence there is a solution to their navigation problem of planninga feasible motion and determining the corresponding steering inputs. Specifi-cally, we first present several simple steering control laws that generate feasibletrajectories for a non-holonomic chained system in a free configuration space.Then, we formulate the real-time trajectory planning problem by consideringa dynamic environment in which both static and moving obstacles are present.As an illustration, a real-time algorithm is presented to generate an optimizedcollision-free and feasible trajectory for a car-like vehicle with a limited sens-ing range. Based on the trajectory planning and replanning in the dynamicenvironment, a feedback control can be designed (in Section 3.3) so that themotion of a non-holonomic vehicle becomes pliable.

3.2.1 Navigation of Chained Systems

Consider a non-holonomic system in the chained form of (3.3), that is,

z1 = u1, z2 = u2, z3 = z2u1, · · · , zn = zn−1u1. (3.38)

3.2 Steering Control and Real-time Trajectory Planning 99

The basic problem is to choose steering control u such that z(t0) = z0 andz(tf ) = zf , where z = [z1, · · · , zn]T is the solution to (3.38), z0 is the giveninitial condition, zf = [zf,1, · · · , zf,n]T is the given final condition, and tfmay be prescribed. In what follows, three typical laws of steering control arepresented. Other approaches such as differential geometry [251], differentialflatness [67], input parameterization [170, 259], and optimal control theory[64] can also be applied to synthesize steering controls.

Sinusoidal Steering Inputs

The method of synthesizing a sinusoidal steering control is based on the sim-ple mathematical facts that the indefinite integral of product sinωt sinkωtproduces the higher-frequency component sin(k + 1)ω, that

∫ Ts

0

sinωtdt =

∫ Ts

0

cos kωtdt = 0,

and that∫ Ts

0

sin jωt sinkωtdt =

0 if j = kπω if j = k

,

where Ts = 2π/ω. Based on these facts, the basic step-by-step process ofgenerating a sinusoidal steering control is to steer state variables zi one-by-one [169] as follows:

Step 1: In the first interval of t ∈ [t0, t0 +Ts], choose inputs u1 = α1 and u2 =β1 for constants α1, β1 such that z1(t0 +Ts) = zf,1 and z2(t0 +Ts) = zf,2,and calculate zi(t0 + Ts) for 3 ≤ i ≤ n by integrating (3.38).

Step 2: In the second interval of t ∈ [t0 + Ts, t0 + 2Ts], let the inputs be

u1(t) = α2 sin ωt, u2(t) = β2 cosωt. (3.39)

Direct integration of (3.38) under (3.39) yields

z1(t0+2Ts) = zf,1, z2(t0+2Ts) = zf,2, z3(t0+2Ts) = z3(t0+Ts)+α2β2

2ωTs,

and zi(t0+2Ts) for i > 3. Hence, β2 can be chosen such that z3(t0+2Ts) =zf,3.

Step k (3 ≤ k ≤ (n − 1)): In the kth interval of t ∈ [t0 + (k − 1)Ts, t0 + kTs],let the steering inputs be

u1(t) = α2 sin ωt, u2(t) = βk cos(k − 1)ωt. (3.40)

It follows from (3.38) under (3.40) that⎧

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎩

z1(t0 + 2Ts) = zf,1,...zk(t0 + kTs) = zf,k,

zk+1(t0 + kTs) = zk+1(t0 + (k − 1)Ts) +αk

2βk

k!(2ω)kTs,


and zi(t0 + kTs) for k + 1 < i ≤ n. Hence, βk can be chosen such thatzk+1(t0 + kTs) = zf,(k+1). The step repeats itself till k = n − 1.

Intuitively, the above steering controls can be designed in terms of simpleproperties of sinusoidal functions because of the mathematical properties ofLie brackets of the chained system. Recall from Section 3.1.2 that adk

g1g2 =

[0 · · · 0 1 0 · · · 0]T which is precisely the motion direction of zk+2, and hencethe motion of zk+2 can be achieved by letting the frequency of u2 be k timesthat of u1.

The above steering control algorithm based on piecewise-sinusoidal func-tions in (3.40) has a completion time of tf = t0 + (n − 1)Ts. Combiningall the piecewise-sinusoidal functions together yields the so-called all-at-oncesinusoidal steering method [259] in which the inputs are defined to be

u1 = α1 + α2 sin ωt, u2 = β1 + β2 cosωt + · · · + βn−1 cos(n − 2)ωt. (3.41)

Integrating (3.38) under (3.41) yields n algebraic equations in terms of (n+1)design parameters. It is straightforward to show that, if z1f = z10, theseequations can be solved to meet terminal condition z(tf) = zf with completiontime tf = t0 + Ts.

Example 3.13. Consider a fourth-order chained system in the form of (3.38).Sinusoidal inputs in (3.41) can be used to steer the system from initial value

z0= [z10 z20 z30 z40]

T to final value zf= [z1f z2f z3f z4f ]T . It is elementary

to solve for control parameters coefficients and obtain

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

α1 =z1f − z10

Ts, α2 = 0,

β1 =z2f − z20

Ts, β2 =

2ω

α2Ts

(

z3f − z30 − α1z20Ts −α1β1T

2s

2+

α2β1Ts

ω

)

,

β3 =8ω2

(2α21 + α2

2)Ts

(

z4f − z40 − α1z30Ts −α2

1T2s z20

2− α2

1T2s β1T

3s

6

−α21β2Ts

ω2− α1α2z20Ts

ω− α1α2β2T

2s

4ω− α2

2β1Ts

2ω2

)

.

Given boundary conditions z0 = [0 0 0 0]T and zf = [1 0 0 2]T , we choose ω =2π/Ts, Ts = 3, and α2 = 0.2. The corresponding values of design parametersα1, β1, β2 and β3 are 0.3333, 0, 0, 89.2168, respectively. Figure 3.2 shows thetrajectories under the sinusoidal steering inputs.

Piecewise-constant Steering Inputs

The piecewise-constant steering method [159] is to choose the inputs as: fort ∈ [t0 + (k − 1)Ts, t0 + kTs),

u1(t) = αk, u2(t) = βk, (3.42)


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

z1 versus z

4

(a) Phase portraits of z1 vs z4

0 0.5 1 1.5 2 2.5 3Ŧ25

Ŧ20

Ŧ15

Ŧ10

Ŧ5

0

5

10

15

20

25

Time (sec)

z2

z3

(b) Trajectories of z2 and z3

Fig. 3.2. Trajectories under sinusoidal inputs


where k = 1, 2, · · · , (n − 1), αk and βk are design constants, and Ts is thesampling period. Integrating Chained System 3.38 over the interval yields

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

z1(t0 + kTs) = z1(t0 + (k − 1)Ts) + αkTs,z2(t0 + kTs) = z2(t0 + (k − 1)Ts) + βkTs,

z3(t0 + kTs) = z3(t0 + (k − 1)Ts) + z2(t0 + (k − 1)Ts)αkTs + αkβkT 2

s

2,

...zn(t0 + kTs) = zn(t0 + (k − 1)Ts) + zn−1(t0 + (k − 1)Ts)αkTs

+ · · · + βkαn−2k

T n−1s

(n − 1)!.

By setting z(tf ) = zf with tf = t0 + (n − 1)Ts and by repeatedly applyingthe above equations, we obtain a set of algebraic equations in terms of z0, zf ,αk, and βk. To simply the calculations, αk = α can be set, and z(tf ) = zf

becomes n linear algebraic equations in terms of n design parameters α and β1

up to βn−1. Solution to the resulting equations provides the piecewise-constantsteering inputs.

Example 3.14. Consider the same steering control problem stated in Example3.13. If the piecewise-constant inputs in (3.42) are used, the control parametersare determined as

⎧

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎩

α =z1f − z10

3Ts,

⎡

⎣

β1

β2

β3

⎤

⎦ =

⎡

⎢

⎣

Ts Ts Ts5αT 2

s

23αT 2

s

2αT 2

s

219α2T 3

s

67α2T 3

s

6α2T 3

s

6

⎤

⎥

⎦

−1⎡

⎣

z2f − z20

z3f − z30 − 3z20αTs

z4f − z40 − 3z30αTs − 4.5z20α2T 2

s

⎤

⎦ .

Under the choice of Ts = 1, the total steering time is identical to that inExample 3.13, the trajectories are shown in Fig. 3.3, and the values of β1, β2

and β3 are 18,−36, 18, respectively.

Polynomial Steering Inputs

The steering inputs can also be chosen to be polynomial time functions as[169], if z1f = z10,

u1(t) = c10, u2(t) = c20 + c21(t − t0) + · · · + c2(n−2)(t − t0)n−2, (3.43)

where c10, c20, c21, · · ·, and c2(n−2) are design constants. Integrating ChainedSystem 3.38 under Input 3.43 yields


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

z1 versus z

4


0 0.5 1 1.5 2 2.5 3Ŧ20

Ŧ15

Ŧ10

Ŧ5

0

5

10

15

20

Time (sec)

z2

z3


Fig. 3.3. Trajectories under piecewise-constant inputs


⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

z1(t) = z1(t0) + c10(t − t0),

z2(t) = z2(t0) + c20(t − t0) +c21(t − t0)

2

2+ · · · + c2(n−2)(t − t0)

n−1

n − 1,

...

zn(t) = zn(t0) +

n−2∑

k=0

k!cn−210 c2k(t − t0)

n+k−1

(n + k − 1)!+

n−1∑

k=2

cn−k10 (t − t0)

n−k

(n − k)!zk(t0).

(3.44)If zf,1 = z1(t0), c10 can be solved for any tf > t0 as

c10 =zf,1 − z1(t0)

tf − t0.

Then, the rest of the (n−1) equations in (3.44) are linear in terms of c2j , andthey can be solved upon setting z(tf) = zf . If zf,1 = z1(t0), an intermediatepoint z(t1) with z1(t1) = z1(t0) and t0 < t1 < tf can be chosen to generatetwo segments of the trajectory: one from z(t0) to z1(t1), and the other fromz1(t1) to zf .

Example 3.15. For the steering problem studied in Example 3.13, polynomialinputs in (3.43) can be used, and their corresponding parameters are: lettingTs = tf − t0,

⎧

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎩

c10 =z1f−z10

Ts,

⎡

⎣

c20

c21

c22

⎤

⎦ =

⎡

⎢

⎣

TsT 2

s

2T 3

s

3c10T 2

s

2c10T 3

s

6c10T 4

s

12c210T 3

s

6c210T 4

s

24c210T 5

s

60

⎤

⎥

⎦

−1⎡

⎣

z2f − z20

z3f − z30 − c10z20Ts

z4f − z40 − c10z30Ts − 0.5c210z20T

2s

⎤

⎦

.

If Ts = 3 is set, the values of c10, c20, c21, c22 are 0.3333, 40,−80, 26.6667,respectively. The corresponding trajectories are shown in Fig. 3.4.

Although the above three steering control designs are all comparable, wesee that the polynomial control is arguably the most user-friendly since boththe steering controls and the resulting trajectories are smooth. It should benoted that, except for more computations, u1(t) in (3.43) can also be chosento be a polynomial function of time. Based on (3.44), tf can be adjusted tosatisfy any additional constraints on ui such as kinematic control saturation.

3.2.2 Path Planning in a Dynamic Environment

Autonomous vehicles are likely to operate in an environment where there arestatic and moving obstacles. In order for a vehicle to maneuver successfullyin such a dynamic environment, a feasible and collision-free trajectory needsto be planned in the physical configuration space. In order to illustrate theprocess of synthesizing the corresponding polynomial steering control in the


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

z1 versus z

4


0 0.5 1 1.5 2 2.5 3Ŧ15

Ŧ10

Ŧ5

0

5

10

15

Time (sec)

z2

z3


Fig. 3.4. Trajectories under polynomial inputs


form of (3.43), a specific vehicle model should be considered, and its boundaryconditions should be specified in the physical configuration space. Accordingly,we adopt the front-steering back-driving vehicle in Fig. 3.5 as the illustrativeexample. Unless mentioned otherwise, a circle of radius r0 is used to representthe vehicle’s physical envelope, and the guidepoint is placed at the vehicle’scenter in order to minimize the envelope. Parallel to the modeling in Section1.3.2, the kinematic model of this vehicle is given by

⎡

⎢

⎢

⎣

xy

θ

φ

⎤

⎥

⎥

⎦

=

⎡

⎢

⎢

⎢

⎣

ρ cos θ − ρ2 tan φ sin θ 0

ρ sin θ + ρ2 tanφ cos θ 0

ρ

ltan φ 0

0 1

⎤

⎥

⎥

⎥

⎦

[

w1

w2

]

, (3.45)

where ρ is the radius of the driving wheels, l is the distance between centersof two wheel axles, w1 is the angular velocity of the driving wheels, w2 is thesteering rate of the guiding wheels, q = [x, y, θ, φ]T is the state, and their initialand final configurations are q0 = [x0, y0, θ0, φ0]

T and qf = [xf , yf , θf , φf ]T ,respectively. In order to apply the design of polynomial steering control, Non-holonomic Model 3.45 is mapped into the chained form

z1 = u1, z2 = u2, z3 = z2u1, z4 = z3u1. (3.46)

under coordinate transformation⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎩

z1 = x − l

2cos θ,

z2 =tan φ

l cos3 θ,

z3 = tan θ,

z4 = y − l

2sin θ,

(3.47)

and control transformation⎧

⎪

⎨

⎪

⎩

w1 =u1

ρ cos θ,

w2 =3 sin θ sin2 φ

l cos2 θu1 +

(

l cos3 θ cos2 φ)

u2.(3.48)

A typical scenario of motion planning is that, for any given vehicle, itssensing range is limited and its environmental changes are due to appear-ance/disappearance and/or motion of objects in the vicinity. For the illustra-tive example, the 2-D version is depicted in Fig. 3.6 in which the vehicle isrepresented by the circle centered at O(t) = (x, y) and of radius r0, its sensorrange is also circular and of radius Rs, and the ith obstacle (i = 1, · · · , no) arerepresented by the circle centered at point Oi(t) and of radius ri. For movingobjects, origin Oi(t) is moving with linear velocity vector vi(t).


y

x

l

r0

GP

Fig. 3.5. Physical envelope of a car-like robot

The real-time trajectory planning problem is to find trajectory q(t) tosatisfy Kinematic Model 3.45 or 3.46, to meet the boundary conditions ofq(t0) = q0 and q(tf ) = qf , and to avoid all the obstacles in the environ-ment during the motion. To ensure solvability and to simplify the technicaldevelopment, the following conditions are introduced:

(a) Physical envelopes of the vehicle and all the obstacles are known. Unlessstated otherwise, the envelopes are assumed to be circular.

(b) Sampling period Ts used by onboard sensors and steering controls is cho-sen such that Ts is small, that k = (tf − t0)/Ts is an integer, that positionOi (i.e., Oi = (xk

i , yki ) at t = t0 + kTs) of all the obstacles within the

sensing range are detected at the beginning of each sampling period, and

that their velocities vki

= [ vk

i,x vki,y ]T are known and (approximately)

constant for t ∈ [t0 + kTs, t0 + (k + 1)Ts).(c) All obstacles must be avoided. The obstacles do not form an inescapable

trap, nor does any of the obstacles prevent the vehicle arriving at qf in-definitely, and the vehicle can avoid any of the obstacles by moving fasterthan them. If needed, intermediate waypoints (and their configurations)can be found such that the feasible trajectory can be expressed by seg-ments parameterized by a class of polynomial functions (of sixth-order).Furthermore, the feasible trajectory is updated with respect to samplingperiod Ts in order to accommodate the environmental changes.

(d) Boundary configurations q0 and qf have the properties that x0− l2 sin θ0 =

xf − l2 sin θf and |θ0 − θf | < π.

(e) For simplicity, no consideration is given to any of additional constraintssuch as maximum speed, minimum turning radius, etc.


1O

2O

3O

4O

)(1 tv

)(2 tv

)(3 tv

)(4 tv

x

y

),( 00 yx

),( ff yx

3r

1r

4r

2r

sR

rv

sensing range

O

0

f

Fig. 3.6. Trajectory planning in a dynamic environment

Under these conditions, an optimized trajectory planning algorithm can bedeveloped as will be shown in Section 3.2.3.

In applications, Conditions (a) - (e) can be relaxed in the following ways.Collision avoidance is achieved by imposing an appropriate minimum distancebetween any two objects of certain physical envelopes. If the envelopes are allcircular (spherical), the minimum distance is in terms of distance between twocenters, and hence the corresponding condition is a second-order inequality.It is straightforward to relax Condition (a) by including polygonal envelopes,in which case some of the collision-free conditions are of first-order inequal-ity. Though small sampling period Ts is required to detect the changes inobstacle’s position and velocity, the velocities in Condition (b) may be es-timated from position measurements. If some of the obstacles are allowedto be overrun, whether to avoid these obstacles can also be optimized [280].The intermediate waypoints required in Conditions (c) and (d) can be deter-mined by applying the heuristic approach of either A∗ or D∗ search [243, 244].Should the vehicle velocity be limited, it may not successfully avoid a fast-moving constant-speed obstacle unless the sensing range is properly increased.Should the vehicle be subject to certain minimum turn radius, Condition (e)can be relaxed by combining the Dubin’s algorithm [25, 56, 251].


3.2.3 A Real-time and Optimized Path Planning Algorithm

It follows from State Transformation 3.47, Chained Form 3.46 and Solution3.44 that polynomial trajectories can be parameterized as

z4(z1) = ab(z1),

where a = [a0, a1, · · · , ap] is the vector of design parameters, and b(z1) =[1, z1(t), (z1(t))

2, · · · , (z1(t))p]T is the vector of basis functions of z1(t). Within

time interval [t0 + kTs, t0 + (k + 1)Ts), the updated class of trajectories isdenoted by

z4(z1) = akb(z1). (3.49)

By continually updating parameter vector ak = [ak0 , ak

1 , · · · , akp], an optimized

feasible trajectory is planned based on vehicle’s current initial configuration(q(t0+kTs) or z(t0+kTs)), final configuration qf , initial position Oi = (xk

i , yki )

and constant velocity vki of neighboring obstacles. The planning is done in

three steps: define a class of feasible trajectories under polynomial steeringcontrols, determine the sub-set of collision-free trajectories, and find the op-timized trajectory.

Parameterized Polynomial Trajectories

A trajectory in (3.49) is feasible if it corresponds to certain polynomial steeringinputs and satisfies the following boundary conditions on Chained Form 3.46:

akb(zk1 ) = zk

4 , ak db(z1)

dz1

∣

∣

∣

∣

z1=zk1

= tan θk, ak d2b(z1)

d(z1)2

∣

∣

∣

∣

z1=zk1

=tan φk

l cos3 θk, (3.50)

akb(zf1 ) = zf

4 , ak db(z1)

dz1

∣

∣

∣

∣

z1=zf

1

= tan θf , ak d2b(z1)

d(z1)2

∣

∣

∣

∣

z1=zf

1

=tanφf

l cos3 θf, (3.51)

where zi(t) are defined by (3.47):

zki

= zi(t0 + kTs), and zf

i

= zi(tf ).

It follows from (3.50) and (3.51) that, if p = 5, there is a unique solutionto the parameterized feasible trajectory of (3.49) for any pair q(t0 + kTs)and q(tf ). By choosing p ≥ 6, we obtain a family of feasible trajectoriesfrom which a collision-free and optimized trajectory can be found. To simplifycomputation, we set p = 6 in the subsequent development. In other words, wechoose to parameterize the class of feasible trajectories in (3.49) by one freeparameter ak

6 . The rest of design parameters are chosen to satisfy the “current”boundary conditions in (3.50) and (3.51), that is, they are the solution to thefollowing algebraic equation:

[

ak0 , ak

1 , ak2 , ak

3 , ak4 , a

k5

]T= (Bk)−1[Zk − Akak

6 ], (3.52)


where

Zk =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎣

zk4

zk3

zk2

zf4

zf3

zf2

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎦

, Ak =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎣

(zk1 )6

6(zk1 )5

30(zk1 )4

(zf1 )6

6(zf1 )5

30(zf1 )4

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎦

, (3.53)

Bk =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎣

1 zk1 (zk

1 )2 (zk1 )3 (zk

1 )4 (zk1 )5

0 1 2zk1 3(zk

1 )2 4(zk1 )3 5(zk

1 )4

0 0 2 6zk1 12(zk

1 )2 20(zk1 )3

1 zf1 (zf

1 )2 (zf1 )3 (zf

1 )4 (zf1 )5

0 1 2zf1 3(zf

1 )2 4(zf1 )3 5(zf

1 )4

0 0 2 6zf1 12(zf

1 )2 20(zf1 )3

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎦

. (3.54)

To determine explicitly the feasible class of parameterized trajectories in(3.49), we need to find their corresponding steering controls ui in ChainedForm 3.46 and also to calculate zk

i required in (3.53) and (3.54). To this end,we choose steering inputs ui to be the polynomials in (3.43). That is, after the“current” kth sampling and for the rest of time t ∈ (t0 + kTs, tf ], the updatedsteering inputs are of the form

uk1(t) = c10,

uk2(t) = ck

20 + ck21(t − t0 − kTs) + ck

22(t − t0 − kTs)2 + ck

23(t − t0 − kTs)3.

Substituting the above steering inputs into Chained Form 3.46, we can inte-grate the equations, find the expressions for z1(t) up to z4(t), and concludethat parameterized trajectories in (3.49) are generated by the following steer-ing inputs:

uk1(t) =

zf1 − z1(t0)

tf − t0

= c10, (3.55)

uk2(t) = 6[ak

3 + 4ak4z

k1 + 10ak

5(zk1 )2 + 20ak

6(zk1 )3]c10 + 24[ak

4 + 5ak5z

k1

+15ak6(z

k1 )2](t − t0 − kTs)(c10)

2 + 60(ak5 + 6ak

6zk1 )(t − t0 − kTs)

2(c10)3

+120ak6(t − t0 − kTs)

3(c10)4. (3.56)

Under Steering Controls 3.55 and 3.56, zk+1i is updated from zk

i according tothe following expressions:

zk+11 = zk

1 + c10Ts,

zk+12 = zk

2 +

∫ t0+(k+1)Ts

t0+kTs

uk2(t)dt,

zk+13 = zk

3 + c10Tszk2 + c10

∫ t0+(k+1)Ts

t0+kTs

∫ s

t0+kTs

uk2(t)dtds,


zk+14 = zk

4 + c10Tszk3 +

T 2s

2c10z

k2

+c10

∫ t0+(k+1)Ts

t0+kTs

∫ τ

t0+kTs

∫ s

t0+kTs

uk2(t)dtdsdτ. (3.57)

Initial transformed state z0i is determined using (3.47) and initial con-

figuration q0. Then, the process of updating the feasible trajectory is donerecursively as follows. Given the “current” initial condition zk

i and based onan adequate choice of ak

6 , the updated trajectory is given by (3.49) after solv-ing Linear Equation 3.52, then the updated steering controls are provided by(3.55) and (3.56), and the trajectory and the steering control laws are main-tained for time interval [t0 + kTs, t0 + (k + 1)Ts). At the (k + 1)th sampling,the current initial condition is updated using the equations in (3.57), and theupdating cycle repeats. For implementation, Steering Controls 3.55 and 3.56defined over each of the sampling periods are mapped back to physical controlvariables w1 and w2 through Control Mapping 3.48.

Collision Avoidance Criterion

The class of polynomial trajectories in (3.49) and with p = 6 contains thefree parameter ak

6 . To achieve collision-free motion in a dynamically changingenvironment, a collision avoidance criterion should be developed, and an ana-lytical solution is needed to update ak

6 real-time according to sampling periodTs.

Consider the typical setting in Fig. 3.6. Within time interval [t0 +kTs, t0 +(k + 1)Ts), the vehicle has coordinates (x(t), y(t)), and the ith obstacle hasinitial coordinates (xk

i , yki ) and constant velocity [ vk

i,x vki,y ]T . Therefore, there

is no collision between the vehicle and the obstacle if

[y − (yki + vk

i,yτ)]2 + [x − (xki + vk

i,xτ)]2 ≥ (ri + r0)2, (3.58)

where τ = t − (t0 + kTs) and t ∈ [t0 + kTs, tf ]. The relationships between(x(t), y(t)) and (z1(t), z4(t)) are defined by Transformation 3.47, and henceInequality 3.58 can be rewritten in the transformed space of z4 vs z1 as:

(

z′4,i +l

2sin(θ) − yk

i

)2

+

(

z′1,i +l

2cos(θ) − xk

i

)2

≥ (ri + r0)2, (3.59)

where z′1,i = z1 − vki,xτ and z′4,i = z4 − vk

i,yτ . It is apparent that Inequality3.59 should be checked only for those time instants at which

xki ∈ [z′1,i + 0.5l cos(θ) − ri − r0, z′1,i + 0.5l cos(θ) + ri + r0]. (3.60)

Since there is no closed-form expression for θ only in terms of z1 and z4,Inequality 3.59 needs to be improved to yield a useful criterion. To this end,let us introduce intermediate variables


(x′i, y

′i) =

(

z′4,i +l

2sin(θ), z′1,i +

l

2cos(θ)

)

.

It is shown in Fig. 3.7 that, for θ ∈ [−π/2, π/2], all possible locations of point(x′

i, y′i) are on the right semi circle centered at (z′1,i, z

′4,i) and of radius l/2.

Therefore, without the knowledge of θ a priori, the ith obstacle must stayclear from all the circles of radius (ri + r0) centered and along the previousright semi circle. In other words, the region of possible collision is completelycovered by the unshaded portion of the circle centered at (z′1,i, z

′4,i) and of

radius (ri + r0 + l/2). That is, the collision avoidance criterion in the (z1, z4)plane is mathematically given by

[z4 − (yki + vk

i,yτ)]2 + [z1 − (xki + vk

i,xτ)]2 ≥(

ri + r0 +l

2

)2

, (3.61)

which needs to be checked only if

xki ∈ [z′1,i − ri − r0, z′1,i + 0.5l + ri + r0]. (3.62)

Recalling from (3.55) that z1 and hence z′1,i change linearly with respect totime, we can rewrite Time Interval 3.62 as

t ∈ [tki , tki ], (3.63)

where

tki =

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

t0 + kTs if xki ∈ [zk

1 − ri − r0, zk1 + 0.5l + ri + r0]

tf if zk1 − xk

i − ri − r0 > 0 and if c10 − vki,x ≥ 0

min

tf , t0 + kTs +zk1−xk

i −ri−r0

|c10−vki,x

|

if zk1 − xk

i − ri − r0 > 0 and if c10 − vki,x ≤ 0

min

tf , t0 + kTs +|zk

1−xki +0.5l+ri+r0|c10−vk

i,x

if zk1 − xk

i + 0.5l + ri + r0 < 0 and if c10 − vki,x ≥ 0

tf if zk1 − xk

i + 0.5l + ri + r0 < 0 and if c10 − vki,x ≤ 0

,

and

tki =

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

min

tf , t0 + kTs +xk

i −zk1+ri+r0

c10−vki,x

if xki ∈ [zk

1 − ri − r0, zk1 + 0.5l + ri + r0] and if c10 − vk

i,x ≥ 0

min


i +0.5l+ri+r0

|c10−vki,x

|

if xki ∈ [zk

1 − ri − r0, zk1 + 0.5l + ri + r0] and if c10 − vk

i,x ≤ 0tf if zk

1 − xki − ri − r0 > 0 and if c10 − vk

i,x ≥ 0

min


i +0.5l+ri+r0

|c10−vki,x

|

if zk1 − xk

i − ri − r0 > 0 and if c10 − vki,x ≤ 0

min

tf , t0 + kTs +|zk

1−xki −ri−r0|

c10−vki,x

if zk1 − xk

i + 0.5l + ri + r0 < 0 and if c10 − vki,x ≥ 0

tf if zk1 − xk

i + 0.5l + ri + r0 < 0 and if c10 − vki,x ≤ 0

.


'

,1 iz

'

,4 iz

),( '

,4

'

,1 ii zz

21

),( ''

ii yx

ir +

k

iO ),( k

i

k

i yx=

'

,1 iz ir'

,1 iz ir l5.0- - + + +

0r

0r 0r

Fig. 3.7. Illustration of collision avoidance in the transformed plane

In summary, we have two collision avoidance criteria: Time Criterion 3.63and Geometric Criterion 3.61.

To determine a class of feasible and collision-free trajectories, substituting(3.49) into (3.61) yields the following second-order inequality:

mint∈[tk

i,t

k

i ]

[

ζ2(z1(t), t)(ak6)2 + ζ1,i(z1(t), t)a

k3 + ζ0,i(z1(t), t)

]

≥ 0, (3.64)

where b′(z1) = [1, z1(t), (z1(t))2, · · · , (z1(t))

5]T ,

ζ2(z1(t), t) =[

(z1(t))6 − b′(z1(t))(B

k)−1Ak]2

,

ζ1,i(z1(t), t) = 2[

(z1(t))6 − b′(z1(t))(B

k)−1Ak]

·[

b′(z1(t))(Bk)−1Y k − yk

i

−vki,yτ

]

,

ζ0,i(z1(t), t) =[

b′(z1(t))(Bk)−1Y k − yk

i − vki,yτ

]2+ (z1(t) − xk

i − vki,xτ)2

−(ri + r0 + 0.5l)2.

It is shown in [207] that, unless a collision already occurred at initial configu-ration q0 or will occur at final configuration qf when t = tf (under Condition(c), the latter case can be avoided by adjusting tf ), ζ2(z1(t), t) > 0 and In-equality 3.64 is always solvable. That is, to ensure that the trajectories in


(3.49) are feasible and collision-free, we can choose ak6 in set Ωk

o where Ωko is

the interval defined by

Ωko

=

⋂

i∈1,···,no

[

(−∞, ak6,i] ∪ [ak

6,i, +∞)]

, (3.65)

where

ak6,i = min

t∈[tki,t

k

i ]

−ζ1,i −√

(ζ1,i)2 − 4ζ2ζ0,i

2ζ2,

ak6,i = max

t∈[tki,t

k

i ]

−ζ1,i +√

(ζ1,i)2 − 4ζ2ζ0,i

2ζ2.

Note that, in (3.65), the union operation is done for all the obstacles withinthe sensing range and hence the knowledge of no is not necessarily required.

Optimized Trajectory Planning

Given the class of feasible trajectories in (3.49), an optimized trajectory can beselected according to certain performance index. In what follows, the followingL2 performance index is utilized:

Jk =

∫ zf

1

zk1

[

z4 −zf4 − z0

4

zf1 − z0

1

(z1 − z01) − z0

4

]2

dz1. (3.66)

Intuitively, the above performance index is suitable for three reasons. First, be-tween a pair of initial and final points in the 2-D (or 3-D) space, the minimumdistance is the straight line segment connecting them. Second, in the presenceof non-holonomic constraints, the straight line segment is usually not feasi-ble (because of other configuration variables), and the problem of minimizingthe distance is generally too difficult to solve analytically for non-holonomicsystems. For real-time implementation, one would like to choose a perfor-mance index under which the optimization problem is solvable in closed-form.Third, as shown in Section 1.3.2 and in Transformation 3.47, (z1(t), z4(t))are the coordinates of the center along the back axle. Figure 3.8 provides agraphical interpretation of Performance Index 3.66. Thus, through minimizingPerformance Index 3.66, the vehicle motion should not deviate much from thestraight line segment between the initial and final coordinates.1

To determine the optimal value of ak6 , we substitute (3.49) and (3.52) into

(3.66) and obtain

1 There are cases that the system may have to deviate far away from the straightline segment in order to avoid obstacles. In these cases, Performance Index 3.66could be modified by inserting zk

1 and zk4 for z0

1 and z04 , respectively.


y

xo

(xo,yo)

(xf,yf)

Initial straight line

Trajectory with given

performance index

Fig. 3.8. The geometrical meaning of performance index

Jk(ak6) =

∫ zf

1

zk1

[

b′(z1)(Bk)−1(Y k − Akak

6) + ak6(z1)

6 − zf4 − z0

4

zf1 − z0

1

(z1 − z01)

−z04

]2dz1.

Direct computation yields

Jk(ak6) = α1(a

k6)2 + α2a

k6 + α3, (3.67)

where

α1=

∫ zf

1

zk1

[

(z1(t))6 − b′(z1(t))(B

k)−1Ak]2

dz1,

α2= 2

∫ zf

1

zk1

[

(z1(t))6 − b′(z1(t))(B

k)−1Ak] [

b′(z1(t))(Bk)−1Y k

−zf4 − z0

4

zf1 − z0

1

(z1(t) − z01) − z0

4

]

dz1,

and


α3=

∫ zf

1

zk1

[

b′(z1(t))(Bk)−1Y k − zf

4 − z04

zf1 − z0

1

(z1(t) − z01) − z0

4

]2

dz1.

Clearly, α1 > 0 unless zk1 = zf . Hence, the unconstrained optimal value of ak

6

can easily be found by differentiating (3.67), that is,

ak6 = − α2

2α1.

Using the boundary conditions of ∂lz4/∂zl1 as defined in (3.50) and (3.51) for

l = 0, 1, 2, one can show through direct integration that

α2

2α1=

13

(

∂2z4

∂z12

∣

∣

∣

z1=zk1

+ ∂2z4

∂z12

∣

∣

∣

z1=zf

1

)

12(zk1 − zf

1 )4+

117

(

∂z4

∂z1

∣

∣

∣

z1=zf

1

− ∂z4

∂z1

∣

∣

∣

z1=zk1

)

10(zk1 − zf

1 )5

+429

[

zf

4−z0

4

zf

1−z01

(zk1 − zf

1 ) − (zk4 − zf

4 )]

10(zk1 − zf

1 )6. (3.68)

In the presence of obstacles, ak6 has to be chosen from set Ωk

o in (3.65). Hence,the constrained optimal value of ak

6 is the projection of the unconstrainedoptimal solution onto Ωk

o , that is,

ak∗6 ∈ Ωk

o :

∥

∥

∥

∥

ak∗6 +

α2

2α1

∥

∥

∥

∥

= minak6∈Ωk

o

∥

∥

∥

∥

ak6 +

α2

2α1

∥

∥

∥

∥

. (3.69)

In summary, the optimized collision-free and feasible trajectory is given by(3.49) with Ωk

o in (3.65), with ak6 from (3.68) and (3.69), and with the rest of

ak from (3.52).As an illustration, consider the following simulation setup:

(a) Vehicle parameters: r0 = 1, l = 0.8 and ρ = 0.1.(b) Boundary conditions: (x0, y0, θ0, φ0) = (0, 12,−π

4 , 0) and (xf , yf , θf , φf ) =(17, 0, π

4 , 0).(c) Time interval and sampling period: t0 = 0, tf = 40, and Ts = 1.(d) Sensor range: Rs = 8.(e) Obstacles: three of size ri = 0.5.

Based on motion profiles of the obstacles, the aforementioned algorithm canbe used to determine the corresponding optimized trajectory. For the choicesof obstacles in Table 3.1, the optimized collision-free trajectory is shown inFig. 3.9. In the figure, all the four objects are drawn using the same scale andwith display period of Td = 3.

3.3 Feedback Control of Non-holonomic Systems

In this section, feedback controls are designed for non-holonomic systemsmapped into Chained Form 3.2. The so-called tracking control is the one that

3.3 Feedback Control of Non-holonomic Systems 117

0 2 4 6 8 10 12 14 16 18

Ŧ2

0

2

4

6

8

10

12

x

y

VehicleObstacle 1Obstacle 2Obstacle 3

Fig. 3.9. The optimized trajectory under the choices in Table 3.1

Table 3.1. A setting of three moving obstacles

Initial locations t ∈ [0, 10] t ∈ [10, 20] t ∈ [20, 30] t ∈ [30, 40]

Obst. 1 (5,2) v11 =

[

00.1

]

v12 =

[

0.50.2

]

v13 =

[

0−0.2

]

v14 =

[

0.1−0.1

]

Obst. 2 (10,6) v21 =

[

−0.50

]

v22 =

[

0.20.1

]

v23 =

[

0.10.1

]

v24 =

[

0.60.1

]

Obst. 3 (19,4) v31 =

[

−0.2−0.1

]

v32 =

[

−0.20.1

]

v33 =

[

−0.10.1

]

v34 =

[

−0.10.1

]

makes the system asymptotically follow a desired trajectory. The desired tra-jectory must be feasible, that is, it has been planned to satisfy non-holonomicconstraints and hence is given by

x1d = u1d, x2d = x3du1d, · · · , x(n−1)d = xndu1d, xnd = u2d, (3.70)

where xd = [x1d, · · · , xnd]T ∈ ℜn is the desired state trajectory, yd =

[x1d, x2d]T ∈ ℜ2 is the desired output trajectory, and ud(t) = [u1d(t), u2d(t)]T ∈

ℜ2 is the corresponding open-loop steering control. If xd is a constant andhence ud = 0, the tracking control problem reduces to the stabilization prob-lem, also called the regulation problem. Should xd converge to a constant,ud would have to be vanishing, and the control problem would be treated asthe stabilization problem. As discussed in Section 3.1, the fact of u1 → 0requires specific remedies in the control design, and stabilizing feedback con-


trols also need to be either continuous but time-varying or time-independentbut discontinuous. Consequently, tracking control and stabilizing controls aredesigned separately in the subsequent subsections.

3.3.1 Tracking Control Design

It is assumed in the tracking control design that desired trajectory xd is uni-formly bounded and that steering control u1d(t) is uniformly non-vanishing.Let

xe = [x1e, · · · , xne]T

= x−xd, ye = [y1e, y2e]T

= y−yd, v = [v1, v2]T

= u−ud

be the state tracking error, the output tracking error, and the feedback controlto be designed, respectively. Then, it follows from (3.2) and (3.70) that thetracking error system consists of the following two cascaded sub-systems:

x1e = v1, y1e = x1e, (3.71)

z = u1d(t)Acz + Bcv2 + G(xd, z)v1, y2e = C2z, (3.72)

where z = [z1, · · · , zn−1]T

= [x2e, · · · , xne]T ∈ ℜn−1, Ac ∈ ℜ(n−1)×(n−1) and

Bc ∈ ℜn−1 are those in (2.43),

C2 =[

1 0 · · · 0]

, and G(xd, z) =[

z2 + x3d z3 + x4d · · · zn−1 + xnd 0]

.

The cascaded structure of Error System 3.71 and 3.72 enables us to applythe backstepping design. Specifically, Sub-system 3.71 is of first-order, anda stabilizing control v1 can easily be designed; Sub-system 3.72 has a lineartime-varying nominal system defined by

z = u1d(t)Acz + Bcv2, (3.73)

and the non-linear coupling from the first sub-system to the second is throughG(xd, z)v1 and does not have to be explicitly compensated for because v1 isstabilizing and hence is vanishing. Given these observations, we can choosethe tracking control to be

v1 = −r−11 p1x1e, v2 = −r−1

2 BTc P2(t)z, (3.74)

where r1, r2, q1, q2 > 0 are scalar constants, p1 =√

q1r1, Q2 is a positivedefinite matrix, and P2(t) is the solution to the differential Riccati equation

0 = P2 + P2Acu1d(t) + u1d(t)ATc P2 −

1

r2P2BcB

Tc P2 + Q2 (3.75)

under the terminal condition that P2(+∞) is positive definite. It follows fromu1d(t) being uniformly non-vanishing, Lemma 3.9 and Theorem 2.33 that so-lution P2 to Riccati Equation 3.75 is positive definite and uniformly bounded.Hence, the Lyapunov function can be chosen to be


V (xe, t) = αp1x21e + zT P2(t)z

= V1(x1e) + V2(z, t), (3.76)

where α > 0 is a constant. Taking the time derivatives of Vi(·) along thetrajectories of Error System 3.71 and 3.72 and under Control 3.74 yields

V = −2αq1x21e − zT Q2z − r−1

2 zT P2BcBTc P2z − 2zT P2G(xd, z)r−1

1 p1x1e

≤ −2αq1x21e − 2c1‖z‖2 + 2c2‖z‖2|x1e| + 2c3‖z‖|x1e|, (3.77)

where

c1 = 0.5λmin(Q2 + r−12 P2BcB

Tc P2), c2 = λmax(P2)r

−11 p1, c3 = c2‖xd‖.

On the other hand, the solution to the first sub-system is

x1e(t) = x1e(t0)e−p1

r1(t−t0). (3.78)

Choosing α > c23/(c1q1) in (3.77), applying inequality 2ab ≤ a2 + b2, and then

substituting (3.78) into the expression yield

V ≤ −αq1x21e − c1‖z‖2 + 2c2‖z‖2|x1e|

≤ −β2V + β3V e−β1(t−t0), (3.79)

where

β1 =q1

p1, β2 = min

β1,c1

λmax(P2)

, β3 =2c2|x1e(t0)|λmin(P2)

.

Based on Lyapunov Function 3.76, asymptotic stability and exponential con-vergence of the tracking error system under Control 3.74 can be concluded byinvoking Comparison Theorem 2.8 to solve for V from Inequality 3.79.

It is worth noting that, as shown in [104] as well as Section 2.6.4, the twocomponents v1 and v2 in (3.74) are optimal controls for Linear Sub-system3.71 and Linear Nominal System 3.73 with respect to performance indices

J1 =

∫ ∞

t0

[q1x21e + r1v

21 ]dt, and J2 =

∫ ∞

t0

[zT Q2z + r2v22 ]dt, (3.80)

respectively. For the non-linear tracking error system of (3.71) and (3.72),Control 3.74 is not optimal, nor can the optimal control be found analytically.Nonetheless, given performance index J = J1 +J2, a near-optimal control cananalytically be designed [208]. It is also shown in [208] that output feedbacktracking control can also be designed and that, if a Lie group operation [163]is used to define error state xe, the desired trajectory xd does not explicitlyappear in the resulting tracking error system and hence is not required tobe uniformly bounded. Control 3.74 requires Lyapunov function matrix P2(t)which only depends on u1d(t) and hence can be pre-computed off line andstored with an adequate sampling period. If u1d(t) is periodic, so is solutionP2(t).


To illustrate performance of Control 3.74, consider the fourth-order ChainedSystem 3.2 and choose desired reference trajectory xd to be that generatedunder the sinusoidal steering inputs in (3.41) with ω = 0.1, α1 = 0.3183,α2 = 0.2, β1 = 0, β2 = 0 and β3 = 0.0525. Feedback Control 3.74 is sim-ulated with control parameters r1 = r2 = 1, q1 = 10, and q2 = 20, and itsperformance is shown in Fig. 3.10(a) for the chained system with “perturbed”initial condition x(t0) = [−1, 1, 0.2, 0]T . The phase portrait is given in Fig.3.10(b), where the solid curve is the portion of the reference trajectory overtime interval [0, 20π].

3.3.2 Quadratic Lyapunov Designs of Feedback Control

In this subsection, feedback controls are designed to stabilize Chained System3.2 using quadratic Lyapunov functions in a form similar to (3.76) based onuniform complete controllability. To stabilize Chained System 3.2 to any givenequilibrium point, control u1 must be vanishing. It follows from the discus-sions in Section 3.1.5 that uniform complete controllability could be recoveredeven if control u1 is vanishing. This means that u1(t) should asymptoticallyconverge to zero even if x1(t0) = 0. In other words, a simple backstepping offirst designing u1 for the first sub-system in (3.25) and then synthesizing u2

for the second sub-system does not work. Rather, we need to meet two basicrequirements in order to carry out quadratic Lyapunov designs:

(a) If ‖x(t0)‖ = 0, u1(t) = 0 for any finite time t even if x1(t0) = 0.(b) Control u1(t) should be chosen such that, under an appropriate transfor-

mation, the second sub-system in (3.25) becomes uniformly completelycontrollable.

In what follows, a few different choices are made for u1(t) to meet the aboverequirements, and the corresponding controls of u2(t) are designed.

A Design Based on Exponential Time Function

Consider the following control:

u1(t) = −(k1 + ζ)x1 + w(xτ (t))e−ζ(t−t0), (3.81)

where k1, ζ > 0 are design constants, xτ (t)= x(τ) : t0 ≤ τ ≤ t rep-

resents the history of x(·), and scalar function w(·) has the properties that0 ≤ w(xτ (t)) ≤ c0 for some c0 > 0, that w(xτ (t)) = 0 only if x(t0) = 0, andthat w(xτ (t)) is non-decreasing with respect to time and consequently has thelimit of limt→∞ w(xτ (t)) = c1 for some non-negative constant 0 < c1 ≤ c0 forall xτ (t) with x(t0) = 0. There are many choices for w(xτ (t)); for instance,the choice of

w(xτ (t)) = c0‖x(t0)‖

1 + ‖x(t0)‖= c1(x0)


0 10 20 30 40 50 60Ŧ1.5

Ŧ1

Ŧ0.5

0

0.5

1

1.5

Time (sec)

x1e

x2e

x3e

x4e

(a) Tracking errors

Ŧ5 0 5 10 15 20Ŧ2

0

2

4

6

8

10

12Desired trajectoryTrajectory under tracking control

(b) Phase portrait of x1 vs x2

Fig. 3.10. Performance under Tracking Control 3.74


is constant, while the choice of

w(xτ (t)) = c0

maxτ∈[t0,t] ‖x(τ)‖2

‖x(t)‖ + maxτ∈[t0,t] ‖x(τ)‖2

can have the following values for its limit c1: c1 = c0 for all convergent trajec-tories of x(t), c1 ≤ c0 for all bounded trajectories, and c1 = c0 for all divergenttrajectories.

Under Control 3.81, the first sub-system in (3.25) has the property that

dx21

dt= −2(k1 + ζ)x2

1 + 2x1w(xτ (t))e−ζ(t−t0) ≤ −(2k1 + ζ)x21 +

c20

ζe−2ζ(t−t0),

from which stability and asymptotic convergence of x1(t) can be concluded.In addition, it follows from the system equation and Control 3.81 that

d[x1(t)eζ(t−t0)]

dt= −k1[x1e

ζ(t−t0)] + w(xτ (t))

and that, unless ‖x(t0)‖ = 0,

limt→∞

[x1(t)eζ(t−t0)] = c1/k1, lim

t→∞[u1(t)e

ζ(t−t0)] = −c1/k1 = 0.

Since [u1(t)eζ(t−t0)] is uniformly bounded and uniformly non-vanishing, a

state transformation similar to (3.32) in Example 3.11 can be applied to thesecond sub-system of (3.25). That is, we have

z′ = A′z′ + Bcu2 + η(x, t)Acz′, (3.82)

wherez′ = diage(n−2)ζ(t−t0), · · · , eζ(t−t0), 1z (3.83)

is the transformed state,

A′ = ζdiag(n − 2), · · · , 1, 0 − c1

k1Ac,

and the residual function

η(x, t)= u1(t)e

ζ(t−t0) + c1/k1

is uniformly bounded and vanishing. Since the pair A′, Bc is controllable,control u2 can be chosen to be

u2 = −BTc P2z

′ = −BTc P2diage(n−2)ζ(t−t0), · · · , eζ(t−t0), 1z, (3.84)

where P2 is the solution to algebraic Riccati equation: for some positive defi-nite matrix Q2,


P2A′ + (A′)T P2 − P2BcB

Tc P2 + Q2 = 0. (3.85)

To demonstrate that transformed state z′ is uniformly bounded and asymp-totically convergent under Control 3.84, consider Lyapunov function V2 =(z′)T P2z

′ whose time derivative is

V2 = −(z′)T (Q2 + P2BcBTc P2)z

′ + 2η(x, t)(z′)T Acz′

≤ −γ1V2 + γ2|η(x, t)|V2,

where

γ1 =λmin(Q2 + P2BcB

Tc P2)

λmax(P2), γ2 =

2λmax(Ac)

λmin(P2).

Applying Comparison Theorem 2.8 and solving V2 from the above inequalityyield

limt→+∞

V2(t) ≤ limt→+∞

e−γ1(t−t0)

[

1− γ2γ1(t−t0)

∫

t

t0|η(x,τ)|dτ

]

V2(t0) = 0,

since η(x, t) is uniformly bounded and vanishing. Thus, z′ is asymptoticallystable and exponentially convergent, and so is z. In summary, Control Compo-nents 3.81 and 3.84 together make Chained System 3.76 asymptotically stableand exponentially convergent, and they are smooth and time-varying in termsof exponential time function e±ζt.

A Dynamic Control Design

Instead of using an exponential function, u1 can be designed to be a simpledynamic controller as

u1 = −(k1 + ζ)u1 − k1ζx1, (3.86)

where u1(t0) = −k1x1(t0) + ‖x(t0)‖, and k1 > 0 and 0 < ζ < k1 are designparameters. It follows that, under Control 3.86, the solution to the first sub-system of (3.25) is

x1(t) = c3e−k1(t−t0) + c2e

−ζ(t−t0).

Hence, the solution to Control 3.86 is

u1(t) = −k1c3e−k1(t−t0) − ζc2e

−ζ(t−t0), (3.87)

where c2(x0)= ‖x(t0)‖/(k1− ζ) and c3(x0)

= x1(t0)− c2(x0) . Through injec-

tion of u1(t0), Control 3.86 is to excite state variable x1 whenever ‖x(t0)‖ = 0while making both u1 and x1 exponentially convergent and asymptoticallystable (with respect to not |x1(t0)| but ‖x(t0)‖).


It follows from (3.87) that

limt→∞

[u1(t)eζ(t−t0)] = −ζc2,

which is non-vanishing unless ‖x(t0)‖ = 0. Hence, the same state transfor-mation of (3.83) can be applied to render Transformed System 3.82 withc1 = ζc2k1, to result in control u2(t) in (3.84), and to yield stability andconvergence.

To apply Controls 3.86 and 3.84 to the fourth-order chained system in theform of (3.2), let us choose k1 = 5.77, ζ = 0.15 and Q2 = 10I2×2. For the initialcondition of x(t0) = [0,−2, 3, 6]T , we can determine that the correspondingvalues of u1(t0) and c2(x0) are 7 and 1.2448, respectively. Solution P2 to (3.85)is

P2 =

⎡

⎣

53.7723 −28.2099 5.7674−28.2099 16.6800 −3.94195.7674 −3.9419 1.2133

⎤

⎦ .

Performance of the system under Controls 3.86 and 3.84 is shown in Fig. 3.11.It is worth noting that control component u1 in (3.86) is a pure-feedback

control law and that, based on the discussion in Section 3.1.4, there is nopure-feedback control law for u2. Given the fact that the overall system isnon-linear, Algebraic Riccati Equation 3.85 may depend upon initial conditionx(t0) through constant c1. Although Transformation 3.83 is globally well de-fined for any initial condition of x(t0) = 0, Control 3.84 has the computationalshortcoming that the ratio of two infinitesimals of zi/e−(n−1−i)ζ(t−t0) mustbe calculated. Such a computation is numerically unstable but unavoidablein order to achieve exponential convergence. In what follows, the time scalingapproach is presented as a remedy to overcome this computational problem,and it achieves asymptotic stabilization rather than exponential convergence.

A Time-scaling Design

Consider the following dynamic feedback control of u1:

u1 = −2λ(t)u1 − ω2x1, λ(t)=

1

t − t0 + 1, (3.88)

where u1(t0) = c0‖x(t0)‖, ω > 0 and c0 > 0 are design parameters. Thesolutions to the first sub-system of (3.25) and to Dynamic Control 3.88 canbe found by direct computation as

x1(t) = λ(t)η1(t, x0), u1(t) = λ(t)η2(t, x0), (3.89)

where time functions ηi(t, x0) are defined by

η1(t, x0)= x1(t0) cos(ωt − ωt0) +

c0‖x(t0)‖ + x1(t0)

ωsin(ωt − ωt0),


0 20 40 60 80 100 120Ŧ2

0

2

4

6

8

10

12

Time (sec)

x1x2x3x4

(a) State trajectories

0 20 40 60 80 100 120Ŧ1

0

1

2

3

4

5

6

7

8

Time (sec)

u1

u2

(b) Control inputs

Fig. 3.11. Performance of Stabilizing Controls 3.86 and 3.84


and

η2(t, x0)= c0‖x(t0)‖ + [1 − λ(t)]x1(t0) cos(ωt − ωt0)

−

λ(t)c0‖x(t0)‖

ω+

[

λ(t)

ω+ ω

]

x1(t0)

sin(ωt − ωt0), (3.90)

and both are uniformly non-vanishing and uniformly bounded in proportionto ‖x(t0)‖. Accordingly, x1(t) is asymptotically stable, and the time-scalingmethod in Example 3.10 can be applied.

Under time mapping τ = ln(t − t0 + 1), we can use (3.89) to rewrite thesecond sub-system of (3.25) as

dz′

dτ= η′

2(τ, x0)Acz′ + Bcu

′2, (3.91)

where z′(τ) = z(eτ + t0 − 1), η′2(τ, x0) = η2(e

τ + t0 − 1, x0), and u′2(τ) =

eτu2(eτ + t0 − 1). Since System 3.91 is linear time-varying and uniformly

completely controllable, it is asymptotically stabilized under the control

u′2(τ) = − 1

r2BcP

′2(τ)z′, (3.92)

where matrix Q2 is positive definite, and P ′2(τ) over τ ∈ [0,∞) is the positive

definite solution to the differential Riccati equation:

0 =dP ′

2(τ)

dτ+η′

2(τ, x0)P′2(τ)Ac +AT

c P ′2(t)η

′2(τ, x0)−

1

r2P ′

2(τ)BcBTc P ′

2(τ)+Q2.

(3.93)Exponential stability of z′ can be shown using Lyapunov function V2 =(z′)T P ′

2(τ)z′. Hence, state vector z(t) = z′(ln(t − t0 + 1)) is asymptoticallystable under the following control equivalent to (3.92):

u2(t) = λ(t)u′2(ln(t−t0+1)) = −λ(t)

r2BT

c P ′2(ln(t−t0+1))z

= −λ(t)

r2BT

c P2(t)z,

(3.94)where λ(t) is that in (3.88), and P2(t) is positive definite and uniformlybounded.

For the fourth-order chained system in the form of (3.2) and with initialcondition x(t0) = [0,−2, 3, 6]T , time-scaling control inputs of (3.88) and (3.94)are simulated with the choices of r2 = 1, Q2 = 10I2×2, ω = 1 and u1(t0) = 10,and their performance is shown in Fig. 3.12.

The asymptotically stabilizing control components of (3.88) and (3.94)do not have the computational problem suffered by Controls 3.81, 3.86 and3.84. As a tradeoff, Control 3.94 requires the solution to a differential Ric-cati equation while Control 3.84 is based on an algebraic Riccati equation.Nonetheless, since η2(t, x0) becomes periodic as the time approaches infinity,Lyapunov matrix P2(t) also becomes periodic in the limit and hence can befound using one of the well-established methods [3]. The two pairs of controls,Control 3.86 and 3.84 as well as Control 3.88 and 3.94, can also be shown tobe inversely optimal [204].


0 10 20 30 40 50 60Ŧ4

Ŧ2

0

2

4

6

8

Time (sec)

x1x2x3x4


0 10 20 30 40 50 60Ŧ40

Ŧ35

Ŧ30

Ŧ25

Ŧ20

Ŧ15

Ŧ10

Ŧ5

0

5

10

Time (sec)

u1

u2

(b) Control inputs

Fig. 3.12. Performance under Controls 3.88 and 3.94


3.3.3 Other Feedback Designs

Besides the quadratic Lyapunov designs based on uniform complete control-lability, there are other approaches to design either discontinuous or time-varying feedback controls. In what follows, several representative designs areoutlined.

Discontinuous Controls

For feedback stabilization of Chained System 3.2, a discontinuous control canbe designed using the partial feedback linearization in Section 3.1.3, also calledσ-process [9]. Specifically, in the case that x1(t0) = 0, control component u1

is set to be that in (3.22) which ensures x1(t) = 0 for all finite time; thenTransformation 3.20 is applied to yield the linear time-invariant system of(3.23) for which control component u2 can easily be designed (using either poleplacement [9] or algebraic Riccati equation [143]). In the case that x1(t0) =0 (but ‖x(t0)‖ = 0), a separate control can be designed over time interval[t0, t0 + δt) (for instance, u1(t) = ±c with c > 0 and u2(t) = 0) such thatx1(t0 + δt) = 0, and the control can then be switched to the previous onesince x1(t0 + δt) = 0.

Because the solution of x1(t) to Control 3.22 is x1(t) = x1(t0)e−k(t−t0),

any choice of control u2(t) based on Transformation 3.20 suffers from the samecomputational issue as Control 3.84. Meantime, Transformation 3.20 has thesingularity hyperplane of x1(t)=0, which can cause excessive chattering inthe presence of noises and disturbances. To limit the chattering, the controlmagnitude can be saturated and the resulting stability becomes semi-global[140]. Also, robustification against certain types of noises and disturbancescan be achieved [100, 191].

Instead of employing the σ-process, discontinuous stabilizing controls canalso be designed using a piecewise-analytic state feedback design [22], a slidingmode design [21], and a series of nested invariant manifolds [139].

Time-varying Controls

A 2π-periodic stabilizing control law can be synthesized through constructinga 2π-periodic Lyapunov function. To expose the basic idea, let us consider thethird-order chained system

x1 = u1, x2 = u2, x3 = x2u1. (3.95)

For System 3.95, the smooth time-varying control is given by

u1 = −(x2 + x3 cos t)x2 cos t − (x2x3 + x1),u2 = x3 sin t − (x2 + x3 cos t).

It follows that Lyapunov function


V (t, x) =1

2(x2 + x3 cos t)2 +

1

2x2

1 +1

2x2

3

is 2π-periodic and that its time derivative along the system trajectory is

V = −(x2 + x3 cos t)2 − [(x2 + x3 cos t)x2 cos t + (x2x3 + x1)]2 ,

which is negative semi-definite and also 2π-periodic. Then, asymptotic stabil-ity can be shown by extending LaSalle’s invariant set theorem to a 2π-periodicLyapunov stability argument, which together with the constructive design forChained System 3.2 can be found in [189].

The idea of sinusoidal steering control can be extended to yield time-varying stabilizing control design by using center manifold theory and aver-aging. To illustrate the concept, consider again System 3.95, and choose thesinusoidal feedback control as

u1 = −x1 − x23 sin t, u2 = −x2 − x3 cos t.

The sinusoidal time functions can be generated from the following exogenoussystem:

w1 = w2, w2 = −w1,

where w1(0) = 0 and w2(0) = 1. Hence, the time-varying control can berewritten as

u1 = −x1 − x23w1, u2 = −x2 − x3w2.

Stability of the augmented system (including the exogenous system) can beanalyzed using center manifold theory [108]. It is straightforward to show thatthe local center manifold is given by

x1 ≈ −1

2x2

3(w1 − w2), x2 ≈ −1

2x3(w1 + w2),

and that the dynamics on the center manifold is approximated by

x3 ≈ −1

4x3

3(w1 − w2)2,

which is asymptotically stable using the averaging analysis. For nth-ordernon-holonomic systems, a sinusoidal stabilizing control can be designed forPower Form 3.34, and local stability can be concluded using center manifoldand averaging analysis [258].

For systems in the Skew-symmetric Chained Form 3.36, a time-varyingfeedback control is given by

u1 = −kuz1 + h(Z2, t), w2 = −kwzn,

where ku and kw are positive constants, Z2 = [z2, z3, · · · , zn]T , and h(Z2, t) =‖Z2‖2 sin t. Global asymptotical stability of the closed-loop system can be


shown [222] by using Barbalat lemma and a Lyapunov-like argument with theLyapunov function

V (Z2) =1

2

(

z22 +

1

k1z23 +

1

k1k2z24 + · · · + 1

∏n−2j=1 kj

z2n

)

,

where ki > 0 are constants.Additional time-varying feedback controls are also available, for instance,

those designed using homogeneous Lyapunov function [77, 153, 162]. Asymp-totic stabilization can also be cast into the so-called ρ-exponential stabiliza-tion, a weaker concept of exponential stability [154, 162].

Hybrid Control

A hybrid control is a set of analytic feedback control laws that are piecewiseswitched according to a discrete event supervising law. For instance, a hybridfeedback control for the third-order Chained System 3.95 is, within the intervalt ∈ [2πk, 2π(k + 1)),

u1(t) = −x1 + |αk(x)| cos t, u2(t) = −x2 + αk(x) sin t,

where αk(x) is the piecewise-switching function defined by

αk(x) =

αk−1 if x3(2πk)αk−1 ≥ 0γ|αk−1|sign(x3(2πk)) if x3(2πk)αk−1 < 0

,

γ > 0 is a constant, α0 = 0 if x(t0) = 0, and α0 = 0 if x(t0) = 0. More detailson hybrid control designs can be found in [22, 111, 112, 239].

3.4 Control of Vehicle Systems

Consider the control problem for a vehicle system which consists of nr ve-hicles. The vehicles are indexed by set Ω = 1, · · · , nr, and they have con-figuration outputs qi respectively. Individually, each of the vehicles can becontrolled using the path planning and feedback control algorithms discussedin the preceding sections. In this section, we discuss the issues arising fromcontrolling nr vehicles as a group. In particular, we investigate the problemsof how to achieve a vehicle group formation and how to ensure that all thevehicles avoid collision both among the group and with any of the obstaclesin their vicinities. In the study, we assume that output information of eachvehicle are available to all others and for all time. In practice, the informationis collected or transmitted via a sensing and communication network, andthe corresponding analysis and design will be carried out in the subsequentchapters.

3.4 Control of Vehicle Systems 131

)(2 te

)(1 tqd

0x

)(1 te

)(2 tqd

)(3 tqd

)(0 tq

0y

Fig. 3.13. Illustration of a formation in 2-D

3.4.1 Formation Control

Let q0(t) ∈ ℜl be the desired trajectory of the virtual leader specifying thegroup motion of these vehicles. A formation specifies desired geometric re-quirements on relative positions of the vehicles while they move, and it canbe expressed in terms of the coordinates in a motion frame F(t) that moves ac-cording to q0(t). Without loss of any generality, we assume F(t) ∈ ℜl, set theorigin of F(t) to be at q0(t), and choose [e1, e2, · · · , el] to be its l orthonormalbasis vectors. Then, for any given formation consisting of nr vehicles, theirdesired positions in frame F(t) can be expressed as P1, · · · , Pnr

, where

Pi =l∑

j=1

αijej . (3.96)

Alternatively, the formation can also be characterized by its edge vectors: forany i = k,

Pi − Pk =

l∑

j=1

(αij − αkj)ej . (3.97)

Note that αij are constant if the formation is rigid over time and that αij

are time functions if otherwise. Figure 3.13 illustrates a formation and itsframe F (t) for the case of l = 2 and nr = 3. In this case, it follows from

q0(t) =[

x0(t) y0(t)]T

that

e1(t) =

⎡

⎢

⎢

⎣

x0(t)√

[x0(t)]2 + [y0(t)]2

y0(t)√

[x0(t)]2 + [y0(t)]2

⎤

⎥

⎥

⎦

, e2(t) =

⎡

⎢

⎢

⎣

− y0(t)√

[x0(t)]2 + [y0(t)]2

x0(t)√

[x0(t)]2 + [y0(t)]2

⎤

⎥

⎥

⎦

.


The formation control problem is to control the vehicles such that theymove together according to q0(t) and asymptotically form and maintain thedesired formation. In what follows, several approaches of formation controlare described.

Formation Decomposition

Based on the desired formation in (3.96) and on the virtual leader q0(t), thedesired trajectories of individual vehicles can be calculated as

qdi (t) = q0(t) +

l∑

j=1

αijej , i = 1, · · · , nr. (3.98)

Through the decomposition, the formation control problem is decoupled intothe standard tracking problem, and the design in Section 3.3.1 can readily beapplied to synthesize control ui to make qi track qd

i .

Example 3.16. To illustrate the formation decomposition and its correspond-ing control, consider a group of three differential-drive vehicles that follow agiven trajectory while maintaining the formation of a right triangle. Supposethat the 2-D trajectory of the virtual leader is circular as

q0(t) = [2 cos t, 2 sin t]T ,

and hence the moving frame is given by

e1(t) = [− sin t, cos t]T , e2(t) = [− cos t,− sin t]T .

For a right triangle, formation parameters are

α11 = 0, α12 = 0, α21 = −1, α22 = 1, α31 = −1, α32 = −1.

It then follows from (3.98) that the desired individual trajectories are

qd1 =

[

2 cos t2 sin t

]

, qd2 =

[

cos t + sin tsin t − cos t

]

, qd3 =

[

3 cos t + sin t3 sin t − cos t

]

,

respectively. In addition to vehicle positions in the formation, assume that thedesired orientation of the vehicles are aligned as specified by θd

i = t + π/2.As shown in Section 3.1, models of the vehicles can be expressed in the

chained form as, given qi = [xi1 xi2]T ,

xi1 = ui1, xi2 = xi3ui1, xi3 = ui2.

Through Transformations 3.8 and 3.9, the desired trajectories can be mappedinto those for the state of the above chained form:


Fig. 3.14. Performance under formation decomposition and control

⎧

⎪

⎪

⎨

⎪

⎪

⎩

xd11 = t + π

2 ,xd

12 = 2,xd

13 = 0,ud

11 = 1, ud12 = 0,

⎧

⎪

⎪

⎨

⎪

⎪

⎩

xd21 = t + π

2 ,xd

22 = 1,xd

23 = −1,ud

21 = 1, ud22 = 0,

⎧

⎪

⎪

⎨

⎪

⎪

⎩

xd31 = t + π

2 ,xd

32 = 3,xd

33 = −1,ud

31 = 1, ud32 = 0.

Then, the tracking control presented in Section 3.3.1 can be readily applied.Simulation is done for initial configurations [xi(t0), yi(t0), θi(t0)]

T given by[0.5, 0, π

4 ], [1,−2, π6 ] and [0,−1.5, 0], respectively; and phase portraits of the

vehicle trajectories are shown in Fig. 3.14.

Desired trajectory in (3.98) requires calculation of q0(t), and the high-orderderivatives of q0(t) may also be needed to write down appropriate trackingerror systems for the tracking control design. If time derivatives of q0(t) arenot available, standard numerical approximations can be used.

While this approach is the simplest, there is little coordination betweenthe vehicles. As such, the formation is not robust either under disturbance orwith noisy measurements or in the absence of synchronized timing becauseany formation error cannot be corrected or attenuated. In certain applications,q0(t) may be known only to some of the vehicles but not all. To overcomethese shortcomings, certain feedback mechanism among the vehicles shouldbe present.

Ŧ4 Ŧ3 Ŧ2 Ŧ1 0 1 2 3 4Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

4

Vehicle 1Vehicle 2Vehicle 3


Leader-follower Design

Let Ωi ⊂ Ω be a non-empty sub-set that contains the indices of the leadervehicles for the ith vehicle to track. Then, it follows from (3.97) that thedesired trajectory of the ith vehicle can be expressed as

qdi (t) =

∑

k∈Ωi

dik

⎡

⎣qdk(t) +

l∑

j=1

(αij − αkj)ej

⎤

⎦ , (3.99)

where dik > 0 are positive weighting coefficients such that∑

k∈Ωidik = 1. As

long as ∪nr

i=1Ωi = Ω, a tracking control design can be applied to make qi trackthe desired trajectory in (3.99) and in turn to achieve the desired formationasymptotically.

Formation 3.99 requires that ej(t) and hence q0(t) be explicitly availableto all the vehicles. To overcome this restriction, we can adopt a leader-followerhierarchy and decompose the formation in (3.98) or (3.99) further in terms ofnr moving frames attached to individual vehicles (rather than just one movingframe attached to the virtual leader), that is,

qdi (t) =

∑

k∈Ωi

d′ik

⎡

⎣qdk(t) +

l∑

j=1

α′ije

′kj

⎤

⎦ , i = 1, · · · , nr, (3.100)

where e′kj : j = 1, · · · , l are the orthonormal basis vectors of the desired

moving frame Fdk (t) at the kth vehicle, α′

ij for j = 1, · · · , l are the desired

coordinates of the ith vehicle with respect to frame Fdk (t) and according to

the desired formation, and d′ik > 0 are positive weighting coefficients suchthat

∑

k∈Ωid′ik = 1.

Example 3.17. To illustrate the leader-follower design, consider the formationcontrol problem that a leader-following structure is imposed among threedifferential-drive vehicles and that each vehicle is to follow its leader(s) withcertain desired offset(s) while the virtual leader for the group moves along acircular path of q0(t) = [2 cos t, 2 sin t]T .

Suppose the first vehicle is to follow the virtual leader with zero offset,that the second vehicle is to track the first with the offset values α′

21 = −√

2and α′

22 = 2−√

2, and that the third vehicle is to follow the second with theoffset values α′

31 = −√

2 and α′32 = 2 −

√2. It follows (3.100) that

qd1(t) = q0(t), qd

2 = qd1 + α′

21e′11 + α′

22e′12, qd

3 = qd2 + α′

31e′21 + α′

32e′22,

where e′kj (with j = 1, · · · , l) are determined along the trajectory of qdk(t).

That is, we have that

e′11(t) =

[

− sin tcos t

]

, e′12(t) =

[

− cos t− sin t

]

=⇒ qd2 =

√2

[

cos t + sin tsin t − cos t

]

,


Fig. 3.15. Performance under a leader-follower formation control

and that

e′21(t) =

[

− sin t+cos t√2

cos t+sin t√2

]

, e′22(t) =

[

− sin t+cos t√2

cos t−sin t√2

]

=⇒ qd3 =

[

2 sin t−2 cos t

]

.

Once again, the desired orientation of all vehicles are assumed to be θdi =

t + π/2.Then, as did in Example 3.16, the desired trajectories can be mapped into

the chained form as⎧

⎪

⎪

⎨

⎪

⎪

⎩

xd11 = t + π

2 ,xd

12 = 2,xd

13 = 0,ud

11 = 1, ud12 = 0,

⎧

⎪

⎪

⎨

⎪

⎪

⎩

xd21 = t + π

2 ,

xd22 =

√2,

xd23 = −

√2,

ud21 = 1, ud

22 = 0,

⎧

⎪

⎪

⎨

⎪

⎪

⎩

xd31 = t + π

2 ,xd

32 = 0,xd

33 = −2,ud

31 = 1, ud32 = 0,

and the tracking control in Section 3.3.1 can be applied. Performance of theformation control is shown in Fig. 3.15 for the vehicles whose initial configura-tions of [xi(t0), yi(t0), θi(t0)]

T are given by [0.5, 0, π4 ], [1,−2, π

6 ] and [0,−1.5, 0],respectively.

To make the leader-follower hierarchy robust and to avoid calculation ofqdi (t) for most of the vehicles, Desired Trajectory 3.100 can further be changed

to

Ŧ3 Ŧ2 Ŧ1 0 1 2 3Ŧ3

Ŧ2

Ŧ1

0

1

2

3



qdi (t) =

∑

k∈Ωi

d′ik

⎡

⎣qk(t) +l∑

j=1

α′ij e

′kj

⎤

⎦ , i = 1, · · · , nr, (3.101)

where e′kj are the basis vectors of frame Fk(t) located at qk(t) (rather than its

desired frame Fdk (t) located at qd

k(t)). Accordingly, time derivatives of qk(t)would appear in e′kj , in the tracking error system of the ith vehicle, and inthe resulting control ui. However, time derivatives of qk(t) are usually notavailable to the ith vehicle. If we choose to eliminate those derivatives bysubstituting the model and its control of the kth vehicle, we will have morecomplicated error dynamics, and the resulting control ui will require statefeedback of xk and possibly uk as well. Consequently, Ωi should be chosensuch that ∩nr

i=1Ωi is empty in order to avoid the potential of algebraic loopin control design. Alternately, to simplify the design of ui, standard observers

can be used to estimate time derivatives of q(j)k (t) from qk(t).

By designating the layers of leaders and followers, the vehicle system hasa fixed structure of command and control. While uncertainties and distur-bances can now be compensated for, its rigid structure means that, if one ofthe vehicles becomes disabled, all the vehicles supposed to follow that vehicle(among other leader vehicles) are too impacted to keep the rest of the forma-tion intact. In order to make the formation robust, a vehicle should follow theprevailing behavior of its peers. That is, to maintain the formation as muchas possible, every vehicle is commanded to track majority of the vehicles in itsvicinity, which leads to the flexible structure of neighboring feedback design.

Neighboring Feedback

Let Ni(t) ⊂ Ω be the index set of those vehicles neighboring the ith vehicleat time t. In the spirit of (3.101), the desired trajectory can be set to be

qdi (t) =

∑

k∈Ni(t)

d′′ik(t)

⎡

⎣qk(t) +

l∑

j=1

α′ij e

′kj

⎤

⎦ , (3.102)

where d′′ik(t) ≥ 0 and∑

k∈Nid′′ik(t) = 1. In essence, the neighboring approach

makes the ith vehicle adjusts its motion according to its neighbors. Based on(3.102), control ui can be designed to make qi track qd

i provided that Ni(t)changes seldom. In the general case, Ni(t) could change quickly and even un-predictably, Desired Trajectory 3.102 would become discontinuous and notdifferentiable and hence the corresponding error system could not be prede-termined. Formation control under these settings will be accomplished usingcooperative control in Sections 5.3.4 and 6.5.

3.4.2 Multi-objective Reactive Control

One way to make the vehicle system become pliable to environmental changesand to ensure no collision among the vehicles in the system is to update


both q0(t) and qdi (t) using a real-time path planning and collision avoidance

algorithm such as that in Section 3.2. However, this approach has severalshortcomings in applications. First, it usually requires too much information,is too time consuming, and may even be unsolvable in many scenarios toreplan q0(t) so that the whole formation avoids all the obstacles. Second, ifthe ith vehicle replans its desired trajectory, Geometrical Requirement 3.98needs to be relaxed but also be maintained as close as possible. Third, ifall the vehicles replan their desired trajectories, these trajectories must becollision-free among each other, which becomes more involved and may requirecentralized coordination. Finally, in the case that there are many vehicles and(moving) obstacles in a congested area, it would be more efficient for eachvehicle to navigate its local traffic while attempting to relate itself to thevirtual leader q0(t).

As the alternative that addresses these issues, a multi-objective reactivefeedback control is preferred for each vehicle to track either the virtual leaderq0(t) or its leader among the vehicle group while avoiding obstacles. The con-trol design is to synthesize reactive forces based on potential field functions:attractive force from an attractive potential field associated with the vir-tual leader, and repulsive forces from repulsive potential field functions builtaround obstacles and the vehicles. The intuition is that, by having the twokinds of potential field functions interact with each other, the multiple controlobjectives are integrated into one by which obstacle avoidance is the priorityduring the transient and goal tracking is to be achieved asymptotically.

Composite Potential Field Function

Consider the following composite potential field function for the ith vehicle:

P (qi − qdi , qi − qoj

) = Pa(qi − qdi ) +

∑

j∈Ni

Prj(qi − qoj

), (3.103)

where q ∈ ℜl, Ni(t) is the set containing all the indices of the obstacles(including other vehicles) in a neighborhood around the ith vehicle (accordingto its sensing range), Pa(·) is the attractive potential field function, Prj

(·) isthe repulsive potential field function around the jth obstacle, qd

i (t) representsits desired position (which may be calculated according to (3.102)), and qoj

(t)is the location of the jth obstacle. In what follows, both Pa(s) and Pr(s) areassumed to be differentiable up to the second-order.

To generate an attractive force which moves the vehicle toward its goal,the attractive potential field function should have the following properties:

(i) Pa(s) is positive definite and locally uniformly bounded,

(ii) sT ∂Pa(s)

∂sis positive definite and locally uniformly bounded.

(3.104)

In (3.104), Property (i) is the basic characteristics of an attractive potentialfield function, and Property (ii) implies that the negative gradient−∂Pa(s)/∂s


can be used a feedback control force to make the value of Pa decrease overtime. On the other hand, for collision avoidance, the repulsive potential fieldfunctions should be chosen such that

⎧

⎨

⎩

(i) Prj(s) = +∞ if s ∈ Ωoj

, and lims→Ωoj

, s∈Ωoj

∥

∥

∥

∥

∂Prj(s)

∂s

∥

∥

∥

∥

= +∞,

(ii) Prj(s) = 0 for s ∈ Ωoj

, and Prj(s) ∈ (0,∞) for all s ∈ Ωoj

∩ Ωcoj

,

(3.105)

where Ωoj⊂ ℜl is the compact set representing the envelope of the jth ob-

stacle, Ωcoj

is the complement set of Ωojin ℜl, and compact set Ωoj

beingthe enlarged version of Ωoj

represents the region in which the repulsive forcebecomes active. In (3.105), Property (ii) states that Prj

(·) is positive semi-definite, and Property (i) implies that configuration q cannot enter regionΩoj

without an infinite amount of work being done. In digital implementa-tion, a maximum threshold value on Prj

(·) may be imposed, in which casecollision avoidance can be ensured by either an adaptive sampling scheme(that is, as the value of Prj

(·) increases, the sampling period is reduced) or avelocity-scaled repulsive potential field function (i.e., the higher the speed ofthe vehicle, the larger compact set Ωoj

while the maximum value on Prj(·) is

fixed).Composite Potential Field Function 3.103 will be used to generate a control

Lyapunov function based on which a reactive control is designed. It should benoted that, to ensure asymptotical convergence, additional conditions on qd

i ,qoj

and P (·) will be required.

Assumption 3.18 Desired trajectory qdi can be tracked asymptotically, that

is, after some finite time instant t∗, [qdi (t) − qoj

(t)] ∈ Ωojfor all t ≥ t∗ and

for all j ∈ Ni(t). Accordingly, repulsive potential field functions Prjshould be

chosen such that [qdi (t) − qoj

(t)] ∈ Ωojfor all t ≥ t∗ and for all j ∈ Ni(t).

Assumption 3.19 In the limit of t → ∞, Composite Potential Field Func-tion 3.103 has qd

i (t) as the only local minimum around qi(t).

Obviously, Assumption 3.18 is also necessary for qi to approach qdi asymp-

totically. In general, there are more than one “stationary” solution q∗i to thealgebraic equation ∂P (q∗i − qd

i , q∗i − qoj)/∂q∗i = 0. It follows from Assumption

3.18 and the properties in (3.104) that q∗i = qdi is the global minimum of

function P (q∗i − qdi , q∗i − qoj

). If the obstacles form a trap encircling the ithvehicle but not its desired trajectory, Assumption 3.19 does not hold. Neitherdoes Assumption 3.19 have to hold during the transient. Without loss of anygenerality, we assume that Ωoj

∩Ωokbe empty for j = k. Hence, we can solve

the algebraic equation

∂P (q∗i − qdi , q∗i − qoj

)

∂q∗i=

∂Pa(q∗i − qdi )

∂q∗i+

∂Prj(q∗i − qoj

)

∂q∗i= 0 (3.106)


for any pair of Pa, Prj and analyze the property of solution q∗i during the

transient as well as in the limit.For instance, consider the following typical choices of potential field func-

tions:Pa(q − qd) = λa[(x − xd)2 + (y − yd)2], (3.107)

and

Pr(q − qo) =

⎧

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎩

+∞ if ‖q − qo‖2 ≤ r2o ,

λr

[

ln(ro + ǫo)

2 − r2o

‖q − qo‖2 − r2o

− (ro + ǫo)2 − ‖q − qo‖2

(ro + ǫo)2 − r2o

]

if r2o ≤ ‖q − qo‖2 ≤ (ro + ǫo)

2,0 if ‖q − qo‖2 ≥ (ro + ǫo)

2,

(3.108)

where q = [x y]T ∈ ℜ2, qd = [xd yd]T , qo = [xo yo]T , λa, λr > 0 are constant

gains, Ωo = q : ‖q − qo‖ ≤ ro, and Ωo = q : ‖q − qo‖ ≤ ro + ǫo withro, ǫo > 0. In the case that yd = yo and xd − xo > ro + ǫo, the correspondingtotal potential force of ∂P (q− qd, q− qo)/∂q has its vector field shown in Fig.3.16. Clearly, q = qd is the only global minimum, and there is a saddle pointon the left of the obstacle.

In general, it follows from (3.106) that stationary point q∗ exists only atlocation(s) where repulsive and attractive force balance each other as

∂Pa(q∗ − qd)

∂q∗= −∂Pr(q

∗ − qo)

∂q∗. (3.109)

Hence, if qd ∈ Ωo, there is no stationary point outside Ωo except for theglobal minimum at qd. Inside Ωo, additional stationary point(s) may exist.The following lemma provides the condition on the stationary point being asaddle point, which ensures Assumption 3.19.

Lemma 3.20. Suppose that the level curves of Pa(q − qd) = ca and Pr(q −qo) = cr are closed and convex, that these two families of level curves aretangent to each other along a straight-line segment, and that qd ∈ Ωo ⊂ ℜ2.Then, q∗ ∈ Ωo is a saddle point if the curvature of Pa(q − qd) = ca at q = q∗

is smaller than that of Pr(q − qo) = cr at q = q∗.

Proof: Consider any stationary point q∗ ∈ Ωo and, at q∗, establish a localcoordinate system of q′ = [x′ y′]T such that its origin is at q = q∗ and its y′

axis is along the direction of ∂Pa(q∗ − qd)/∂q∗. That is, according to (3.109),axis y′ points toward the obstacle and also the goal qd further along, it isparallel to the normal vector of both level curves of Pa(q − qd) = ca andPr(q − qo) = cr at q′ = 0, and the x′ axis is along the tangent direction ofboth level curves of Pa(q − qd) = ca and Pr(q − qo) = cr at q′ = 0. Therefore,we know from the geometry that q = Rq′ + q∗ for some rotational matrix R,that

∂Pa

∂x′

∣

∣

∣

∣

q′=0

=∂Pr

∂x′

∣

∣

∣

∣

q′=0

= 0 (3.110)


Fig. 3.16. Vector field of ∂P (q − qd, q − qo)/∂q under (3.107) and (3.108)

and∂Pa

∂y′

∣

∣

∣

∣

q′=0

= −∂Pr

∂y′

∣

∣

∣

∣

q′=0

> 0, (3.111)

and that, since the level curves are closed and convex,

∂2Pa

∂(y′)2

∣

∣

∣

∣

q′=0

< 0,∂Pr

∂(y′)2

∣

∣

∣

∣

q′=0

< 0. (3.112)

By assumption and according to (3.110), the two sets of level curves are tan-gent along the y′ axis and consequently equations

∂Pa

∂x′

∣

∣

∣

∣

q′=[0 δy′]T=

∂Pr

∂x′

∣

∣

∣

∣

q′=[0 δy′]T= 0

hold for all small δy′, which in turn implies

∂2P

∂y′∂x′

∣

∣

∣

∣

q′=0

=∂2P

∂x′∂y′

∣

∣

∣

∣

q′=0

= limδy′→0

∂P∂x′

∣

∣

q′=[0 δy′]T− ∂P

∂x′

∣

∣

q′=0

δy′ = 0. (3.113)

Stability of stationary point q∗ is determined by Hassian matrix ∂2P/∂(q′)2

at that point. Therefore, it follows from (3.112) and (3.113) that q∗ is a saddlepoint if

∂2Pa

∂(x′)2

∣

∣

∣

∣

q′=0

+∂Pr

∂(x′)2

∣

∣

∣

∣

q′=0

> 0, (3.114)

and it is a stable node if the above inequality is reversed.


On the other hand, the curvature of function y = f(x) is (d2y/dx2)/[1 +(dy/dx)2]3/2. Taking the first-order derivative along the level curve of Pa(q −qd) = ca yields

dya

dx′

∣

∣

∣

∣

q′=0

= −∂Pa

∂x′

∂Pa

∂y′

∣

∣

∣

∣

∣

q′=0

= 0,

in which (3.110) is used, and ya = ya(x′) is the solution to the implicit equa-tion of Pa(q − qd) = ca. Using the above result and taking the second-orderderivative along the level curve of Pa(q − qd) = ca yields

dy2a

d(x′)2

∣

∣

∣

∣

q′=0

= −∂2Pa

∂(x′)2

∂Pa

∂y′

∣

∣

∣

∣

∣

∣

q′=0

, (3.115)

whose absolution value is the curvature of level curve Pa(q − qd) = ca. Simi-larly, it follows that the curvature of level curve Pr(q−qo) = cr is the absolutevalue of

dy2r

d(x′)2

∣

∣

∣

∣

q′=0

= −∂2Pr

∂(x′)2

∂Pr

∂y′

∣

∣

∣

∣

∣

∣

q′=0

. (3.116)

Recalling that the level curves are convex and hence have opening upwardsin the coordinate of q′ and around q′ = 0, we know that dy2

a/d(x′)2 anddy2

r/d(x′)2 defined by (3.115) and (3.116) are both positive. It follows from(3.111) that

∂2Pa

∂(x′)2

∣

∣

∣

∣

q′=0

< 0,∂Pr

∂(x′)2

∣

∣

∣

∣

q′=0

> 0.

Hence, Inequality 3.114 can be concluded by comparing curvatures.

Lemma 3.20 can be visualized graphically, and its proof also implies that q∗

is a stable node (and hence a local minimum) if the curvature of Pa(q−qd) = ca

at q = q∗ is larger than that of Pr(q − qo) = cr at q = q∗. Presence ofstable node(s) in Ωo should be avoided by properly selecting the repulsivepotential field function for any given obstacle. The global minimum solutionof qd characterizes the tracking control objective, and the rest of stationarysolutions q∗ introduced inevitably by the presence of repulsive potential fieldfunctions should be saddle points in order to achieve both the objectivesof collision avoidance and goal tracking. Nonetheless, the presence of saddlepoints q∗i needs to be accounted for in the subsequent reactive control design.

Reactive Control

To simplify the technical development, let us consider the case that dynamicfeedback linearization has been applied (as in Section 3.1.3) to transfer thevehicle model into


qi = ui, (3.117)

where qi ∈ ℜl is the coordinate of configuration output variables, and ui isthe control to be designed. Then, the multi-objective reactive control can bechosen to be

ui = −∂P (qi − qdi , qi − qoj

)

∂qi− ξi(qi − qd

i , qi − qoj)(qi − qd

i ) + qdi

−k∑

j∈Ni

∂2Prj(qi − qoj

)

∂(qi − qoj)2

(qi − qoj)‖qd

i − qoj‖2

−2k∑

j∈Ni

∂Prj(qi − qoj

)

∂(qi − qoj)

(qdi − qoj

)T (qdi − qoj

)

+ϕi(q∗i , qi − qd

i , qi − qoj) (3.118)

where ξi(·) ≥ 0 is the non-linear damping gain which vanishes as (qi − qoj)

approaches set Ωojand is uniformly positive anywhere else (i.e., ξi = ξi(qi −

qdi ) > 0 if (qi − qoj

) ∈ Ωoj), k > 0 is a gain associated with the terms

that resolve the potential conflict between asymptotic tracking and collisionavoidance, and ϕi(·) is a uniformly bounded perturbation term which is non-zero only if the vehicle stays idle (in the sense that qi − qd

i = qi − qoj= 0)

at saddle point q∗i (which can be solved from (3.106)). At the saddle point, asimple choice of ϕi(·) would be a bounded force along the tangent directionof the level curve of Prj

(qi − qoj) = Prj

(q∗i − qoj).

Collision avoidance is established in the following theorem by showing that,under Control 3.118, the vehicle never enters the moving set Ωoj

around qoj.

Asymptotic tracking convergence is also shown under Assumptions 3.18 and3.19.

Theorem 3.21. Suppose that Potential Field Function 3.103 satisfies Prop-erties 3.104 and 3.105, where [qd

i (t)− qoj(t)] and qd

i (t) are uniformly bounded.Then, as long as qi(t0) ∈ Ωoj

, System 3.117 under Control 3.118 is collision-free. Furthermore, if Assumptions 3.18 and 3.19 hold, qi converges asymptot-ically to qd

i (t).

Proof: Should moving sets Ωojand Ωok

overlap for some j = k and duringcertain time period, the two obstacles can be combined into one obstacle.Thus, we need only consider in the proof of collision avoidance that compactmoving sets Ωoj

are all disjoint in the configuration space.Consider first the simpler case that qd

i − qoj= 0. In this case, Control 3.118

with ϕi(·) = 0 reduces to

ui = −∂P (qi − qdi , qi − qoj

)


i , qi − qoj)(qi − qd

i ) + qdi . (3.119)

Choose the Lyapunov function candidate


V1(t) =1

2‖qi − qd

i ‖2 + V0(t),

whereV0(t) = Pa(qi − qd

i ) +∑

j

Prj(qi − qoj

).

It is obvious that V1(t0) is finite for any finite and collision-free initial condi-tions. It follows that, along the trajectory of System 3.117 and under Control3.119,

V1 = (qi − qdi )T (qi − qd

i ) + (qi − qdi )T ∂Pa(qi − qd

i )

∂(qi − qdi )

+∑

j∈Ni(t)

(qi − qoj)T ∂Prj

(qi − qoj)

∂(qi − qoj)

= −ξi(qi − qdi , qi − qoj

)‖qi − qdi ‖2 +

∑

j∈Ni(t)

(qdi − qoj

)T ∂Prj(qi − qoj

)

∂(qi − qoj)

= −ξi(qi − qdi , qi − qoj

)‖qi − qdi ‖2,

which is negative semi-definite. Thus, V1(t) is non-increasing as V1(t) ≤ V1(t0)for all t, Prj

(qi − qoj) < ∞ for all t and for all j, and hence the vehicle is

collision-free.For the general case of qd

i − qoj= 0, let us consider the Lyapunov function

V2(t) =1

2

∥

∥

∥

∥

∥

∥

qi − qdi + k

∑

j

∂Prj(qi − qoj

)

∂(qi − qoj)

‖qdi − qoj

‖2

∥

∥

∥

∥

∥

∥

2

+ V0(t).

Similarly, V2(t0) is finite for any finite and collision-free initial conditions. Itfollows from (3.117) and (3.118) that

V2 =

⎡

⎣qi − qdi + k

∑

j∈Ni

∂Prj(qi − qoj

)

∂(qi − qoj)

‖qdi − qoj

‖2

⎤

⎦

T

[

qi − qdi

+k∑

j∈Ni

∂2Prj(qi − qoj

)

∂(qi − qoj)2

(qi − qoj)‖qd

i − qoj‖2

+2k∑

j∈Ni

∂Prj(qi − qoj

)

∂(qi − qoj)

(qdi − qoj

)T (qdi − qoj

)

⎤

⎦+ (qi − qdi )T ∂Pa(qi − qd

i )

∂(qi − qdi )

+∑

j∈Ni

(qi − qoj)T ∂Prj

(qi − qoj)

∂(qi − qoj)

= −k∑

j∈Ni

∥

∥

∥

∥

∂Prj(qi − qoj

)

∂(qi − qoj)

∥

∥

∥

∥

2

‖qdi − qoj

‖2 − ξi(qi − qdi , qi − qoj

)‖qi − qdi ‖2

+w(qi − qdi , qi − qoj

, qi − qdi , qi − qoj

), (3.120)


where

w(qi − qdi , qi − qoj


)

=∑

j∈Ni

(qdi − qoj

)T ∂Prj(qi − qoj

)

∂(qi − qoj)

−k∑

j∈Ni

(

∂Prj(qi − qoj

)

∂(qi − qoj)

)T∂Pa(qi − qd

i )

∂(qi − qdi )

‖qdi − qoj

‖2

−k∑

j∈Ni

(

∂Prj(qi − qoj

)

∂(qi − qoj)

)T

ξi(qi − qdi , qi − qoj

)(qi − qdi )‖qd

i − qoj‖2.

It follows from (3.104) and from qdi (t) being uniformly bounded that, for any qi

satisfying (qi−qoj) ∈ Ωoj

, ∂Pa(qi−qdi )/∂(qi−qd

i ) is uniformly bounded. Hence,we know from (qi − qoj

) being uniformly bounded that, for some constantsc1, c2 ≥ 0,

|w(qi − qdi , qi − qoj


)|

≤ c1

∑

j∈Ni

∥

∥

∥

∥

∂Prj(qi − qoj

)

∂(qi − qoj)

∥

∥

∥

∥

‖qdi − qoj

‖

+c2

∑

j∈Ni

ξi(qi − qdi , qi − qoj

)

∥

∥

∥

∥

∂Prj(qi − qoj

)

∂(qi − qoj)

∥

∥

∥

∥

‖qdi − qoj

‖‖qi − qdi ‖.

Substituting the above inequality into (3.120), we can conclude using theproperty of ξi(qi − qd

i , qi − qoj) that, as (qi − qoj

) → Ωoj, V2 < 0. Thus, V2(t)

will be finite in any finite region and hence there is no collision anywhere.Since perturbation force ϕi(·) is uniformly bounded and is only active atsaddle points, its presence does not have any impact on collision avoidance.

To show asymptotic convergence under Assumptions 3.18 and 3.19, wenote from Lemma 3.20 (and as illustrated by Fig. 3.16) that after t = t∗,the vehicle cannot stay in Ωoj

indefinitely and upon leaving Ωojthe tracking

error dynamics of System 3.117 and under Control 3.118 reduce to

ei1 = ei2, ei2 = −∂Pa(ei1)

∂ei1− ξi(ei1)ei2,

where ei1 = qi − qdi and ei2 = qi − qd

i . Adopting the simple Lyapunov function

V3(t) = Pa(ei1) +1

2‖ei2‖2

we have

V3 = eTi2

∂Pa(ei1)

∂ei1+ eT

i2

[

−∂Pa(ei1)

∂ei1− ξi(ei1)ei2

]

= −ξi(ei1)‖ei2‖2,


from which asymptotic convergence can be concluded using Theorem 2.21.

It is worth noting that trajectory planning and reactive control can becombined to yield better performance toward both collision avoidance andgoal tracking. Specifically, if moving sets Ωoj

are all relatively far apart fromeach other, Control 3.118 can be simplified to be

ui = −∂P (qi − qdi , qd

i − qoj)


i , qi − qoj)(qi − ˙q

d

i ) + ¨qd

i . (3.121)

where qdi is the re-planned version of qd

i such that qdi never enters set Ωoj

around qoj(although qd

i may enter temporarily), ˙qd

i − qoj= 0 holds whenever

(qdi − qoj

) enters set Ωoj, and limt→∞ qd

i − qdi = 0. In other words, when

(qdi − qoj

) enters moving set Ωoj, qd

i (t) is kept outside by choosing qdi such

that (qdi − qoj

) is constant; and when (qdi − qoj

) exits set Ωoj, qd

i (t) is replannedto track qd

i asymptotically or in finite time. Comparing (3.121) and (3.119), weknow that Control 3.121 is collision-free. It follows from the proof of Theorem3.21 that Control 3.121 also makes (qd

i − qdi ) asymptotically convergent. The

uniformly bounded detouring force ϕi(·) added in Control 3.118 is to move thevehicle away from any saddle equilibrium point so that asymptotic trackingbecomes possible, and hence it should also be added into (3.121). Finally,in the proof of Theorem 3.21, a control Lyapunov function can be found toreplace V3 once the expression of attractive potential field function Pa(·) isavailable.

To illustrate performance of the multi-objective reactive control in (3.118),consider the 2-D motion that one vehicle is in the form of (3.117), has its initialposition at q(t0) = [2.5 1]T , and is to follow the desired trajectory given by

qd(t) =[

2 cos π20 t 2 sin π

20 t]T

.

In this case, Reactive Control 3.118 is designed with potential field functions

Pa(q) =1

2λa‖q − qd‖2

Prj(q) =

⎧

⎪

⎪

⎨

⎪

⎪

⎩

1

2λrj

(

1

‖q − qoj‖ − ρj

− 1

ρj + ǫj

)2

if ρj < ‖q − qoj‖ ≤ ρj + ǫj ,

0 if ‖q − qoj‖ > ρj + ǫj ,

(3.122)

where λa, λrj, ρj and ǫj are positive constants. Two scenarios are simulated.

In the first scenario, there is only one static obstacle located at qo1 = [−2 0]T .Control 3.118 is implemented with design parameters ξ = 2, k = 20, ϕi = 0,λa = 5, λr1 = 10, and ρ1 = ǫ1 = 0.2, and the simulation result is shown in Fig.3.17. In the second scenario, there is additionally one moving obstacle whichoscillates between the two points of (1, 0) and (3, 0), parallel to the x axis,


(a) Phase portrait

0 10 20 30 40 50 60 70 80

Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

Time (sec)

u1

u2

(b) Control inputs

Fig. 3.17. Reactive Control 3.118: a static obstacle

Ŧ2.5 Ŧ2 Ŧ1.5 Ŧ1 Ŧ0.5 0 0.5 1 1.5 2 2.5

Ŧ2.5

Ŧ2

Ŧ1.5

Ŧ1

Ŧ0.5

0

0.5

1

1.5

2

2.5

Vehicle trajectoryDesired trajectory


and with a constant velocity of ±1/40. Control 3.118 is then implementedwith additional design parameters of λr2 = 30 and ρ2 = ǫ2 = 0.2, and thesimulation results are given in Figs. 3.18 and 3.19. In the figures, the obstaclesare represented by the larger circles, and the vehicle is denoted by the smallercircle. Figure 3.18(a) shows all the trajectories over the time period of t ∈[0, 160]; that is, the vehicle passes through the points numbered 1 up to 14 andin the direction along the marked arrows, while the moving obstacle oscillatesonce from the starting point (3, 0) to point (1, 0) and back. As illustratedby the snapshots in Fig. 3.19(a) and (b), the vehicle successfully avoids bothobstacles while tracking the desired trajectory.


The chained form is the canonical model for analyzing non-holonomic systems[123, 168]. By their nature, non-holonomic systems are small-time controllablebut not uniformly completely controllable in general, and they can be partiallydynamically feedback linearizable but cannot be asymptotically stabilized un-der any smooth time-invariant feedback control. Accordingly, the open-loopand closed-loop control problems of non-holonomic systems are interesting,and several approaches to solve these problems are presented in this chap-ter. Specifically, open-loop steering controls can be parameterized in termsof basis functions (such as sinusoidal functions, piecewise-constant functions,and polynomial functions). To follow a desired trajectory generated under anon-vanishing input, a feedback tracking control can be designed using theLyapunov direct method and Riccati equation. For asymptotic stabilizationwith respect to a fixed-point, a feedback control should be either discontinuousor time-varying or both. Among the feedback control design approaches, thetechnique of recovering uniform complete controllability and then solving forquadratic Lyapunov function from the Riccati equation has the advantages ofsimplicity and inverse optimality.

Without considering obstacles, motion planning under non-holonomic con-straints is to generate appropriate steering controls, and among the availableapproaches are differential geometry [250, 251], differential flatness [67], in-put parameterization [159, 170, 259], the spline method [59, 126, 188], andoptimal control [64]. It is shown through recasting the non-holonomic motionplanning problem as an optimal control problem that the feasible and shortestpath between two boundary conditions is a concatenation of arcs and straight-line segments belonging to a family of 46 different types of three-parametersteering controls [211, 251]. Trajectories planned for non-holonomic systemsneed to be both collision-free and feasible. For obstacles moving with knownconstant velocities, collision avoidance can be achieved using the concept ofvelocity cone [66]. In [206, 279], a pair of time and geometrical collision avoid-ance criteria are derived, and an optimized trajectory planning algorithm isproposed to handle both non-holonomic constraints of a vehicle and its dy-


(a) Phase portrait

0 20 40 60 80 100 120 140 160Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

4

5

Time (sec)

u1

u2

(b) Reactive control

Fig. 3.18. Performance of reactive control: static and moving obstacles

Ŧ3 Ŧ2 Ŧ1 0 1 2 3

Ŧ3

Ŧ2

Ŧ1

0

1

2

3Vehicle trajectoryDesired trajectoryMoving obstacle

1

2

3

4

5

6

7

8

9

10

11

12

13

14


(a) Phase portrait for t ∈ [1, 44]

(b) Phase portrait for t ∈ [117, 125]

Fig. 3.19. Reactive Control 3.118: static and moving obstacles

Ŧ3 Ŧ2 Ŧ1 0 1 2 3

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

Vehicle trajectoryDesired trajectoryMoving obstacle

t=40

t=42

t=44

t=0

∈

Ŧ3 Ŧ2 Ŧ1 0 1 2 3

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

Vehicle trajectoryDesired trajectoryMoving obstacle

t=117

t=119

t=121

t=123

t=125

moving obstacle


namically changing environment. On the other hand, standard approaches ofmotion planning deal with holonomic systems in the presence of static obsta-cles [122], and they include potential field [36, 95, 109, 218], vector field his-togram [26], exhaustive search methods [15, 58, 93, 125], the iterative method[53, 106], dynamic programming [55] and optimal control [124, 249]. Thesemethods could also be used to generate feasible trajectories by embeddingeither an integration or approximation component, both of which may becomputationally involved.

For a vehicle system, formation control of multiple vehicles can be de-signed using several approaches [6]. For instance, navigation and explorationcan be pursued using cellular robot systems [17], or relative geometry approach[269], or artificial intelligence methods [167], or a probabilistic approach [69],or computer architectural method [183]. There is the behavior-based approach[7, 13, 31, 150] by which a set of primitive behaviors such as moving-to-goal,avoiding-obstacle and maintaining-formation are prescribed, their importanceare represented by a set of weights, and the combined behavior is determinedby aggregating the weights. And, basic artificial behaviors can be improvingby setting up a reward function and its learning mechanism, for example, re-inforcement learning [151]. The notion of string stability [253] can be used forline formations. To achieve precisely a general formation, one option is to im-pose a virtual structure on geometrical relationship among the vehicles, anda centralized formation control can be designed to minimize vehicles’ errorswith respect to the virtual structure [131]. The virtual structure can also bebroken down using a set of moving frames so the formation problem becomesa set of individual tracking problems [105, 231]. The other option of maintain-ing a formation is to adopt a leader-following strategy [50, 257] by which theformation control problem is reduced to a tracking control problem. Artificialpotential field functions have been used for distributed control of a multi-vehicle formation. In [128, 177], virtual leaders and artificial potentials areused to maintain a group geometry, although a unique desired formation maynot be achieved and several virtual vehicles may be required to yield a com-posite potential field function with a unique minimum. In [179], a formationgraph and its structural potential field functions are used to design distributedformation control for achieving a desired formation, and avoidance of staticobstacles may be achieved if two kinds of special-purpose agents can be phys-ically added through flocking protocols [180]. Cooperative control of unicyclerobots have been studied by employing the combination of discontinuous con-trol, non-smooth Lyapunov theory and graph theory [52], or by using thecombination of Frenet-Serret model, time-varying control and average theory[135, 164], or by assuming constant driving velocity [227, 278]. By imposingseveral conditions including freezing the reference trajectory within avoidancezone, collision avoidance and formation control of differential-drive vehicles arestudied in [148, 246], and tracking errors are ensured to be bounded. In Section3.4.1 and later in Section 5.3.4, the formation control problem is formulatedtoward designing a neighboring-feedback cooperative and pliable formation


control. Explicit conditions on how to choose an appropriate composite poten-tial field function are found, and a reactive control is provided for each vehicleto asymptotically track its goal while avoiding static objects, moving obsta-cles and other vehicles. The multi-objective reactive control design, originallyproposed in [42], makes the vehicles pliable to their dynamical surroundings.In Section 6.6, this pliable control will be combined into cooperative controlto achieve any cooperative and pliable formation of vehicles in a dynamicallychanging environment, while the consensus problem of non-holonomic systemswill be addressed in Section 6.5.

4

Matrix Theory for Cooperative Systems

In this chapter, non-negative matrices and their properties are introduced.Among the important concepts to characterize non-negative matrices are ir-reducibility and reducibility, cyclicity and primitiveness, and lower-triangularcompleteness. The so-called Perron-Frobenius theorem provides the funda-mental results on eigenvalues and eigenvectors of a non-negative matrix.

Non-negative matrices arise naturally from systems theory, in particular,non-negative matrices for discrete-time positive systems and/or cooperativesystems, row-stochastic matrices for Markov chains and cooperative systems,Metzler matrices for continuous-time positive systems and/or cooperative sys-tems, and M-matrices for asymptotically stable systems. Geometrical, graph-ical and physical meanings of non-negative matrices are explicitly shown.

Analysis tools and useful properties of the aforementioned matrices aredetailed for their implications in and applications to dynamical systems, es-pecially cooperative systems. Specifically, eigenvalue properties are explored,existence of and explicit solution to Lyapunov function are determined, andconvergence conditions on matrix sequences are obtained.

4.1 Non-negative Matrices and Their Properties

Consider two matrices/vectors E, F ∈ ℜr1×r2 . The notations of E = F , E ≥F , and E > F are defined with respect to all their elements as, for all i andj, eij = fij , eij ≥ fij , and eij > fij , respectively. Operation E = |F | of anymatrix F is defined element-by-element as eij = |fij |. Matrix/vector E is saidto be non-negative if E ≥ 0 and positive if E > 0. The set of all non-negativematrices is denoted by ℜr1×r2

+ .Matrix Jr×r ∈ ℜr×r

+ and vector 1r ∈ ℜr+ are the special positive matrix

and vector, respectively, whose elements are all 1. Matrix E is said to bebinary if its elements are either 0 or 1. Matrix E ∈ ℜr×r

+ is said to be diagonally

positive if eii > 0 for all i = 1, · · · , r. Matrix E ∈ ℜr1×r2+ is said to be row-

154 4 Matrix Theory for Cooperative Systems

stochastic if E1r2 = 1r1 , and it is said to be column-stochastic if ET is row-stochastic.

4.1.1 Reducible and Irreducible Matrices

Non-negative matrices have special properties, and the most distinctive isirreducibility or reducibility that captures physical connectivity or topologicalstructure of any dynamical system associated with the given matrix.

Definition 4.1. A non-negative matrix E ∈ ℜr×r with r ≥ 2 is said to be

reducible if the set of its indices, I = 1, 2, · · · , r, can be divided into two

disjoint non-empty sets S = i1, i2, · · · , iμ and Sc

= I/S = j1, j2, · · · , jν(with μ + ν = r) such that eiαjβ

= 0, where α = 1, · · · , μ and β = 1, · · · , ν.Matrix E is said to be irreducible if it is not reducible.

Permutation matrix corresponds to reordering of the indices, and its ap-plication as a coordinate (or state) transformation rearranges matrix rowsand columns. It follows from Definition 4.1 and its associated permutationmatrix that the following lemma provides the most basic properties of botha reducible matrix and an irreducible matrix. Hence, the lower triangularstructure of matrix F\ in (4.1) is the canonical form for reducible matrices.

Lemma 4.2. Consider matrix E ∈ ℜr×r+ with r ≥ 2. If E is reducible, there

exist an integer p > 1 and a permutation matrix T such that

T T ET =

⎡

⎢

⎢

⎢

⎣

F11 0 · · · 0F21 F22 · · · 0...

.... . .

...Fp1 Fp2 · · · Fpp

⎤

⎥

⎥

⎥

⎦

= F\, (4.1)

where Fii ∈ ℜri×ri is either square and irreducible sub-matrices of dimensionhigher than 1 or a scalar, and

∑pi=1 ri = r. If E is irreducible, vector z′ =

(Ir×r + E)z has more than η positive entries for any vector z ≥ 0 containingexactly η positive entries, where 1 ≤ η < r and Ir×r ∈ ℜr×r is the identitymatrix.

The following corollaries can directly be concluded from Lemma 4.2, andthey can be used as the two simple tests on irreducibility or reducibility.

Corollary 4.3. Consider matrix E ∈ ℜr×r+ . Then, if and only if E is irre-

ducible, inequality γz ≥ Ez with constant γ > 0 and vector z ≥ 0 implieseither z = 0 or z > 0.

Corollary 4.4. Consider matrix E ∈ ℜr×r+ . Then, E is irreducible if and only

if (cIr×r + E)r−1 > 0 for any scalar c > 0. If all the matrices in sequence

4.1 Non-negative Matrices and Their Properties 155

E(k) are irreducible and diagonally positive, E(k + η) · · ·E(k + 1)E(k) > 0for some 1 ≤ η ≤ r − 1 and for all k.

An irreducible matrix needs to be analyzed as a whole; and by utilizingthe lower triangular structure in (4.1), analysis of a reducible matrix couldbe done in terms of both irreducible blocks on the diagonal and blocks in thelower triangular portion.

Definition 4.5. A reducible matrix E is said to be lower triangularly completeif, in its canonical form of (4.1) and for every 1 < i ≤ p, there exists at leastone j < i such that Fij = 0. It is said to be lower triangularly positive if, in(4.1), Fij > 0 for all j < i.

Physical and geometrical meanings of a lower triangularly complete matrixwill be provided in Section 4.2, and its role in matrix sequence convergencewill be studied in Section 4.4.

Example 4.6. Binary matrix

A1 =

⎡

⎣

0 1 00 0 11 0 0

⎤

⎦

is irreducible, which can be easily verified by Definition 4.1. Alternatively, thisconclusion can be shown using either Corollary 4.3 or Corollary 4.4; that is,z = A1z yields z = z11, or

(I + A1)2 > 0.

Identity matrix I of dimension higher than 1 and its permutated versionsof dimension higher than 2 are all reducible. Again, this can be shown byDefinition 4.1 or by one of the two tests.

Matrix

A2 =

⎡

⎣

1 0 0a21 1 0a31 a32 1

⎤

⎦

is reducible for any a21, a31, a32 ≥ 0. Non-negative matrix A2 is lower trian-gularly complete if a21 > 0 and a31 + a32 > 0.

4.1.2 Perron-Frobenius Theorem

The Perron-Frobenius theorem, given below as Theorem 4.8, is the fundamen-tal result on eigenvalue and eigenvector analysis of non-negative matrices,positive matrices, and irreducible matrices. If A is reducible, A can be per-muted into its lower triangular canonical form A′, and then Statement (c)can be applied to the resulting diagonal blocks A′

ii, and the spectrum of Ais the union of the spectra of the A′

ii’s. Proof of Perron-Frobenius theorem isfacilitated by the following Brouwer’s fixed-point theorem.


Theorem 4.7. Let K ⊂ ℜn be compact and convex, and let f : K → Kbe continuous. Then, f has a fixed-point, i.e., there exists x ∈ K such thatf(x) = x.

Theorem 4.8. Consider A ∈ ℜn×n+ . Then,

(a) Spectrum radius ρ(A) ≥ 0 is an eigenvalue and its corresponding eigen-vector is non-negative.

(b) If A is positive, ρ(A) > 0 is a simple eigenvalue of A (of multiplicity 1,both algebraically and geometrically), and its eigenvector is positive.

(c) If A is irreducible, there is a positive vector v such that Av = ρ(A)v, andρ(A) is a simple eigenvalue of A.

Proof: It follows from Section 2.1 that, if x ≥ 0, ‖x‖1 = 1T x and that, ifA ≥ 0, ‖A‖∞ = ‖AT1‖∞.

(a) Let λ ∈ σ(A) have the maximum modulus and v be its correspondingeigenvector with ‖v‖1 =

∑

i |vi| = 1. Then, we have

ρ(A)|v| = |λv| = |Av| ≤ A|v|. (4.2)

We need to find λ∗ (among those λ’s) such that λ∗ = ρ(A) and v∗ ≥ 0.To this end, define

K= x ∈ ℜn : x ≥ 0,

n∑

i=1

xi = 1, Ax ≥ ρ(A)x.

It is apparent that K is compact and convex. Also, (4.2) shows |v| ∈ Kand hence K is not empty. Two distinct cases are analyzed.The first case is that there exists x ∈ K such that Ax = 0. It follows from0 = Ax ≥ ρ(A)x that ρ(A) = 0, and Statement (a) is established in thiscase.In the second case, Ax = 0 for all x ∈ K. Define the mapping

f(x) =1

‖Ax‖1Ax, x ∈ K.

It is apparent that f(x) is continuous, f(x) ≥ 0, ‖f(x)‖1 = 1, and

Af(x) =1

‖Ax‖1AAx ≥ 1

‖Ax‖1Aρ(A)x = ρ(A)f(x).

These properties show that f(x) is a continuous mapping from K to K.Hence, by Theorem 4.7, there exists v∗ ∈ K such that f(v∗) = v∗, thatis, with λ∗ = ‖Av∗‖1,

λ∗v∗ = Av∗ ≥ ρ(A)v∗,

which implies v∗ ≥ ρ(A) and in turn v∗ = ρ(A).


(b) Since A > 0, it follows from (a) and from Av = ρ(A)v that v > 0.Next, let us prove that ρ(A) has algebraic multiplicity 1, that is, eigen-vector v is the only eigenvector associated with ρ(A). Suppose that therewere another, v′ which is linearly independent of v. Since v > 0, it is pos-sible to find a linear combination w = cv + v′ such that w ≥ 0 but w > 0.However, since Aw = ρ(A)w is strictly positive, there is a contradiction.Therefore, algebraical multiplicity of ρ(A) is 1.Geometrical multiplicity of ρ(A) being 1 is also shown by contradiction.Suppose there were a Jordan chain of length at least two associated withρ(A). That is, there is a vector z such that [A − ρ(A)I]z = v whileAv = ρ(A)v. Now let w be the positive eigenvector of AT correspondingto ρ(A). It follows that

0 = wT [A − ρ(A)I]z = ρ(A)wT v > 0,

which is contradictory.(c) According to Corollary 4.4, (αI + A)k > 0 for some k > 0 and any

constant α > 0. Since eigenvalues and eigenvectors of A can be computedfrom those of (αI+A)k > 0, we know that ρ(A) is simple and v is positive.

Example 4.9. Consider the two matrices

A1 =

⎡

⎣

0 1 00 0 11 0 0

⎤

⎦ , A2 =

⎡

⎣

1 0 01 1 00 1 1

⎤

⎦ .

Matrix A1 has eigenvalues 1 and −0.5000 ± 0.8660j, and ρ(A1) = 1 (witheigenvector 1) is unique since A1 is irreducible. Matrix A2 has identical eigen-values of 1, and ρ(A2) = 1 is not unique since A2 is reducible.

In general, a non-negative matrix A can have complex eigenvalues, andsome of its eigenvalues (other than ρ(A)) may be in the left half plan. Sym-metric (non-negative) matrices have only real eigenvalues. While lower tri-angular matrices are only triangular in blocks and may not be strictly tri-angular, eigenvalues of strictly triangular non-negative matrices are all real,non-negative, and equal to those elements on the diagonal.

Matrix Jn×n has special properties. First, it is symmetric and hence di-agonalizable. Second, J = 11T is of rank 1, and J1 = n1. Third, there arecolumn vectors si for i = 1, · · · , n− 1 such that si are orthonormal (sT

i si = 1and sT

i sj = 0 for i = j) and that 1T si = 0, i.e., Jsi = 0. Fourth, choosing Tto be the unitary matrix with columns si and with 1/

√n as its last column,

we haveT−1JT = T TJT = T T1(T T1)T = diag0, · · · , 0, n.

Hence, all the eigenvalues of J are zero except for the unique eigenvalue ofρ(J) = n.


4.1.3 Cyclic and Primitive Matrices

Statement (c) of Perron-Frobenius theorem also has a non-trivial second part:if A is irreducible and if p ≥ 1 eigenvalues of A are of modulus ρ(A), then theseeigenvalues are all distinct roots of polynomial equation λp − [ρ(A)]p = 0, andthere exists a permutation matrix P such that

PAPT =

⎡

⎢

⎢

⎢

⎢

⎢

⎣

0 E12 0 · · · 00 0 E23 · · · 0...

......

. . ....

0 0 0 · · · E(p−1)p

Ep1 0 0 · · · 0

⎤

⎥

⎥

⎥

⎥

⎥

⎦

, (4.3)

where the zero blocks on the diagonal are square. In (4.3), index p is calledthe cyclicity index of A, or equivalently, matrix A is called p-cyclic. The abovestatement has the following partial inverse: Given matrix E in (4.3), matrixE (and hence A) is irreducible if the product of E12E23 · · ·E(p−1)pEp1 is irre-ducible.

A non-negative matrix E is said to be primitive if there exists a positiveinteger k such that Ek > 0. The following lemma shows that a primitivematrix has (up to scalar multiples) only one non-negative eigenvector.

Lemma 4.10. Let E be a primitive matrix with a non-negative eigenvector vassociated with eigenvalue λ. Then, v > 0 is unique and λ = ρ(E).

Proof: Since E is primitive, we have k > 0 such that Ek > 0 and thusEkv > 0. It follows from Ekv = λkv that v > 0 and λ > 0. Consider eigenvalueρ(E) of E and its eigenvector w. It follows from Ekw = ρk(E)w and fromPerron theorem, (b) in Theorem 4.8, that w > 0 and hence can be scaled bya positive constant such that 0 < w < v. Therefore, we have that, for all k,

0 < ρk(E)w = Ekw ≤ Ekv = λkv,

which is impossible if λ < ρ(E). Thus, λ = ρ(E).

Corollary 4.4 suggests there is a relationship between E being irreducibleand primitive. Indeed, matrix A is irreducible and 1-cyclic if and only if A isprimitive, which is applied in the following example.

Example 4.11. Consider the following matrices and their eigenvalues:

A1 =

[

0 21 1

]

, λ(A1) = 2,−1;

A2 =

[

0 41 0

]

, λ(A2) = 2,−2;

A3 =

[

1 10 1

]

, λ(A3) = 1, 1.

Among the three matrices, only A1 is primitive.


In addition to the above necessary and sufficient condition for a matrix tobe primitive, a sufficient condition is that, if A is irreducible and diagonallypositive, it is also primitive. This is due to the fact that A being irreducibleimplies (ǫI +A)n−1 > 0 for any ǫ > 0 and, if A is diagonally positive, ǫ can beset to be zero. The following example shows that matrix A being irreducibledoes not necessarily imply Ak > 0.

Example 4.12. Consider irreducible matrix

A =

⎡

⎣

0 0 11 0 00 1 0

⎤

⎦ .

It follows that

A2 =

⎡

⎣

0 1 00 0 11 0 0

⎤

⎦

is also irreducible, but A3 = I is reducible. Matrix A is 3-cyclic, and its poweris never positive.

In general, for an irreducible and p-cyclic matrix A, Ap is block diagonal.If p > 1, A is not primitive, and the whole spectrum σ(A) contains points ofmagnitude ρ(A) and angles θ = 2π/p apart. For instance, matrix

A =

⎡

⎢

⎢

⎣

0 0 2 00 0 0 10 1 0 02 0 0 0

⎤

⎥

⎥

⎦

is irreducible, is 4-cyclic, and has the spectrum

σ(A) = √

2, −√

2,√

2j, −√

2j.

If matrix A is either diagonal or in Jordan form, it is reducible and notprimitive. In this case, ρ(A) may not be a simple eigenvalue, and the corre-sponding Perron eigenvector may not be strictly positive.

As an interesting application of Theorem 4.8, we can study the limit ofpowers of a primitive matrix. Consider the matrix

E =

[

1 32 2

]

,

which has left eigenvector w = [2 3] and right eigenvector v = [1 1]T associatedwith spectral radius ρ(E) = 4. The other eigenvalue is λ = −1. It follows that

1

wvvw =

1

5

[

2 32 3

]

,


which is of rank 1. It is not difficult to verify that

En = 4n

[

25

35

25

35

]

+ (−1)n

[

35 − 3

5− 2

525

]

,

and that

limn→∞

En

ρn(E)=

1

wvvw. (4.4)

The general result illustrated by the example is given below.

Corollary 4.13. Let E be a primitive matrix with left eigenvector w and righteigenvector v associated with eigenvalue ρ(E). Then, the limit in (4.4) holds,and the convergence is exponential.

Proof: Since E is primitive, ρ(E) is a simple eigenvalue, and any other eigen-value λ has the property that |λ| < ρ(E). Letting A = vw/(wv), we canestablish (4.4) by showing the following four equations:

Av = v =Ev

ρ(E),

wA = w =wE

ρ(E),

Aβr = 0 = limn→∞

Enβr

ρn(E),

βlA = 0 = limn→∞

βlEn

ρn(E),

where βl is any of left eigenvectors of E but not associated with ρ(E), and βr

is any of right eigenvectors of E and associated with λ = ρ(E). The first twoare obvious. To show the third equation, we know that

ρn(E)Aβr =1

wvv[ρn(E)w]βr =

1

wvvwEnβr =

1

wvvwλnβr = λnAβr,

and that1

ρn(E)Enβr =

λn

ρn(E)βr,

which implies that

Aβr = limn→∞

λn

ρn(E)Aβr = 0 = lim

n→∞1

ρn(E)Enβr.

The fourth equation can be established similarly.

4.2 Importance of Non-negative Matrices 161

4.2 Importance of Non-negative Matrices

We begin with the following definitions:

Definition 4.14. Matrices whose off-diagonal elements are non-negative aresaid to be Metzler matrices.

Definition 4.15. M1(c0) denotes the set of matrices with real entries suchthat the sum of the entries in each row is equal to the real number c0. LetM2(c0) be the set of Metzler matrices in M1(c0). A matrix D is said to berow-stochastic if D ≥ 0 and D ∈ M1(1).

A Metzler matrix A can be expressed as

A = −sI + E, s ≥ 0, E ≥ 0. (4.5)

Therefore, properties of Metzler matrix A are analogous to those of non-negative matrix E. It also follows that M2(c0) are those matrices of A suchthat αI + βA is row-stochastic for some positive numbers α and β. Thus,properties of those matrices in M2(c0) can be derived from properties of row-stochastic matrices. The following lemma restates parts of Perron-Frobeniustheorem, Theorem 4.8.

Lemma 4.16. Consider two matrices A1 ∈ M1(c0) and A2 ∈ M2(c0). Then,

(a) c0 is an eigenvalue of A1 with eigenvector 1.(b) Real parts of all the eigenvalues of A2 are less than or equal to c0, and

those eigenvalues with real part equal to c0 are real.(c) If A2 is also irreducible, then c0 is a simple eigenvalue of A2.

As an application of non-negative row-stochastic matrix, consider a homo-geneous discrete-time Markov chain Xk described by its transition proba-bility matrix P = [pij ], where

pij = PrXk = j|Xk−1 = i

is the probability of a transition from state i to state j at any time step. Ifthe number of possible states is finite, stochastic matrix P is square and finitedimensional, and P1 = 1. On the other hand, a row vector

p =[

p1 p2 · · · pn

]

is called a probability distribution vector if p ≥ 0 and p1 = 1. Let p0 be theinitial probability distribution. Then, the probability distribution at step k is

pk = pk−1P = p0P k, k = 1, 2, · · ·

If the Markov chain is ergodic, there exists a steady state probability distri-bution independent of p0, that is,


p∞ = p∞P,

which says that p∞ is the left eigenvector of P corresponding to the eigenvalue1. In matrix terms, ergodicity of the homogeneous discrete-time Markov chainmeans that P is irreducible and primitive (and hence λ = 1 is a simple eigen-value, it is the only eigenvalue on the unit circle, and p∞ > 0). Applicationto non-homogeneous discrete-time Markov chain will be mentioned in Section4.4.

In systems and control, non-negative matrices arise naturally from theclass of linear positive systems defined below.

Definition 4.17. A linear system with triplet A, B, C is said to be positiveif and only if, for every non-negative initial condition and for every non-negative input, its state and output are non-negative.

Lemma 4.18. A continuous-time linear system

x = A(t)x + B(t)u, y = C(t)x

is positive if and only if A is a Metzler matrix, B ≥ 0, and C ≥ 0. A discrete-time linear system of triplet A, B, C is positive if and only if A ≥ 0, B ≥ 0and C ≥ 0.

Proof: Consider first the continuous-time system, and necessary is proven bycontradiction. Assume that matrix A contain aij < 0 for some i = j. If so,under zero input and under the initial conditions of xk(t0) = 0 for all k exceptthat xj(t0) > 0, differential equation of xi(t) becomes

xi = aij(t)xj , xi(t0) = 0.

according to which xi(t0 + δt) will leave set Rn+. This result contradicts with

Definition 4.17 and hence aij ≥ 0 for all i = j, i.e., matrix A is Metzler.Similarly, we can show that B ≥ 0 and C ≥ 0 are also necessary. Sufficiencycan be established by noting that, given Metzler matrix A and non-negativematrices B and C, x ≥ 0 along the boundary of Rn

+. The result on discrete-time system can be shown in a similar fashion.

Among the class of positive systems, there are cooperative systems de-scribed by

x = k2[−k1I + D(t)]x, (4.6)

where k1, k2 > 0 are scalar gains, and D(t) is a non-negative and row-stochastic matrix. Cooperative systems will be studied in depth in Chapter5. It is worth noting that, although matrix E in (4.5) is not necessarily row-stochastic, a linear time-invariant (or piecewise-constant) positive system canalways be transformed into the cooperative system in (4.6) for the purpose


of asymptotic stability analysis. To validate this point, consider any positivesystem z = [−sI + E]z with E ≥ 0. If E is irreducible, it follows from Theo-rem 4.8 that, for some vector v > 0, Ev = ρ(E)v where ρ(E) > 0. Letting Tbe the diagonal matrix such that T1 = v, we can introduce the transformedstate x = T−1z and have

x = [−sI+T−1ET ]x = ρ(E)

[

− s

ρ(E)I +

1

ρ(E)T−1ET

]

x= ρ(E)[−k1I+D]x,

in which matrix D is non-negative and row-stochastic since T−1ET1 =T−1Ev = ρ(E)T−1v = ρ(E)1. If E is reducible, the same process can beapplied. For instance, let us say

E =

[

E11 0E21 E22

]

where E11 and E22 are irreducible. As before, we can find the diagonal statetransformation z = T−1x with T = diagT1, T2, Ti1 = vi, and Eiivi =ρ(Eii)vi such that the resulting system becomes

x =

[

−sI + ρ(E11)D11 0D21 −sI + ρ(E22)D22

]

x,

where both D11 and D22 are row-stochastic. Asymptotic stability of the abovesystem is the same as that of system

x =

[

−sI + ρ(E11)D11 00 −sI + ρ(E22)D22

]

x,

Hence, we can use (4.6) as the canonical form to study asymptotic stabilityof positive systems. The following is such a result, and it comes directly fromLemma 4.16.

Corollary 4.19. Consider System 4.6 with k1, k2 > 0 and with D being non-negative, row-stochastic, and constant. Then,

(a) The system is asymptotically stable if k1 > 1.(b) The system is unstable if k1 < 1.(c) The system is Lyapunov stable if k1 = 1 and if D is irreducible.

The above stability result, in particular, Statement (c) will be extendedfurther in Section 4.3 to the case that matrix D is reducible. Similar resultscan be stated for discrete-time positive system xk+1 = Axk with non-negativematrix A; that is, the system is unstable if ρ(A) > 1, it is Lyapunov stable ifρ(A) = 1 and if A is irreducible, and it is asymptotically stable if ρ(A) < 1.

A non-negative matrix associated with a positive system has explicit phys-ical meanings. For illustration, let us use the cooperative system in (4.6) asan example and view each of its state variables as a separate entity. Then,


dynamics of each entity consists of negative feedback about itself and positivefeedback from others. The individual negative feedback is necessary to main-tain stability for continuous-time systems, while positive feedback from otherentities represent information exchanges and their impacts.

Irreducibility or reducibility is the structural property of a non-negativematrix, whose physical meaning is dynamical grouping or coupling amongthe entities. Should matrix D be irreducible, the value of (or the informationabout) any of the entities can propagate to any other entity through thesystem dynamics. We can see this fact from the following state solution toSystem 4.6 with constant matrix D:

x(t) = e−(k1+1)k2tek2(I+D)tx(0) = e−(k1+1)k2t∞∑

j=0

kj2t

j

j!(I + D)jx(0),

in which (I + D)j ≥ 0 for all j. Since (I + D)n−1 > 0 for any irreduciblematrix D, x(t) > 0 for all finite time t > 0 under any initial condition on theboundary of ℜn

+ but not at the origin. Hence, we know that all the entitiescorresponding to an irreducible matrix move together as one group.

On the other hand, if System 4.6 has a reducible matrix D, we can findits triangular canonical form shown in (4.1). Its permutation matrix tells ushow the state variables can be separated into the groups corresponding todiagonal irreducible blocks of D\. If matrix D is lower triangularly complete,then the group of the entities corresponding to the first diagonal block ofD\ acts as the unique leader group by sending its information to the restof entities but receiving nothing back, and all the other groups of entitiesact as followers by receiving information from the leader. Thus, we knowthat the groups of entities corresponding to a reducible but lower triangularlycomplete matrix also move together as one larger composite group. If matrixD is not lower triangularly complete, matrix D\ contains two or more isolateddiagonal blocks and hence there are at least two groups of entities that moveindependently from each other and from the rest of entities.

For discrete-time system xk+1 = Dxk, the same observation holds exceptfor the differences that negative feedback about an entity itself is no longerneeded for asymptotic stability and that Dk > 0 only if D is primitive. IfD is cyclic (irreducible but not primitive), there are two or more groups ofentities among which information is transmitted in such a way that they acteach other in cycles. In the same fashion as that of continuous-time systems,grouping of discrete entities depends upon whether D\ is lower triangularlycomplete if D is reducible.

It is because of the above physical implications that non-negative matricesand their irreducibility or lower triangular completeness play a critical role inanalysis and synthesis of cooperative systems. Discussions in the next twosubsections reinforce these implications on dynamical systems.


4.2.1 Geometrical Representation of Non-negative Matrices

Let K ⊂ ℜn be a set. Then, set K is said to be convex if K contains thelinear segment between any two points therein, and set K is said to be a coneif K contains all finite non-negative linear combinations of its elements. Thesimplest convex cones are K = ℜn and K = ℜn

+, and any square matrix Acan be viewed as a mapping of A : ℜn → ℜn. It follows that A ∈ ℜn×n

+ if andonly if A(ℜn

+) ⊂ ℜn+.

Convex cone K is said to be solid if the interior of K, denoted by int K,is not empty; K is pointed if K ∩ (−K) = 0; and K is called a proper coneif it is solid, pointed, and closed. Among proper cones is ℜn

+. Given a propercone K, F is said to be a face of K if F ⊂ K is closed and pointed and if

x ∈ F, x − y ∈ K =⇒ y ∈ F.

The face F is non-trivial if F = 0 and F = K. As an example, the faces ofRn

+ are of the form

FI = x ∈ Rn+ : xi = 0 if i ∈ I,

where I ⊂ 1, · · · , n.Non-negativeness, positiveness, and irreducibility can be defined equiva-

lently but geometrically as follows: For any given proper cone K,

(a) Matrix A ∈ ℜn×m is said to be K-non-negative if it leaves K invariant,i.e., A(K) ⊂ K.

(b) Matrix A is said to be K-positive if it maps non-zero elements of K intothe interior, i.e., A(K − 0) ⊂ int K.

(c) Matrix A is said to be K-irreducible if and only if faces of K that it leavesinvariant are 0 and K itself. Matrix A is said to be K-reducible if itleaves invariant a non-trivial face of K.

Geometrically, the Perron-Frobenius theorem tells us the following:

(a) If A is K-non-negative, ρ(A) is an eigenvalue, and K contains an eigen-vector of A corresponding to ρ(A).

(b) Matrix A is K-irreducible if and only if no eigenvector of A lies on theboundary of K. Matrix A is K-irreducible if and only if A has exactly one(up to scale multiples) eigenvector in K, and this vector is in its interiorint K.

(c) Matrix A is K-irreducible if and only if (I + A)n−1 is K-positive.

Letting K = ℜn×n+ renders geometrical meanings about the results in Section

4.1.


(a) (b)

Fig. 4.1. Examples of graphs

4.2.2 Graphical Representation of Non-negative Matrices

Given any matrix A ∈ ℜn×n, we can associate it with a directed graph or

digraph G(A) = (I, E) with vertex set (or nodes) I = 1, 2, · · · , n and their

arc set (or edges) E ⊂ I×I, where (i, j) ∈ E if and only if aij = 0. Conversely,given a directed graph G = (I, E), there is an adjacency matrix A whoseelements are binary and defined by

aij =

1, if (i, j) ∈ E0, else

.

The out-degree (or in-degree) of a node is the number of arcs originating (orterminating) at the node. A node is balanced if its in-degree and out-degree areequal, and a digraph is balanced if all of its nodes are balanced, or equivalentlyin terms of adjacency matrix A, AT1 = A1 = c1 for some c ∈ ℜ+. If matrixA is symmetrical and row-stochastic, it corresponds to an undirected graph(whose edges are all bi-directional), and hence the graph is balanced.

Example 4.20. Consider matrices:

A1 =

⎡

⎣

0 0 11 0 00 1 1

⎤

⎦ , and A2 =

⎡

⎣

0 1 00 0 11 0 0

⎤

⎦ .

Their corresponding graphs are shown in Fig. 4.1. Graph in Fig. 4.1(b) isbalanced, but A2 is not symmetrical and hence graph in Fig. 4.1(a) is notbalanced.

A path from node i to j is a sequence of successive edges (i, k1), · · · , (kl, j)in E that connects node j to node i. Directed graph G is strongly connectedif, for any pair of vertices i, j ∈ I, there exists a path in G that starts at i andterminates at j. Directed graph G has a global reachable node or a spanningtree if it has a node to which there exists a path from every other node. A

4.3 M-matrices and Their Properties 167

loop of length k in G is a path of length k which begins and ends at the samevertex.

The following are useful facts that are apparent from the above discussions:For any A ≥ 0,

(a) The (i, j)th element of matrix Ak is positive for some k > 0 if and only ifthere is a path from node i to node j.

(b) Matrix A is irreducible if and only if its graph G(A) is strongly connected.(c) Matrix A is lower triangularly complete in its canonical form if and only

if its graph G(A) has a global reachable node or a spanning tree.(d) The cyclicity (period) of matrix A is the greatest common divisor of the

lengths of all the loops in G(A).

As such, the matrix-theoretical results in the subsequent sections and chapterscan be stated analogously and equivalently using the terminology of the graph-theoretical approach [19, 51, 78].

4.3 M-matrices and Their Properties

We begin with the following definition.

Definition 4.21. Let Z denote the set of square matrices whose off-diagonalelements are non-positive, that is,

Z = A = [aij ] ∈ ℜn×n : aij ≤ 0, i = j.

Then, matrix A is called a non-singular (or singular) M -matrix if A ∈ Z andif all its principal minors are positive (or non-negative).

By definition, matrix A ∈ Z can also be expressed in the form of

A = sI − B, s > 0, B ≥ 0; (4.7)

that is, −A is a Metzler matrix. If A is a non-singular M-matrix, s > ρ(B)and (−A) is both a Metzler matrix and a Hurwitz matrix. If A is a singularM-matrix, s ≥ ρ(B). A symmetrical non-singular (or singular) M-matrix mustbe positive definite (or positive semi-definite).

4.3.1 Diagonal Dominance

The following Gerschgorin’s circle criterion, also referred to as Gerschgorindisc theorem [90], is typically used to estimate the locations of eigenvalues ofa matrix.


Theorem 4.22. All the eigenvalues of matrix A = [aij ] ∈ ℜn×n are locatedwithin the following union of n discs:

n⋃

i=1

⎧

⎨

⎩

z ∈ C : | z − aii | ≤∑

j =i

|aij |

⎫

⎬

⎭

.

Diagonal dominance of a matrix enables us to apply Gerschgorin’s circlecriterion and to conclude the eigenvalue properties in Corollary 4.24.

Definition 4.23. Matrix E = [eij ] ∈ ℜn×n is said to be diagonally dominantif, for all i,

eii ≥∑

j =i

|eij |.

Matrix E is said to be strictly diagonally dominant if the above inequalitiesare all strict.

Corollary 4.24. If matrix A ∈ ℜn×n is diagonally dominant (strictly diago-nally dominant), then eigenvalues of A have non-negative (positive) real parts.Additionally, if A ∈ Z, matrix A is a singular (non-singular) M-matrix.

4.3.2 Non-singular M-matrices

The following theorem summarizes several useful properties of non-singularM-matrices, and a comprehensive list of 50 properties can be found in [18](Theorem 2.3, p.134).

Theorem 4.25. Matrix A ∈ Z is a non-singular M-matrix under one of thefollowing conditions, and the conditions are all equivalent:

(a) The leading principal minor determinants of A are all positive.(b) The eigenvalues of A have positive real parts.(c) A−1 exists and is non-negative.(d) There exist vectors x, y > 0 such that both Ax and AT y are positive,

denoted by Ax, Ay > 0.(e) There exists a positive diagonal matrix S such that AS + SAT is strictly

diagonally dominant and hence also positive definite, that is,

aiisi >∑

j =i

|aij |sj , ∀ i. (4.8)

Proof: Equivalence between (a) and (b) is classical and well known.Since A ∈ Z, we can rewrite matrix A as

A = λI − B


where B is non-negative. It follows from the property of non-negative ma-trices that spectrum ρ(B) ≥ 0 is an eigenvalue of B and the correspondingeigenvector is v ≥ 0. It follows that Property (b) is equivalent to ρ(B) < λ.

Letting T = B/λ, it follows that A−1 = (I − T )−1/λ and that ρ(T ) < 1 ifand only if (b) holds. On the other hand, let us consider the infinite series

(I − T )−1 =

∞∑

k=1

T k, (4.9)

which exists and is non-negative provided that the series is convergent. Sincethe above series converges if and only if ρ(T ) < 1, the equivalent between (b)and (c) is established.

Now, choosing x = A−11, we know that Ax = 1 > 0 and also x > 0 sinceA−1 is non-negative, invertible and hence cannot have a zero row. That is, weknow that (c) implies (d). Letting S = diagx1, · · · , xn, we have

AS1 = Ax = 1 > 0,

which means strict diagonal dominance of AS. Hence, (d) implies (e). On theother hand, letting P = S−1, we know that

PA + AT P = P [AS + SAT ]P

is positive definite, which means that (e) implies (a).

Statement (a) implies that, if A ∈ Z and if A is symmetrical and positivedefinite, A must be a non-singular M-matrix. If A ∈ Z and if AT +A is positivedefinite, A is also a non-singular M-matrix since A + AT ∈ Z. However, if Ais a non-singular M-matrix, AT + A may not be positive definite. Instead,statement (e) tells us that, for any non-singular M-matrix, there exists apositive diagonal matrix P such that AP + AT P is positive definite.

In some literature, Inequality 4.8 is also referred to as strict pseudo-diagonal dominance, and it includes strict diagonal dominance as a specialcase. Since −(AS + SAT ) is a symmetric and Metzler matrix, P = S−1 is adiagonal Lyapunov matrix and, by Lyapunov direct method, Corollary 4.24yields statement (a) from (e) of Theorem 4.25. In comparison, Theorem 4.25establishes in matrix set Z the equivalence between strict pseudo-diagonaldominance and non-singular M-matrix.

Example 4.26. Matrix

A =

[

1 0−3 1

]

is a non-singular M-matrix. It follows that

A + AT =

[

2 −3−3 2

]


is not positive definite and that, under the choice of

S =

[

1 00 3

]

, AS + SAT =

[

2 −3−3 6

]

is positive definite.

4.3.3 Singular M-matrices

The following theorem summarizes several useful properties of singular M-matrices, i.e., matrices of form A = ρ(B)I − B with B ≥ 0.

Theorem 4.27. Matrix A ∈ Z is called a singular M-matrix under one ofthe following equivalent conditions:

(a) The leading principal minor determinants of A are all non-negative.(b) The eigenvalues of A have non-negative real parts.(c) (A+D)−1 exists and is non-negative for each positive diagonal matrix D.(d) There exist non-negative non-zero vectors x, y ≥ 0 such that both Ax and

AT y are non-negative, denoted by Ax, Ay ≥ 0.(e) There exists a non-negative diagonal matrix S such that S = 0 and

aiisi ≥∑

j =i

|aij |sj . (4.10)

Proof: It follows from

A + ǫI = (ρ(B) + ǫ)I − B

that A + ǫI is a non-singular M-matrix for all ǫ > 0 if and only if A is asingular M-matrix. Hence, statements (a), (b), (c), (d) and (e) are parallel tothose in Theorem 4.25, and they can be shown using Theorem 4.25 by takingthe limit of ǫ → 0.

If S is diagonal and positive definite, Inequality 4.10 is referred to aspseudo-diagonal dominance. Since S is only positive semi-definite, Statement(e) of Theorem 4.27 does not imply that SA + SAT is diagonally dominantor matrix −A is Lyapunov stable. To ensure these properties, an additionalcondition needs to be imposed on A in addition to A being a singular M-matrix. To this end, the following definition is introduced.

Definition 4.28. An M-matrix A is said to have “Property c” if A can besplit as A = ρ(B)I − B with B ≥ 0 and ρ(B) > maxi aii and if T = B/ρ(B)is semi-continuous in the sense that limj→∞ T j exists.

Note that, as shown in (4.9), matrix sum∑

j T j is convergent and

limj→∞ T j = 0 if and only if ρ(T ) < 1. Hence, all non-singular M-matriceshave “Property c,” but not all singular M-matrices. Also, note that the split


of A = ρ(B)I−B is not unique but should be done such that ρ(B) > maxi aii.For example, consider

A =

[

1 −1−1 1

]

.

It follows that, upon splitting A as A = λ(I − T ), limj→∞ T j exists for anyλ > 1 but does not exist for λ = 1.

The following theorem provides the characterization of singular M-matriceshaving “Property c.” Note that, in (c), Lyapunov matrix W is not necessarilydiagonal.

Theorem 4.29. For any matrix A ∈ Z, the following statements are equiva-lent to each other:

(a) A is a singular M-matrix with “Property c.”(b) Rank of Ak is invariant as rank rank(A) = rank(A2).(c) There exists a symmetric positive definite matrix W such that WA+AT W

is positive semi-definite; that is, matrix −A is Lyapunov stable.

Proof: Recall from Perron-Frobenius theorem that, for any singular M-matrixA, A = ρ(B)I − B with B ≥ 0. The choice of ρ(B) > maxi aii ensures thatT = B/ρ(B) is diagonally positive, hence the cyclicity index of T is 1 andρ(T ) = 1 is the unique eigenvalue of modulus 1. Suppose that S is the non-singular transformation matrix under which

S−1TS =

[

J 00 H

]

is the Jordan form for T , where J has 1 on its diagonal and ρ(H) < 1. Itfollows that

limj→∞

T j = S−1 limj→∞

[

Jj 00 Hj

]

S = S−1

[

limj→∞ Jj 00 0

]

S,

which exists if and only if J = I. On the other hand, J = I is true if and only ifrank(I −T ) = rank((I −T )2), and hence if and only if rank(A) = rank(A2).This shows that (a) and (b) are equivalent.

To show that (a) and (c) are equivalent, note that (a) is equivalent to

A = ρ(B)(I − T ) = ρ(B)S−1

[

0 00 I − H

]

S,

where ρ(H) < 1. Hence, the above is true if and only if there exists symmetricpositive definite matrix Ph such that Ph(I − H) + (I − H)T Ph is positivedefinite, thus if and only if there exists W = ST diagI, PhS such that

WA + AT W = ρ(B)ST

[

0 00 Ph(I − H) + (I − H)T Ph

]

S

is positive semi-definite.


The following lemma provides an alternative condition.

Lemma 4.30. If A ∈ Z and if there exists vector x > 0 such that Ax ≥ 0,then A is a singular M-matrix with “Property c.”

Proof: Let y = x/ρ(B). It follow that y > 0 and (I − T )y ≥ 0 or simplyy ≥ Ty. Recalling that T is non-negative, we have

y ≥ Ty ≥ T (Ty) = T 2y ≥ · · · ≥ T jy.

Since y > 0, the above inequality implies T j is finite for all j. Proof of Theorem4.25 also shows that A is a singular M-matrix with “Property c” if and only ifthe corresponding matrix T has the property that T j is finite for all j. Thus,the proof is done.

4.3.4 Irreducible M-matrices

Irreducibility or reducibility of M-matrix A = sI − B is defined in terms ofthat of non-negative matrix B. The following theorem shows that Lyapunovstability of −A is always guaranteed for a singular but irreducible M-matrixA.

Theorem 4.31. Let A ∈ Z be an nth-order irreducible singular M-matrix.Then,

(a) A has rank n − 1.(b) There exists a vector x > 0 such that Ax = 0.(c) Ax ≥ 0 implies Ax = 0.(d) Each principal sub-matrix of A of order less than n is a non-singular M-

matrix.(e) Matrix A has “Property c.”(f) There exists a positive diagonal matrix P such that PA+ AT P is positive

semi-definite; that is, matrix A is pseudo-diagonally dominant.

Proof: Let A = ρ(B)I−B with B ≥ 0 and ρ(B) > maxi aii. It follows that Bis irreducible and, by Perron-Frobenius theorem, ρ(B) is a simple eigenvalueof B. Then, matrix A has 0 as its simple eigenvalue and is of rank n − 1.

It follows from Perron-Frobenius theorem that there exists a vector x > 0such that Bx = ρ(B)x, that is, Ax = 0.

Suppose that Ax ≥ 0 for some x. It follows from (b) that there existsy > 0 such that AT y = 0. Thus, yT (Ax) = (AT y)T x = 0 for all x. If Ax = 0,yT (Ax) = 0, which is a contradiction.

It follows from (b) that there exist x, y > 0 such that Ax = 0 and AT y = 0.

Defining Aadj= cxyT for some c ∈ ℜ, we know from AAadj = AadjA =

det(A)I = 0 that Aadj is the adjoint matrix of A. It follows from (a) thatA is rank n − 1 and hence c = 0. If c < 0, there would exist an ǫ > 0 such


that the adjoint matrix of A + ǫI would be negative, which contradicts theproof of Theorem 4.27. Thus, c > 0 and hence Aadj > 0, from which all theprincipal minors of order n− 1 of A are positive. This in turn implies that allthe principal minors of order less than n are positive.

Since ρ(B) is a simple eigenvalue of B, 1 is a simple eigenvalue of T =B/ρ(B), and it follows from the proof of Theorem 4.29 that A has “Propertyc.”

Let x, y > 0 be the right and left eigenvector of B and associated withρ(B), that is, Ax = 0 and AT y = 0. Defining P = diagy1/x1, · · · , yn/xn,we have that PA + AT P ∈ Z and that

(PA + AT P )x = P (Ax) + AT (Px) = 0 + AT y = 0,

from which (PA + AT P ) being a singular M-matrix is concluded from (d) ofTheorem 4.27. Since (PA + AT P ) is symmetrical, it must be positive semi-definite.

Under irreducibility, there is a simpler transition from singular M-matrixto non-singular M-matrix as stated below.

Lemma 4.32. If A ∈ Z is a singular but irreducible M-matrix, then ma-trix [A + diag0, · · · , 0, ǫ] is a non-singular M-matrix, and matrix [−A +diag0, · · · , 0, ǫ] is an unstable matrix, where ǫ > 0.

Proof: Partition matrix A = [aij ] as follows:

A =

[

A11 A12

A21 ann

]

,

where A11 ∈ ℜ(n−1)×(n−1). Clearly, principal sub-matrices of matrix

(A + diag0, · · · , 0, s)up to (n − 1)th-order are identical to those of A. By definition, we have thefollowing property on determinant calculation:

det(A + diag0, · · · , 0, s) = det(A) + det

[

A11 A12

0 s

]

= det(A) + s det(A11).

Since A is a singular irreducible M-matrix, det(A) = 0 and, by (d) of Theorem4.31, det(A11) > 0. Thus, det(A + diag0, · · · , 0, s) is non-zero and has thesame sign as s, from which the conclusions are drawn.

It follows from (e) of Theorem 4.25 that, if A ∈ ℜn×n is a non-singular M-matrix, so is matrix (A+diag0, · · · , 0, ǫ) with ǫ > 0. The following corollariescan be concluded from Lemma 4.32 by mixing permutation with addition ofmatrix diag0, · · · , 0, ǫi and by invoking (f) of Theorem 4.31. In Corollary4.34, by requiring irreducibility, strict pseudo-diagonal dominance is relaxedfrom all the rows in (4.8) to just one of the rows.


Corollary 4.33. If A ∈ Z is a singular but irreducible M-matrix, then matrix[A + diagǫi] is a non-singular M-matrix, and matrix [−A + diagǫi] is anunstable matrix, where ǫi ≥ 0 for i = 1, · · · , n and at least one of them ispositive.

Corollary 4.34. Consider matrix A ∈ Z. If A is irreducible and if thereexists a positive diagonal matrix S such that

aiisi ≥∑

j =i

|aij |sj , ∀ i,

in which at least one of the inequalities is strict, then matrix A is a non-singular M-matrix.

4.3.5 Diagonal Lyapunov Matrix

For any non-singular M-matrix, Statement (e) of Theorem 4.25 provides aquadratic Lyapunov function whose special expression is a weighted sum ofsquares of all the state variables, and its proof is constructive for finding P .The following is an alternative way to find the diagonal Lyapunov matrix P .Let x, y > 0 be the vectors such that u = Ax > 0 and v = AT y > 0. DefiningP = diagy1/x1, · · · , yn/xn, we have that PA + AT P ∈ Z and that

(PA + AT P )x = P (Ax) + AT (Px) = Pu + AT y = Pu + v > 0,

from which (PA + AT P ) is a non-singular M-matrix according to (d) of The-orem 4.25. Since (PA + AT P ) is symmetrical, it must be positive definite.Should matrix A = λI −B also be irreducible, the above vectors u and v usedin finding diagonal Lyapunov matrix P can simply be chosen to the right andleft eigenvectors of B and associated with ρ(B) > 0 respectively, in whichcase u, v > 0, Au = (λ − ρ(B))u and vT A = (λ − ρ(B))vT . In addition, itfollows from (4.9) that A is a non-singular irreducible M-matrix if and only ifA−1 > 0.

For a singular but irreducible matrix, Statement (f) of Theorem 4.31 pro-vides a quadratic Lyapunov function also in terms of a diagonal Lyapunov ma-trix, and its proof is also constructive. For a singular and reducible M-matrixA, Statement (e) of Theorem 4.27 does not provide a Lyapunov function tostudy Lyapunov stability of −A. If singular M-matrix A has “Property c,”Lyapunov stability of −A is shown in Theorem 4.29, but the correspondingLyapunov matrix is not necessarily diagonal. In general, a Lyapunov matrixdoes not exist without “Property c” since matrix −A is not Lyapunov stable.The following example illustrates these points.

Example 4.35. Consider the following reducible and singular M matrices:

A1 =

[

ǫ 0−ǫ 0

]

, A2 =

[

0 0−ǫ ǫ

]

, A3 =

[

0 0−ǫ 0

]

,


where ǫ > 0. Apparently, for any two-dimensional vector x > 0, neither canA1x ≥ 0 or AT

2 x ≥ 0 or A3x ≥ 0 hold. Both matrices −A1 and −A2 are Lya-punov stable and also have “Property c”, hence there are Lyapunov matricesPi such that PiAi + AT

i Pi is positive semi-definite for i = 1, 2, but Pi are notdiagonal. For instance, for matrix A2, we can choose positive definite matrix

P2 =1

ǫ

[

2 −1−1 1

]

and obtain

P2A2 + AT2 P2 =

[

2 −2−2 2

]

,

which is positive semi-definite. Nonetheless, there is no diagonal Lyapunovfunction P for matrix A2 since

P =

[

p1 00 p2

]

=⇒ PA2 + AT2 P =

[

0 −ǫp2

−ǫp2 2ǫp2

]

.

Matrix A3 is unstable and thus Lyapunov matrix P3 does not exist suchthat P3 is positive definite and P3A3 +AT

3 P3 is positive semi-definite. Indeed,matrix A3 does not have “Property c.”

The above results show that, unless M-matrix A is both reducible andsingular, its quadratic Lyapunov function exists and the corresponding Lya-punov matrix P can simply be chosen to be diagonal. These results can beapplied directly to continuous-time positive system x = −Ax, where −A isMetzler. If A is both reducible and singular, one should proceed with analysiseither by checking “Property c” or by first finding its canonical form of (4.1)and then utilizing properties of irreducible diagonal blocks.

For discrete-time positive system xk+1 = Axk, the following theorem pro-vides a similar result on the existence of diagonal Lyapunov matrix. Onceagain, if A is reducible, one can proceed with analysis by employing the canon-ical form of (4.1).

Theorem 4.36. If A is a non-negative and irreducible matrix and if ρ(A) <1 (or ρ(A) = 1), then there exists a positive diagonal matrix P such thatP − AT PA is positive definite (or positive semi-definite).

Proof: It follows from Theorem 4.8 that there exists positive vectors u, v > 0and eigenvalue ρ(A) > 0 such that

Au = ρ(A)u, AT v = ρ(A)v.

Letting P = diagvi/ui, we know that off-diagonal elements of matrix (P −AT PA) are all non-positive and that

(P − AT PA)u = Pu − ρ(A)AT Pu = v − ρ(A)AT v = (1 − ρ2(A))v ≥ 0,


from which (P − AT PA) being a non-singular or singular M-matrix can beconcluded from Theorems 4.25 or 4.27. Since (P −AT PA) is also symmetrical,it must be positive definite or positive semi-definite.

4.3.6 A Class of Interconnected Systems

Consider the following collection of sub-systems: for i = 1, · · · , q,

xi = Axi + B(u + Diy), yi = Cxi, (4.11)

where xi ∈ ℜn, u ∈ ℜ, yi ∈ ℜ,

x =[

xT1 · · · xT

q

]T, y =

[

y1 · · · yq

]T, B =

[

0 1]T

,

C =[

1 0]

, D =[

DT1 · · · DT

q

]T,

and matrices A, B, C and D are of proper dimensions. Individual dynamicsof each sub-system are identical, and dynamic interaction among the sub-systems are through matrix D. But, different from a typical interconnectedsystem, the above system has three distinct properties:

(a) Matrix D ∈ M2(0) is a Metzler matrix with D1q = 0, and it is alsoirreducible and symmetric. It follows from Theorem 4.31 that eigenvaluesof D are λ1 ≤ · · · ≤ λq−1 < λq = 0.

(b) All the sub-systems have the same control input.(c) In designing the only control, the available feedback is limited to the mean

of the sub-system states. That is, control u has to be of form

u = −1

qK

q∑

j=1

xj , (4.12)

where K ∈ ℜ1×n is a feedback gain matrix.

The following lemma provides the corresponding stability condition.

Lemma 4.37. Consider System 4.11 with an irreducible and symmetric ma-trix D in set M2(0) and under Control 4.12. Then, the system is asymptoti-cally stable if and only if matrices (A−BK) and (A+ λiBC) are all Hurwitzfor i = 1, · · · , q − 1, where λi are negative real eigenvalues of D.

Proof: It follows that

qBu = −BK[

In×n · · · In×n

]

x

= −(1Tq ⊗ (BK))x

and that

4.4 Multiplicative Sequence of Row-stochastic Matrices 177

BDiy = (Di ⊗ B)diagC, · · · , Cx= (Di ⊗ (BC))x.

Hence, dynamics of the overall system can be written as

x = (Iq×q ⊗ A)x − 1

q(Jq×q ⊗ (BK))x + (D ⊗ (BC))x

= Ax.

Based on Property (a), we know there exists a unitary matrix T with 1/√

qas its last column such that

T−1DT = T T DT = diagλ1, · · · , λq.

It has been shown in Example 4.9 that

T−1Jq×qT = diag0, · · · , 0, q.

Thus, we have

(T ⊗ In×n)−1A(T ⊗ In×n)

= (Iq×q ⊗ A) − (diag0, · · · , 0, 1 ⊗ (BK)) + (diagλ1, · · · , λq−1, 0 ⊗ (BC))

= diag(A + λ1BC), · · · , (A + λq−1BC), (A − BK),

from which the condition on asymptotic stability can be concluded.

Lemma 4.37 proves to be very useful in friction control of nano-particles[82, 83]. It is worth noting that, by defining the average state xa = (

∑qj=1 xj)/q

and by recalling 1Tq D = 1T

q DT = (D1q)T = 0 , dynamics of the average state

are described by

xa =1

q(1T

q ⊗ In×n)x = Axa − BKxa +1

q((1T

q D) ⊗ (BC))x = (A − BK)xa,

which can always be stabilized as long as pair A, B is controllable. Sinceall the sub-systems have the same average-feedback control, the average stateis expected to be stabilized. Whether the first q − 1 states are all stabilizeddepends upon property of (A + λjBC) for j = 1, · · · , q − 1. Note that, if thefictitious system of z = Az + Bv and y = Cz is stabilizable under any staticoutput feedback control, matrices (A + λjBC) are Hurwitz.

4.4 Multiplicative Sequence of Row-stochastic Matrices

Consider an infinite countable series of non-negative row-stochastic matricesPk : Pk ∈ ℜn×n

+ , Pk1 = 1, k ∈ ℵ+, where ℵ+ is the set of positive integers.In the subsequent discussion, let us denote that, for any k > l,

Pk:l= PkPk−1 · · ·Pl+2Pl+1.


The objective of this section is to determine the conditions (including topo-logical or structural conditions) under which the infinitely pre-multiplicativesequence of P∞:0 has the following limit: for some vector c ∈ ℜn,

P∞:0= lim

k→∞

k∏

j=1

Pj= lim

k→∞PkPk−1 · · ·P2P1 = 1cT . (4.13)

It will be shown in Chapter 5 that the convergence corresponds to the so-calledasymptotic cooperative stability.

It is beneficial to begin convergence analysis with four simple facts. First,Limit 4.13 implies cT1 = 1 since Pj1 = 1. Second, Limit 4.13 does not meannor is it implied by irreducibility. For instance, it is shown in Example 4.12that the infinite sequence of P k does not have Limit 4.13 for any irreduciblecyclic matrix P . Third, Limit 4.13 is different from Limit 2.35. Nonetheless,the general implications derived from the discussions in Section 2.4.2 do ap-ply here. As a sufficient condition, if Ak = αiPi where 0 ≤ αi ≤ 1 − ǫ forconstant ǫ > 0 and for all i, Limit 2.35 can be concluded using Limit 4.13.Fourth, for a non-homogeneous discrete-time Markov chain Xk with tran-sition probability matrix Pk, its ergodicity is characterized by the followinglimit of post-multiplicative sequence:

limk→∞

P1P2 · · ·Pk−1Pk= lim

k→∞⊓k

j=1Pj = 1cT .

Although the above sequence is different from P∞:0, both sequences can beanalyzed using the same tools and their convergence conditions are essentiallythe same.

4.4.1 Convergence of Power Sequence

A special case of Limit 4.13 is that, for some c ∈ ℜn,

limk→∞

P k = 1cT . (4.14)

The following theorem provides a necessary and sufficient condition, its proofis based on the next lemma (which is a restatement of Corollary 4.33 and canalso be proven by directly showing that row sums of Eri

ii are less than 1).

Lemma 4.38. Suppose that non-negative row-stochastic matrix E\ is in thelower triangular canonical form of (4.1), where Eii ∈ ℜri×ri are irreducibleblocks on the diagonal. If Eij = 0 for some i > 1 and for some j < i, then(I − Eii) is a non-singular M-matrix, or simply ρ(Eii) < 1.

Theorem 4.39. Let E\ in (4.1) be the lower triangular canonical form ofrow-stochastic matrix P . Then, Limit 4.14 exists if and only if E\ is lowertriangularly complete and its E11 is primitive.


Proof: Assume without loss of any generality that, in (4.1), p = 2 (if other-wise, the proof can be extended by induction). It follows that

Ek\ =

[

Ek11 0

Wk Ek22

]

,

where Wk = 0 if E21 = 0. Indeed, it follows from Lemma 4.38 that, if E21 = 0,Ek

22 → 0 and hence Wk1 → 1. It follows from Corollary 4.13 and Example4.12 that Ek

11 → 1cT1 if and only if E11 is primitive. Hence, combining the two

conditions yields

E2k\

=

[

E2k11 0

WkEk11 E2k

22

]

→[

1cT1 0

1cT1 0

]

, E2k+1\

→ E\

[

1cT1 0

1cT1 0

]

=

[

1cT1 0

1cT1 0

]

,

which completes the proof.

In fact, Lemma 4.38 and Corollary 4.13 also imply that, if Limit 4.14 exists,convergence is exponential. In [63], lower triangularly complete matrices arealso said to be indecomposable. If E11 is primitive (while other Eii may notbe primitive), it has cyclicity 1 and hence is aperiodic. Therefore, a row-stochastic matrix having Limit 4.14 or satisfying Theorem 4.39 is also referredto as stochastic-indecomposable-aperiodic or SIA matrix [274] or regular matrix[85].

Lower triangularly complete and diagonally positive matrices are alwaysSIA, and it will be shown in Subsection 4.4.4 that they are instrumental toensuring Limit 4.13 arising from control systems. In general, matrices Pj beingSIA do not guarantee Limit 4.13 as shown by the following example.

Example 4.40. Matrices

D1 =

⎡

⎣

0 1 00 0 1

0.4 0 0.6

⎤

⎦ , D2 =

⎡

⎣

0.6 0 0.41 0 00 1 0

⎤

⎦

are both irreducible and primitive, hence they are SIA. In fact, D2 is a per-muted version of D1. However, it follows that

D1D2 =

⎡

⎣

1 0 00 1 0

0.24 0.6 0.16

⎤

⎦ ,

which is reducible, not lower triangularly complete, and hence not SIA. Inother words, if sequence P∞:0 has Limit 4.13, simply inserting permutationmatrices into the convergent sequence may make the limit void.

4.4.2 Convergence Measures

Existence of Limit 4.13 can be studied using the idea of contraction mappingdiscussed in Section 2.2.1. To this end, let us define the following two measures:given a squarely or rectangularly row-stochastic matrix E ∈ ℜr1×r2

+ ,


δ(E) = max1≤j≤r2

max1≤i1,i2≤r1

|ei1j − ei2j |,

λ(E) = 1 − min1≤i1,i2≤r1

r2∑

j=1

min(ei1j , ei2j).(4.15)

It is obvious that 0 ≤ δ(E), λ(E) ≤ 1 and that λ(E) = 0 if and only ifδ(E) = 0. Both quantities measure how different the rows of E are: δ(E) = 0if all the rows of E are identical, and λ(E) < 1 implies that, for every pair ofrows i1 and i2, there exists a column j (which may depend on i1 and i2) suchthat both ei1j and ei2j are positive.

For matrix E ∈ ℜn×n+ with row vectors Ei, let us define vector minimum

function minETi , ET

j ∈ ℜn+ and their convex hull co(E) ⊂ ℜn

+ as

minETi , ET

j =[

minei1, ej1, · · · , minein, ejn]T

,

and

co(E)=

⎧

⎨

⎩

n∑

j=1

ajETj : aj ≥ 0,

n∑

j=1

aj = 1

⎫

⎬

⎭

.

Convex hull co(E) is a simplex (an n-dimensional analogue of a triangle) withvertices at ET

j , and its size can be measured by its radius (or diameter) definedin p-norm by

radp(E)=

1

2max

i,j‖ET

i − ETj ‖p, diamp(E) = 2radp(E),

where ‖·‖p (including ‖·‖∞) is the standard vector norm. The following lemmaprovides both geometrical meanings and simple yet powerful inequalities interms of norms and the aforementioned two measures.

Lemma 4.41. Consider row-stochastic matrices E, F, G ∈ ℜn×n+ with G =

EF . Then,

δ(E) = diam∞(E), λ(E) = 1 − mini,j

‖minETi , ET

j ‖1, λ(E) = rad1(E),

(4.16)and

diamp(G) ≤ λ(E)diamp(F ), or radp(G) ≤ λ(E)radp(F ), (4.17)

or simplyδ(G) ≤ λ(E)δ(F ), λ(G) ≤ λ(E)λ(F ), (4.18)

where δ(·) and λ(·) are the measures defined in (4.15).

Proof: The first two norm expressions in (4.16) are obvious from definition.To show the third expression, note that


rad1(E) =1

2max

i,j

n∑

k=1

|eik − ejk|. (4.19)

For any index pair (i, j) with i = j, the index set of 1, · · · , n can be dividedinto two complementary and mutually-exclusive sets Ωij and Ωc

ij such that(eik − ejk) is positive for k ∈ Ωij and non-positive for k ∈ Ωc

ij . Hence,

n∑

k=1

|eik − ejk| =∑

k∈Ωij

(eik − ejk) −∑

k∈Ωcij

(eik − ejk).

On the other hand, E being row-stochastic implies that

∑

k∈Ωij

(eik − ejk) +∑

k∈Ωcij

(eik − ejk) =

n∑

k=1

(eik − ejk) = 0,

and

1 =n∑

k=1

eik =∑

k∈Ωij

eik +∑

k∈Ωcij

eik.

Combining the above three equations yields

1

2

n∑

k=1

|eik − ejk| =∑

k∈Ωij

(eik − ejk)

=∑

k∈Ωij

eik −∑

k∈Ωij

ejk

= 1 −∑

k∈Ωcij

eik −∑

k∈Ωij

ejk

= 1 −n∑

k=1

mineik, ejk,

which together with (4.19) and (4.15) yields a third expression in (4.16).Next, consider again any index pair (i, j) with i = j. Let Ek, Fk, and Gk

be the kth row of matrices E, F , and G, respectively. It follows that

GTi =

n∑

k=1

eikFTk , GT

j =

n∑

k=1

ejkFTk .

Denoting βk = mineik, ejk or[

β1, · · · , βn

]T = minET

i , ETj , we know from

Ei1 = Ej1 = 1 that βij,k ≥ 0 and, for some ǫ ≥ 0,

n∑

k=1

βk + ǫ = 1,


in which ǫ = 0 if and only of eik = ejk = βk for all k. Let us now introducematrix H whose lth row Hl is given by, for l = 1, · · · , n,

HTl

= ǫFT

l +

n∑

k=1

βkFTk .

It is obvious that convex hull co(H) is a simplex with vertices at HTj , that by

the above definition co(H) ⊂ co(F ), and that

diamp(H) =

(

1 −n∑

k=1

mineik, ejk)

diamp(F ) (4.20)

since, for any (l, m),

HTl − HT

m = ǫ(FTl − FT

m) =

(

1 −n∑

k=1

mineik, ejk)

(FTl − FT

m).

If ǫ = 0, Gi = Gj = Hl, that is, Gi, Gj ∈ co(H). If 0 < ǫ ≤ 1, we chooseak = (eik − βk)/ǫ and have that

ak ≥ 0,

n∑

k=1

ak =1

ǫ

n∑

k=1

[eik − βk] =1

ǫ− 1

ǫ

n∑

k=1

βk = 1,

and thatn∑

l=1

alHTl = ǫ

n∑

l=1

alFTl +

n∑

l=1

al

n∑

k=1

βkFTk

= ǫ

n∑

l=1

alFTl +

n∑

k=1

βkFTk

=

n∑

m=1

[ǫam + βm]FTm

=

n∑

m=1

eimFTm = GT

i ,

which implies GTi ∈ co(H). Similarly, GT

j ∈ co(H). Combining the two caseson the value of ǫ, we know Gi, Gj ∈ co(H) and hence

‖GTi − GT

j ‖p ≤ diamp(H). (4.21)

Since the pair (i, j) is arbitrary, Inequality 4.17 can be concluded by combining(4.20) and (4.21).

According to the alternative definitions in (4.16), the inequalities in (4.18)are special cases of Inequality 4.17 with p = ∞ and p = 1, respectively.

Lemma 4.41 essentially shows that, for convergence of sequence P∞:0 toLimit 4.13, pre-multiplication of any matrix Pk with λ(Pk) ≤ 1 − ǫ for someǫ > 0 is a contraction mapping. As such, the following definition is introduced.


Definition 4.42. Row-stochastic matrix E is said to be a scrambling matrixif λ(E) < 1. Sequence E∞:0 is said to be sequentially scrambling if thereexists a scalar strictly-increasing sub-series ml ∈ ℵ+ : l ∈ ℵ such thatλ(Eml:ml−1

) < 1. Sequence E∞:0 is said to be uniformly sequentially scram-bling if there exist a constant ǫ > 0 and a scalar strictly-increasing sub-seriesml ∈ ℵ+ : l ∈ ℵ such that λ(Eml:ml−1

) ≤ 1 − ǫ for all l.

By the above definition, whether matrix E is scrambling or not dependssolely on locations of its zero and non-zero entries. Hence, we can define theso-called characteristic matrix as the following binary matrix: for any non-negative matrix E,

B(E)= [sign(eij)], (4.22)

where sign(·) is the standard sign function (i.e., sign(0) = 0 and sign(ǫ) = 1

if ǫ > 0). Obviously, there are at most 2n2

choices for matrix B(·); B(EF ) isinvariant for any pair of non-negative matrices (E, F ) as long as B(E) andB(F ) do not change; and, if B(E) ≥ B(F ), λ(F ) < 1 implies λ(E) < 1. Thefollowing lemma further provides useful results on both λ(·) and B(·).

Lemma 4.43. Consider row-stochastic matrices E, F, G, W ∈ ℜn×n+ with

G = EF .

(a) If E is diagonally positive, B(G) ≥ B(F ). If F is diagonally positive,B(G) ≥ B(E). If both E and F are diagonally positive, then B(G) ≥B(E + F ).

(b) If W is diagonally positive and λ(G) < 1, then λ(WEF ) < 1, λ(EWF ) <1, and λ(EFW ) < 1. That is, scramblingness of a finite-length matrixproduct is invariant under an insertion of any diagonally positive matrixanywhere.

(c) If E is a SIA matrix and B(G) = B(F ), then λ(F ) < 1 and λ(G) < 1.Similarly, if F is a SIA matrix and B(G) = B(E), then both E and G arescrambling.

Proof: It follows from

gij =

n∑

k=1

eikfkj

that, if ekk > 0, fkj > 0 implies gkj > 0 and that, if gkk > 0, eik > 0 impliesgik > 0. Hence, (a) is concluded.

It follows from (a) that, if W is diagonally positive, B(WG) ≥ B(G),B(GW ) ≥ B(G), and B(EW ) ≥ B(E) which implies B(EWF ) ≥ B(EF ).Thus, (b) is validated.

It follows from B(EF ) = B(F ) that B(E2F ) = B(EF ) and henceB(E2F ) = B(F ). By induction, we have B(EkF ) = B(F ). It follows fromthe proof of Theorem 4.39 and from (4.15) that, since E is SIA, λ(Ek) < 1


for sufficiently large k. By Lemma 4.41, λ(EkF ) ≤ λ(Ek)λ(F ) < 1. Thus,Statement (c) can be claimed by recalling B(EkF ) = B(G) = B(F ).

By Definition 4.42, a lower triangularly positive and diagonally positivematrix is scrambling (and, by Theorem 4.39, it is also SIA). For the purpose ofconcluding that sequence E∞:0 is sequentially scrambling, the following lemmareduces the requirement of Ek being lower triangularly positive to that of Ek

being lower triangularly complete.

Lemma 4.44. Consider series Ek ∈ ℜn×n+ : k ∈ ℵ of diagonally positive

and row-stochastic matrices. If matrices Ek are in its lower triangular canon-ical form and are lower-triangularly complete, there exists a finite κ for everychoice l such that E(l+κ):l is lower triangularly positive, and hence E(l+κ):l isboth scrambling and SIA.

Proof: Assume without loss of any generality that

Ek =

[

Ek,11 0Ek,21 Ek,22

]

,

where Ek,11 ∈ ℜr1×r1 and Ek,22 ∈ ℜr2×r2 are irreducible and diagonallypositive. If Ek contains more block rows, induction should be used to extendthe subsequent analysis. Let us first consider the case that dimensions of Ek,11

and Ek,22 are independent of k. In this case, it follows that

E(l+1):(l−1) =

[

E(l+1),11El,11 0El+1,21El,11 + E(l+1),22El,21 E(l+1),22El,22

]

=

[

E(l+1):(l−1),11 0E(l+1):(l−1),21 E(l+1):(l−1),22

]

,

in which, according to Statement (a) of Lemma 4.43,

B(E(l+1):(l−1),21) ≥ B(El+1,21 + El,21) (4.23)

By Corollary 4.4, we know that, for all m ≥ maxr1, r2−1, diagonal productsE(l+jm−1):(l+jm−m−1),ii are positive for i = 1, 2 and j = 1, 2, 3. It follows from(4.23) and from Ek being lower-triangularly complete that, by simply increas-ing the finite integer m, lower-triangular blocks E(l+jm−1):(l+jm−m−1),21 be-come non-zero for j = 1, 2, 3. Furthermore, it follows from E(l+2m−1):(l+m−1),ii

and E(l+m−1):(l−1),ii being positive and from

E(l+2m−1):(l−1),21 = E(l+2m−1):(l+m−1),21E(l+m−1):(l−1),11

+E(l+2m−1):(l+m−1),22E(l+m−1):(l−1),21

that a positive entry in matrix E(l+2m−1):(l+m−1),21 induces a positive row inmatrix E(l+2m−1):(l−1),21 and a positive entry in E(l+2m−1):(l+m−1),21 inducesa positive row in E(l+2m−1):(l−1),21. Therefore, by taking another multiplica-tion and repeating the argument of positive-entry propagation once more, we


know that E(l+3m−1):(l−1),21 > 0. In the second case that dimensions of diag-onal blocks of Ek vary as k increases, we know again from Statement (a) ofLemma 4.43 that, by computing products of finite length, sizes of irreducibleblocks on the diagonal are all non-decreasing and also upper bounded. Thus,the second case can be handled by applying the result of the first case to thediagonal blocks of largest sizes appeared. In summary, the conclusion is drawnby choosing κ ≥ 3m.

If λ(Pk) (or λ(Pml:ml−1)) is less than 1 but approaches 1 as l increases, se-

quence P∞:0 is sequentially scrambling but not uniformly sequentially scram-bling, and it may not have the limit of (4.13). To illustrate this point, severalmatrix sequences are included in the following example.

Example 4.45. Matrix sequence P∞:0 with

Pk =

[

1 01

k+2k+1k+2

]

, k ∈ ℵ+

is diagonally positive, lower triangularly positive and sequentially scramblingwith λ(Pk) = 1 − 1/(k + 2). Direct computation yields P∞:0 = 1

[

1 0]

. Incontrast, matrix sequence P ′

∞:0 with

P ′k =

[

1 01

(k+2)2(k+2)2−1(k+2)2

]

, k ∈ ℵ+

is also diagonally positive, lower triangularly positive and sequentially scram-bling with λ(Pk) = 1 − 1/(k + 2)2, but it does not have the limit of (4.13)as

P ′∞:0 =

[

1 013

23

]

.

Sequences P∞:0 and P ′∞:0 are not uniformly sequentially scrambling because

lower triangular elements of Pk and P ′k are vanishing as k increases.

Next, consider the pair of matrix series E∞:0 and E′∞:0, where

Ek =

⎡

⎢

⎣

1 0 01

k+2k+1k+2 0

0 (k+2)2−1(k+2)2

1(k+2)2

⎤

⎥

⎦,

E′k =

⎡

⎢

⎣

1 0 0k+1k+2

1k+2 0

0 1(k+2)2

(k+2)2−1(k+2)2

⎤

⎥

⎦.

Both sequences are sequentially scrambling but not uniformly sequentiallyscrambling as E(k+2):k and E′

(k+2):k are lower triangularly positive but criticalelements in their lower triangular forms as well as some of their diagonalelements are vanishing. It follows from direct computation that


E′∞:0 =

⎡

⎣

1 0 01 0 0

107465

48465

23

⎤

⎦ ,

while E∞:0 = 1[

1 0 0]

.

The following corollary further relaxes lower triangular completeness tosequential lower triangular completeness for diagonally positive and row-stochastic non-negative matrices. Matrix product E(l+k):l being lower triangu-larly complete is much less restrictive than any of Ek being lower triangularlycomplete. In light of Example 4.45, the uniform non-vanishing property isrequired in the definition of sequential lower triangular completeness. UnderDefinition 4.47, Lemma 4.44 reduces to Corollary 4.48.

Definition 4.46. Given matrix series Ek ∈ ℜn×n+ : k ∈ ℵ, sequence E∞:0

is said to be sequentially lower-triangular (or lower triangularly positive) ifthere exists one permutation matrix that is independent of k and maps all Ek

into the lower triangular canonical form of (4.1) (or their canonical forms areall lower-triangularly positive).

Definition 4.47. Sequence E∞:0 is said to be sequentially lower-triangularlycomplete if it is sequentially lower-triangular and if there exists a scalarstrictly-increasing sub-series ml ∈ ℵ+ : l ∈ ℵ such that products Eml:ml−1

are all lower triangularly complete and diagonally positive and that both diago-nal positiveness and lower triangular completeness are uniform1 with respect tol. Sequence E∞:0 is said to be uniformly sequentially lower-triangularly com-plete if it is lower-triangularly complete and if the differences of (ml −ml−1)are all uniformly bounded with respect to l.

Corollary 4.48. Consider series Ek ∈ ℜn×n+ : k ∈ ℵ of diagonally pos-

itive and row-stochastic matrices. If sequence E∞:0 is sequentially lower-triangularly complete, the sequence is uniformly sequentially scrambling. Ifsequence E∞:0 is uniformly sequentially lower-triangularly complete, there ex-ist a finite integer κ > 0 and a scalar strictly-increasing sub-series ml ∈ ℵ+ :l ∈ ℵ such that λ(Eml:ml−1

) ≤ 1 − ǫ (and hence sequence E∞:0 is uniformlysequentially scrambling) and ml − ml−1 ≤ κ for all l.

In the next subsection, sufficient conditions of convergence to Limit 4.13are presented, among which convergence of a uniformly sequentially scram-bling sequence is shown. In Section 4.4.4, necessary and sufficient conditionsof convergence are provided and, in particular, values of measure λ(Eml:ml−1

)need to be computed in order to determine convergence of a non-uniformlysequentially scrambling sequence E∞:0.

1 The uniformity means that diagonal elements of Eml:ml−1 are uniformly non-vanishing with respect to l and that, in every block row i of the lower triangularcanonical form of Eml:ml−1 , there is at least one j < i such that the correspondingblock on the ith block row and the jth block column is uniformly non-vanishingwith respect to l.


4.4.3 Sufficient Conditions on Convergence

The following theorem provides several sufficient conditions on sequence con-vergence as defined by (4.13).

Definition 4.49. Sequence F∞:0 is said to be a complete sub-sequence ofsequence P∞:0 if there exists a scalar sub-series mk : k ∈ ℵ, mk+1 >mk, m1 ≥ 0 of ℵ such that Fk = Pmk+1:mk

. Sequence F∞:0 is saidto be a sub-sequence of sequence P∞:0 if there exists a scalar sub-seriesmk : k ∈ ℵ, mk+1 > mk, m1 ≥ 0 of ℵ such that Fk = Pm2k+1:m2k

while(m2k+2 − m2k+1) (and hence the maximum number of consecutive matricesbeing removed) is upper bounded.

Theorem 4.50. Consider series Pk : k ∈ ℵ+ of squarely row-stochasticmatrices. Then, sequence P∞:0 is guaranteed to have the limit in (4.13) if oneof the following conditions holds:

(a) Sequence F∞:0 is sequentially lower-triangularly complete.(b) Pj2:j1 = J/n (i.e., λ(Pj2 :j1) = 0) for some pair of finite values of j1 < j2.(c) Matrices Pj are chosen from a set of finite many distinct matrices and if

all finite length products of these matrices are SIA.(d) Matrices Pj are chosen from a set of finite many distinct matrices that,

under the same permutation matrix, are transformed into the canonicalform of (4.1) and are lower triangularly complete and diagonally positive.

Proof: It follows from Lemma 4.41 that inequality

λ

⎛

⎝

k∏

j=1

Pnj+1:nj

⎞

⎠ ≤ λ

⎛

⎝

k∏

j=1

Pnj+1:nj

⎞

⎠ ≤k∏

j=1

λ(Pnj+1 :nj) (4.24)

holds for any k ∈ ℵ+, where n1 < n2 < · · · < nk. Statement (a) is obviousby applying Inequality 4.24 to a uniformly sequentially scrambling sequencein (4.13) and by invoking Corollary 4.48.

Statement (b) is also obvious since Pj is row-stochastic and hence, for anychoice of Pj with j > j2, Pj:j1 = J/n.

To show (c), consider the finite-length product Pk:j = Pk:lPl:j , wherek > l + 1 and l > j. Since characteristic matrix B(·) assumes a finite num-ber of matrix values, B(Pk:j) = B(Pl:j) must hold for some l when (k − j)becomes sufficiently large. By assumption, Pk:l is SIA. Hence, it follows fromStatement (c) of Lemma 4.43 that λ(Pk:j) < 1, and uniformity of λ(Pk:j) < 1is ensured by the fact that only a finite number of seed matrices are used.Thus, Statement (c) is concluded by applying Statement (a) to the completesub-sequence consisting of Pk:j .

Statement (d) is obviously from (c) since any finite-length product of lowertriangular and diagonally positive matrices are always lower triangular and


diagonally positive and since, by Theorem 4.39, a lower triangularly completeand diagonally positive matrix is SIA.

As evidenced in Example 4.40, Condition (c) of Theorem 4.50 requiresthat all combinations of sufficiently long matrix products be tested for the SIAproperty. In comparison, the rest of conditions in Theorem 4.50 are in termsof individual matrices and hence more desirable for applications. Nonetheless,all the conditions in Theorem 4.50 are sufficient but not necessary in general.

4.4.4 Necessary and Sufficient Condition on Convergence

The following theorem provides necessary and sufficient conditions on con-vergence to Limit 4.13. The trivial case, Statement (b) of Theorem 4.50, isexcluded in the development of the following necessary and sufficient con-ditions. Among the conditions, (d) and (e) depend only on structural (ortopological) properties of matrices Pj and hence do not involve any numericalcomputation unless some of positive entries in matrices Pj are vanishing as jincreases but they determine the aforementioned properties.

Definition 4.51. Sequence P∞:0 is said to be convergent to the limit in (4.13)if, given any integer j and any constant ǫ′ > 0, there exist κ(ǫ′, j) ∈ ℵ andcj ∈ ℜn such that, for all k ≥ κ + j,

‖Pk:j − 1cTj ‖∞ ≤ ǫ′.

Sequence P∞:0 is said to be uniformly convergent if κ(ǫ′, j) = κ(ǫ′).

Definition 4.52. Sequence P∞:0 is said to be (uniformly) sequentially com-plete if one of its sub-sequences is (uniformly) sequentially lower-triangularlycomplete.

Theorem 4.53. Consider sequence P∞:0 of squarely row-stochastic matricesand, in light of (b) in Theorem 4.50, assume without loss of any generalitythat λ(Pj2 :j1) = 0 for any j2 > j1. Then,

(a) Sequence P∞:0 is convergent to the limit in (4.13) if and only if one of itscomplete sub-sequences is.

(b) Sequence P∞:0 is convergent to the limit in (4.13) if and only if P∞:0 isuniformly sequentially scrambling.

(c) Sequence P∞:0 is convergent to the limit in (4.13) if and only if there exista constant ǫ > 0 and a scalar sub-series mk : k ∈ ℵ, mk+1 > mk, m1 ≥0 such that

∞∏

k=1

λ(Pmk+1:mk) = 0.


(d) Sequence P∞:0 is convergent to the limit in (4.13) if and only if there existsa scalar sub-series mk : k ∈ ℵ, mk+1 > mk, m1 ≥ 0 such that

∞∑

k=1

[1 − λ(Pmk+1:mk)] = +∞.

(e) If sequence P∞:0 is sequentially lower triangular and diagonally positive, itis uniformly convergent to the limit in (4.13) if and only if it is uniformlysequentially lower triangularly complete.

(f) If matrices Pk are diagonally positive uniformly respect to k, sequenceP∞:0 is uniformly convergent to the limit in (4.13) if and only if it isuniformly sequentially complete.

Proof: Necessity for (a) is trivial. To show sufficiency, assume that completesub-sequence F∞:0 be convergent, that is, by Definition 4.51, inequality

‖Fk:j − 1cTj ‖∞ ≤ ǫ′

holds for all k ≥ κ + j. Recall that ‖A‖∞ = 1 for any non-negative and row-stochastic matrix A. It follows that, for any (l′, l) satisfying l′ < mj < mk < l,Pl:l′ = Pl:mk

Fk:jPmj :l′ and

‖Pl:l′ − 1(Pmj :l′c)T ‖∞ = ‖Pl:mk

[Fk:j − 1cT ]Pmj :l′‖∞≤ ‖Pl:mk

‖∞ · ‖Fk:j − 1cT ‖∞ · ‖Pmj :l′‖∞≤ ǫ′,

from which convergence of sequence P∞:0 becomes obvious.Sufficiency for (b) is obvious from (a) and from (a) of Theorem 4.50. To

show necessity, we note from (4.16) that

λ(Pk:l) =1

2max

i,j‖PT

k:l,i − PTk:l,j‖1 ≤ max

i‖PT

k:l,i − cTl ‖1 = ‖Pk:l − 1cT

l ‖∞,

(4.25)where Pk:l,i is the ith row of Pk:l. It follows from (4.24) and from Definition4.51 that the value of λ(Pk:l) is monotone decreasing with respect to k andthat, if sequence P∞:0 is convergent, λ(Pk:l) → 0 for any l as k → ∞. Hence,for any finite mj and for any 0 < ǫ < 1, there must be a finite mj+1 such thatλ(Pmj+1 :mj

) ≤ 1 − ǫ, and necessity is shown.Sufficiency for (c) follows from (a) of Theorem 4.50, and necessity for (c)

is apparent from (b).Equivalence between (c) and (d) can be established by noting that, for any

0 ≤ βk < 1,∞∏

k=1

(1 − βk) = 0 ⇐⇒∞∑

k=1

βk = ∞. (4.26)

The above relationship is trivial if βk → 0. If βk → 0, the above relationshipcan be verified by using Taylor expansion of


log(

∏

(1 − βk))

=∑

log(1 − βk) ≈ −∑

βk.

Sufficiency of (e) is obvious from Corollary 4.48 and from (a) of Theorem4.50. It follows from (4.25) that, since the sequence is uniformly convergent,λ(Pk:l) ≤ 1− ǫ for some ǫ > 0, for all k > κ + l, and all l. Thus, the sequencemust be uniformly sequentially lower triangularly complete, and necessity of(e) is shown.

To show necessity of (f), note that, if P∞:0 is not uniformly sequentiallycomplete, none of its sub-sequences can be uniformly sequentially lower tri-angularly complete. On the other hand, uniform convergence implies thatλ(P(l+κ):l) ≤ 1 − ǫ holds for some κ, ǫ > 0 and for all l and, since thereare only a finite number of permutation matrices, at least one sub-sequence ofP∞:0 is uniformly sequentially lower triangularly complete. This apparent con-tradiction establishes the necessity. To show sufficiency of (f), assume withoutloss of any generality that E∞:0 with Ev = Pk2v+2 :k2v+1 and v ∈ ℵ denote theuniformly sequentially lower-triangularly complete sub-sequence contained inP∞:0. According to Corollary 4.48, for any ǫ > 0, there exists κ such thatλ(Ev+κ · · ·Ev+1) ≤ 1 − ǫ. It follows that

∞∏

k=0

Pk =

∞∏

v=1

[EvFv] ,

where k0 = 0, Fv = Pk2v+1:k2v, F0 = I if k1 = 0, and F0 = Pk1:0 if k1 > 0.

By Definition 4.49, Fv are diagonally positive uniformly with respect to v.Invoking (b) of Lemma 4.43, we know that λ(Ev+κFv+κ · · ·Ev+1Fv+1) ≤ 1−ǫ′,where ǫ′ > 0 is independent of v. Thus, uniform convergence of sequence P∞:0

to the limit in (4.13) can be concluded using (4.24).

The following example uses two-dimensional row-stochastic matrix se-quences to illustrate uniform convergence and non-uniform convergence toLimit 4.13 as well as non-convergence.


Example 4.54. Consider matrix series Ek : k ∈ ℵ, where 0 ≤ βk ≤ 1,

Ek =

[

1 0βk 1 − βk

]

,

and matrix Ek is lower triangularly complete. Matrix sequence E∞:0 providesthe solution to dynamic system

[

x1(k + 1)x2(k + 1)

]

= Ek

[

x1(k)x2(k)

]

.

It follows that x(k + 1) = Ek:0x(0), that the limit in (4.13) exists if and onlyif [x1(k + 1) − x2(k + 1)] → 0 as k → ∞, and that

[x1(k + 1) − x2(k + 1)] = (1 − βk)[x1(k) − x2(k)].

Hence, it follows from (4.26) that sequence E∞:0 is convergent to the limit in(4.13) if and only if the sum of

∑

k βk is divergent.Clearly, if βk does not vanish as k increases, say βk = 1/3,

∑

k βk = ∞,sequence E∞:0 is uniformly sequentially lower triangularly complete and henceis uniformly convergent to the limit in (4.13). On the other hand, βk canvanish as k increases, in which case βk : k ∈ ℵ is a Cauchy series and lowertriangular completeness of matrix Ek is not uniform. Should the Cauchy seriesbe divergent as

∑

k βk = ∞, for instance,

βk =1

k + 1,

sequence E∞:0 is not sequentially lower triangularly complete but it isconvergent to the limit in (4.13). If the Cauchy series is convergent aslimk→∞

∑

k βk < ∞, for instance,

βk =1

3k,

sequence E∞:0 does not have the limit in (4.13).Now, consider matrix series Pk : k ∈ ℵ where

Pk =

I2×2 if k = mj

Emjif k = mj

,

where mj : j ∈ ℵ is a scalar sub-series, and Emj: j ∈ ℵ is the sub-series

of Ek : k ∈ ℵ. Should the sequence of Emj: j ∈ ℵ be sequentially lower

triangularly complete, so is sequence P∞:0. On the other hand, if sequenceE∞:0 is not sequentially lower triangularly complete but is convergent and ifEmj

is lower triangularly complete (say, βk = 1/(k + 1)), the choice of scalarsub-series mj : j ∈ ℵ determines whether sequence P∞:0 is convergent. Forinstance, if βk = 1/(k + 1) and mj = 3j − 1, sequence P∞:0 is not convergentsince

βmj=

1

3j;

and if βk = 1/(k + 1) and mj = 2j, sequence P∞:0 is convergent.


Theorem 4.39 on convergence of power sequence implies that lower trian-gular completeness of certain-length matrix products is needed for sequenceP∞:0 to converge to the limit in (4.13), and diagonal positiveness required in(e) of Theorem 4.53 is also necessary in general because, due to the possibilityof arbitrary permutation, there is no other alternative to ensure primitivenessfor the first diagonal block in the canonical form of Pk. Matrix sequences aris-ing from control systems can be made diagonally positive since any entity canalways have (output) feedback from itself. The property of sequential lowertriangular completeness or sequential completeness is structural (or topolog-ical), and it ensures convergence of sequence P∞:0. The proof of (d) and (e)in Theorem 4.53 indicates that convergence of Sequence 4.13 is exponentialif it is uniformly sequentially complete. Without sequential completeness, itis evidenced by Examples 4.45 and 4.54 that the values of measure λ(·) haveto be computed for certain-length matrix products, and convergence needsto be established in general by verifying that P∞:0 is uniformly sequentiallyscrambling.


This chapter introduces non-negative matrices (as well as Metzler matricesand M-matrices) and presents their properties that are most relevant to sta-bility, Lyapunov function, and sequence convergence. Extensive discussions onnon-negative matrices can be found in [14, 18, 74, 157, 262].

Perron-Frobenius theorem is the fundamental result on non-negative ma-trices. Theorem 4.7 is the famous fixed-point theorem by Brouwer [32], andthe elegant proof of Statement (a) of Theorem 4.8 is given in [1]. Statement(b) of Theorem 4.8 is due to Perron [186]. The simple dominant eigenvalueρ(A) and its eigenvector are also known as Perron root and eigenvector ofA, respectively. Later, Frobenius extended the result to irreducible matrices,Statement (c) of Theorem 4.8. The original proof [73] used expansions of de-terminants and was rather involved. The proof on (c) of Theorem 4.8 is dueto Wielandt [273] and is based on the min-max property of ρ(A) [74].

Matrix sequences arise naturally in the study of probability theory andsystems theory. Convergence measures in Section 4.4.2 and the first inequal-ity in (4.18) are classical, and they, together with Statements (b) and (c)of Theorem 4.53, are due to Hajnal [85, 86]. Seen as a generalization, thesecond inequality in (4.18) is reported in [54]. Statement (c) of Lemma 4.43and Statement (d) of Theorem 4.50 were first reported in [274]. A historicalreview on early results on sequence convergence can be found in [225, 226].General convergence of pre-multiplicative matrix sequence (that may not berow-stochastic) is studied in [48]. Diagonally-positive matrix sequences arestudied in [200, 201], Statements (b) and (d) of Lemma 4.43 and (c) and (d)of Theorem 4.53 are developed in [202].


Applications of non-negative matrices have been found in such importantareas as probability and Markov chains [63, 70, 226], input-output modelsin economics [129, 254], criticality studies of nuclear reactors [20], popula-tion models and mathematical bioscience [35], iterative methods in numericalanalysis [262], data mining (DNA, recognition of facial expression, etc.) [127],web search and retrieval [120], positive systems [60], and cooperative controlof dynamical systems [202].

5

Cooperative Control of Linear Systems

In this chapter, the cooperative control problem of linear systems is stud-ied by first analyzing properties of linear cooperative systems. The resultson multiplicative sequence of non-negative matrices from Chapter 4 naturallyrender the cooperative stability condition for linear systems. Based on thecondition, a class of linear cooperative controls are designed, and cooperativecontrollability can be determined by the cumulative connective property ofsensing/communication network topologies over time. As applications, coop-erative control is used to solve the problems of consensus, rendezvous, trackingof a virtual leader, formation control, and synchronization. The issue of main-taining network connectivity is also addressed.

In the cooperative control problem, information are collected or transmit-ted through a sensing/communication network, and the cooperative controllaw is designed to account for topological changes of the network. Availabilityof the feedback needed for implementing decentralized controls is not contin-uous or continual or even predictable, which is the main distinction from astandard control problem. This distinction is obviously desirable for applica-tions and positive for implementation, but it makes stability analysis muchmore involved. To establish the connection back to the standard methods andto describe the cumulative effect of topology changes, an average system is in-troduced and derived for cooperative systems. The average system enables usto determine Lyapunov function and control Lyapunov function for coopera-tive systems, although common Lyapunov function is not expected to exist ingeneral. Robustness of cooperative systems against measurement noises anddisturbance is also studied.

5.1 Linear Cooperative System

Dynamics of a continuous-time linear cooperative system can be expressed as

x = [−I + D(t)]x, x(t0) given, t ≥ t0, (5.1)

196 5 Cooperative Control of Linear Systems

where x ∈ ℜn, and matrix D(t) ∈ ℜn×n is assumed to have the followingproperties:

(a) Matrix D(t) is non-negative and row-stochastic.(b) Changes of matrix D(t) are discrete and countable. That is, matrix D(t)

is time-varying but piecewise-constant.(c) Matrix D(t) is diagonally positive1.

The properties come from the fact that, as explained in Section 1.1, sub-systems of a cooperative system are to achieve the same objective while theirenvironment keeps changing. Specifically, the sub-systems moving toward thesame objective imply that the cooperative system must be a positive systemand, as shown in Section 4.2, D(t) should be non-negative and could be row-stochastic. In the sequel, further analysis shows that Property (a) capturesthe essence of cooperativeness. As to Property (b), the presence of a sens-ing/communication network yields discrete and intermittent changes in thefeedback topology and hence the same type of changes in System Dynamics5.1. In the subsequent analysis, tk : tk > tk−1, k ∈ ℵ denotes the sequenceof time instants at which changes occur in matrix D(t). Property (c) canalways be assumed since every sub-system can have feedback continuouslyabout itself.

5.1.1 Characteristics of Cooperative Systems

Basic characteristics of Cooperative System 5.1 can be seen from the followingobservations.

Set of equilibrium points: It follows from D(t) being row-stochastic that1/n is the right eigenvector associated with eigenvalue ρ(D) = 1 and that, for

any state xe ∈ Ωe with Ωe = x ∈ ℜn : x = c1, c ∈ ℜ,

xe = [−I + D(t)]xe = −xe + cD(t)1 = −xe + c1 = 0.

Hence, xe = c1 includes the origin as the special case of c = 0, points xe areamong the equilibria of System 5.1, and set Ωe of all the state variables beingequal provides a mathematical description of the cooperative system reachingthe common objective. By Theorem 4.31, Ωe is the set of all the equilibriumpoints of System 5.1 if D(t) = D is irreducible. If D(t) = D is reducible,System 5.1 may have other equilibrium points outside Ωe. It follows from(5.1) that, given D(t) = D, all the equilibrium points are the solution to thematrix equation

Dxe = xe, or equivalently, D∞xe = xe.

1 This property is not necessarily needed for continuous-time systems but is re-quired for discrete-time systems.

5.1 Linear Cooperative System 197

We know from Theorem 4.39 (if Property (c) holds) together with Theorem4.31 (if Property (c) is not imposed) that there is no equilibrium point otherthan those in Ωe if and only if matrix D is lower triangularly complete.

Stationary center of mass: Within the time interval [tk, tk+1), matrixD(t) = D(k) is constant. Letting ηk be the left eigenvector of D(tk) associated

with eigenvalue ρ(D) = 1 and defining the center of mass Ok(t)= ηT

k x(t), weknow that

dOk(t)

dt= ηT

k [−I + D(t)]x = −ηTk x + ηT

k D(t)x = −ηTk x + ηT

k x = 0, (5.2)

and hence Ok(t) is a also piecewise-constant time function over time intervalst ∈ [tk, tk+1). According to Theorem 4.8, ηk > 0 and Ok(t) is defined in termsof the whole state if D(t) is irreducible, and Ok(t) may become degenerate ifD(t) is reducible. Over time, Ok(t) generally jumps from one point to anotheraccording to k. If D(t) is also column-stochastic, ηk ≡ 1/n, and Ok(t) reducesto the average of the state variables and is invariant over time.

Uniform and non-decreasing minimum: Let xmin(t)= mini xi(t) denote

the element-wise minimum value over time, and let Ωmin(t) be the timedindex set such that xi(t) = xmin(t) if i ∈ Ωmin(t). It follows that, for anyi ∈ Ωmin(t),

xi = −xi +n∑

j=1

dijxj =n∑

j=1

dij(xj − xi) ≥ 0,

since dij ≥ 0 and (xj − xi) = [xj(t) − xmin(t)] ≥ 0. Hence, System 5.1 isminimum preserving in the sense that xi(t) ≥ xmin(t0) for all i and for allt ≥ t0.

Uniform and non-increasing maximum: Let xmax(t)= maxi xi(t) denote

the element-wise maximum value over time, and let Ωmax(t) be the timedindex set such that xi(t) = xmax(t) if i ∈ Ωmax(t). It follows that, for anyi ∈ Ωmax(t),

xi = −xi +n∑

j=1

dijxj =n∑

j=1

dij(xj − xi) ≤ 0,

since dij ≥ 0 and (xj − xi) = [xj(t) − xmax(t)] ≤ 0. Hence, System 5.1 ismaximum preserving in the sense that xi(t) ≤ xmax(t0) for all i and for allt ≥ t0.

Combining the last two properties, we know that ‖x(t)‖∞ ≤ ‖x(t0)‖∞ andthus System 5.1 is ∞-norm preserving and Lyapunov stable. Due to equilib-rium points of xe = c1, System 5.1 is not locally asymptotically stable (withrespect to the origin). Stability of equilibrium points xe = c1 will be analyzedin the next subsection.

It should be noted that System 5.1 is the special case of System 4.6 withk1 = k2 = 1 and that the presence of k2 > 0 and k2 = 1 does not affect


stability analysis. As demonstrated in Section 2.4.2, eigenvalue analysis cannotbe applied in general to either System 5.1 or System 4.6 due to the presenceof time-varying matrix D(t). Nonetheless, the above analysis has establishedStatement (c) of Corollary 4.19 for Time-varying System 4.6, and it can alsobe used to show that Statements (a) and (b) of Corollary 4.19 hold in generalfor System 4.6. The subsequent analysis is to determine the condition(s) underwhich Time-varying System 5.1 converges to equilibrium set Ωe.

5.1.2 Cooperative Stability

We begin with the following definition. It follows from the ∞-norm preservingproperty that System 5.1 is always cooperatively stable. Hence, asymptoticcooperative stability remains to be investigated for System 5.1. It is worthmentioning that, if k1 > 1, System 4.6 is asymptotically stable and henceDefinition 5.1 is satisfied with c = 0. This is the trivial case of asymptoticcooperative stability and not of interest in the subsequent discussions.

Definition 5.1. System x = F(x, t) is said to be cooperatively stable if, forevery given ǫ > 0, there exists a constant δ(t0, ǫ) > 0 such that, for initialcondition x(t0) satisfying ‖xi(t0) − xj(t0)‖ ≤ δ, ‖xi(t) − xj(t)‖ ≤ ǫ for alli, j and for all t ≥ t0. The system is said to be asymptotically cooperativelystable if it is cooperatively stable and if limt→∞ x(t) = c1, where the value ofc ∈ ℜ depends upon the initial condition x(t0) and changes in the dynamics.The system is called uniformly asymptotically cooperatively stable if it isasymptotically cooperatively stable, if δ(t0, ǫ) = δ(ǫ), and if the convergence oflimt→∞ x(t) = c1 is uniform.

In control theory, it is typical to convert a convergence problem intoa standard stability problem. To determine the convergence condition oflimt→∞ x(t) = c1 without the knowledge of c, we can define the error stateeci

as, for some chosen i ∈ 1, · · · , n,

eci=[

(x1 − xi) · · · (xi−1 − xi) (xi+1 − xi) · · · (xn − xi)]T ∈ ℜn−1. (5.3)

It follows that, given the ∞-norm preserving property, System 5.1 is asymp-totically cooperatively stable if and only if limt→∞ eci

(t) = 0. The followinglemma is useful to find the dynamic equation of error state eci

.

Lemma 5.2. Consider matrix set M1(c0) defined in Section 4.2. Let Wi ∈ℜ(n−1)×n be the resulting matrix after inserting −1 as the ith column intoI(n−1)×(n−1), and let Gi ∈ ℜn×(n−1) denote the resulting matrix after elimi-nating as the ith column from In×n, that is,

W1=[

−1 I(n−1)×(n−1)

]

, G1=

[

0I(n−1)×(n−1)

]

.

5.1 Linear Cooperative System 199

Then, WiGi = I(n−1)×(n−1), H = HGiWi for any matrix H ∈ M1(0), andWiD = WiDGiWi for matrix D ∈ M1(c0) with c0 ∈ ℜ.

Proof: The first equation is obvious. To show H = HGiWi, note that GiWi

is the resulting matrix after inserting 0 as the ith row into Wi. Thus, all thecolumns of HGiWi are the same as those of H except for the ith columnwhich is the negative sum of the rest columns of H and hence is the same asthe ith column of H since H ∈ M1(0).

It follows from D ∈ M1(c0) that [−cI+D] ∈ M1(0) and, by the precedingdiscussion, [−cI + D] = [−cI + D]GiWi. Pre-multiplying Wi on both sidesyields

WiD = Wi[−cI + D] + cWi

= Wi[−cI + D]GiWi + cWi

= WiDGiWi − cWiGiWi + cWi

= WiDGiWi,

which is the last equation.

It follows from Lemma 5.2 that, under transformation eci

= Wix, System

5.1 is mapped into

eci= Wi[−I + D(t)]x

= Wi[−I + D(t)]GiWix

= Wi[−I + D(t)]Gieci

= [−I + WiD(t)Gi]eci, (5.4)

where eci∈ ℜn−1 with eci

(0) = Wix(0). Hence, asymptotic cooperative sta-bility of System 5.1 is converted into the standard stability problem of System5.4 which is also linear and piecewise-constant.

Conceptually, there are three methods to analyze asymptotic cooperativestability for System 5.1. The first method is to extend further the analysis inSection 5.1.1, and it will be presented in Chapter 6 as it applies to both lin-ear and non-linear systems. The second method is to convert the asymptoticcooperative stability problem into a standard asymptotic stability problem asshown above in (5.4) but, since the matrix product WiD(t)Gi is no longer anon-negative matrix, this method is studied later in Section 5.6. The thirdmethod is to utilize explicitly all the properties of D(t) and then apply The-orem 4.53, which leads to the following results. Specifically, it follows from(5.1) that

x(tk+1) = Pkx(tk), x0 given, k ≥ 0, (5.5)

where Pk = e[−I+D(tk)](tk+1−tk) is the state transient matrix, Lemma 5.3 sum-marizes the properties of Pk, and hence Theorem 5.4 is the consequence ofapplying Lemma 5.3 and Theorem 4.53.


Lemma 5.3. Consider matrix P = e(−I+D)τ where D ∈ ℜn×n+ is row-

stochastic and constant. Then, for any finite τ > 0, matrix P is diagonallypositive and also row-stochastic, P is positive if and only if D is irreducible,and P is lower triangularly positive if and only if D is lower triangularlycomplete.


P = e(−I+D)τ = e−2τe(I+D)τ = e−2τ∞∑

i=0

1

i!(I + D)iτ i,

from which diagonal positiveness is obvious. It follows from (I + D)i1 = 2i1

that P1 = 1. It follows from Corollary 4.4 that (I + E)i > 0 for i ≥ (n − 1)and hence P > 0 if and only if D is irreducible. If D is lower triangularlycomplete, so are (I + E)i and hence P , and vice versa. By Lemma 4.44, P isalso lower triangularly positive.

Theorem 5.4. System 5.1 or its piecewise solution in (5.5) is uniformlyasymptotically cooperatively stable if and only if D(tk) : k ∈ ℵ is uniformlysequentially complete.

Theorem 5.4 provides the conditions on both stability and cooperativecontrollability, and the conditions are in terms of topologies changes as well asthe properties of matrix D(t). While topology changes may not be predicted,cooperative control design is to render a networked control system in the formof (5.1) and to ensure the properties for matrix D(t).

5.1.3 A Simple Cooperative System

Arguably, the simplest cooperative system is the alignment problem shownin Fig. 5.1 in which particles move at the same constant absolute velocityand their directions of motion are adjusted according to the local informationavailable. Through experimentation, Vicsek [47, 263] discovered that align-ment can be achieved by the so-called neighboring rule: motion direction ofany given particle is adjusted regularly to the average direction of motion ofthe particles in its neighborhood of radius r. To ensure alignment for thoseparticles far from the rest, random perturbation should be added.

Mathematically, the neighboring rule can be expressed as

θi(k + 1) =1

1 + ni(k)

⎛

⎝θi(k) +∑

j∈Ni(k)

θj(k)

⎞

⎠ , (5.6)

where Ni(k) is the index set of those particles in the neighborhood of theith particle, ni(k) is the number of elements in Ni(k), and both of them aretime-varying. The continuous-time counterpart of (5.6) is given by

5.2 Linear Cooperative Control Design 201

jp

ip

lp

v

v

v

j

i

l

Fig. 5.1. Motion alignment and cooperative behavior of self-driven particles

θi = −θi(t) +1

1 + ni(t)

⎛

⎝θi(t) +∑

j∈Ni(t)

θj(t)

⎞

⎠ , (5.7)

where Ni(t) and ni(t) correspond to Ni(k) and ni(k), respectively. SinceNeighboring Rule 5.7 is already in the form of (5.1), Theorem 5.4 providesthe theoretical guarantee that this remarkably simple model can render thecooperative behavior of θi(t) → c for all i. Later, (5.6) will be studied (as aspecial case of (5.20)), and its convergence will be claimed (as an applicationof Theorem 5.12).

It should be noted that Neighboring Rule 5.6 is intuitive and reflects thegroup moving behavior. Indeed, this simple rule has been used in cellularautomata [275], reaction-diffusion models [160, 224], flow of granular materials[88], animal aggregation [217, 260], etc.

5.2 Linear Cooperative Control Design

In this section, the cooperative control problem is studied for a set of hetero-geneous systems whose dynamics can be transformed into the following form:for i = 1, · · · , q,

xi = (−I + Ai)xi + Biui, yi = Cixi, ϕi = ψi(t, ϕi, xi), (5.8)

where yi ∈ ℜm is the output, ui ∈ ℜm is the cooperative control to bedesigned, integer li ≥ 1 represents the relative degree of the ith system, xi ∈ℜlim is the linear state of the ith system, Kc ∈ ℜm×m is a non-negative designmatrix chosen to be row-stochastic and irreducible,


Ai =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎣

0 Im×m 0 · · · 0 0

0 0 Im×m. . . 0 0

.... . .

. . .. . .

. . ....

0 0 · · · 0 Im×m 00 0 0 · · · 0 Im×m

Kc 0 0 · · · 0 0

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎦

=

[

0 I(li−1)×(li−1) ⊗ Im×m

Kc 0

]

,

Bi =

[

0Im×m

]

∈ ℜ(lim)×m, Ci =[

Im×m 0]

∈ ℜm×(lim),

ϕi ∈ ℜni−lim is the vector of state variables associated with internal dy-namics, and the internal dynamics of ϕi = ψi(t, ϕi, xi) are assumed to beinput-to-state stable. If li − 1 = 0, the corresponding rows and columnsof I(li−1)×(li−1) ⊗ Im×m are empty, i.e., they are removed from matrix

Ai ∈ ℜ(lim)×(lim).Equation 5.8 is selected to be the canonical form for heterogenous systems

because it has the following general yet useful properties:

(a) Except for the presence of Kc, matrix pair Ai, Bi is in the controllablecanonical form, matrix pair Ai, Ci is in the observable canonical form;and hence pairs −I + Ai, Bi and −I + Ai, Ci are controllable andobservable, respectively. Indeed, by defining a proper output vector, anystabilizable linear time-invariant system can be transformed (under a statetransformation and a self-feedback control) into (5.8), and so can an input-output feedback linearizable non-linear (or linear) system with input-to-state stable internal dynamics.

(b) It is obvious that matrix Ai has the same properties of being row-stochastic and irreducible as matrix Kc. Thus, by Theorem 5.4, the ithsystem in (5.8) and with ui = 0 is asymptotically cooperatively stable byitself.

(c) It is straightforward to show that, under the output-feedback set-pointcontrol ui = Kc(c01 − yi) = c01 − Kcyi for any fixed c0 ∈ ℜ, the linearportion in (5.8) is by itself asymptotically cooperatively stable as xi →c01.

The linear cooperative control problem to be addressed is to ensure thatthe collection of all the systems becomes asymptotically cooperatively stable.To solve this problem, cooperative controls need to be designed accordingto the information available, and the corresponding feedbacks are typicallyselective and intermittent, which represents one of main differences betweencooperative control and standard control problems.

5.2.1 Matrix of Sensing and Communication Network

Unless mentioned otherwise, we consider the general case that the systemsmay operate by themselves for most of the time and that exchange of output


information among the systems occurs only intermittently and selectively (ac-cording to their bandwidth, physical postures, distances, etc.). To focus uponthe analysis and design of cooperative control, we choose to admit any sens-ing/communication protocol that enables networked information exchange.

Without loss of any generality, let infinite sequence tk : k ∈ ℵ denotethe time instants at which the topology of sensing/communication networkchanges. Then, the following binary and piecewise-constant matrix S(t), calledsensing/communication matrix, captures the instantaneous topology of infor-mation exchange: S(t) = S(k) for all t ∈ [tk, tk+1) and

S(t) =

⎡

⎢

⎢

⎢

⎣

s11 s12(t) · · · s1q(t)s21(t) s22 · · · s2q(t)

......

......

sq1(t) sq2(t) · · · sqq

⎤

⎥

⎥

⎥

⎦

(5.9)

where sii ≡ 1, sij(t) = 1 if the output of the jth dynamical system is knownto the ith system at time t, and sij = 0 if otherwise. As the default setting,it is assumed that 0 < ct ≤ tsk+1 − tsk ≤ ct < ∞, where ct and ct are constantbounds. If S(t) is constant or becomes so after some finite time, time sequencetk : k ∈ ℵ can always be chosen to be infinitely long with incrementalupper bound ct by letting S(tk) be identical. In the case that some topologychanges occurred in certain time intervals of infinitesimal or zero length, thesechanges could not be detected or reacted by the physical systems, and hencethey should be excluded from control design, and the subsequent analysis anddesign hold for the rest of topological changes.

Analysis and design of cooperative control do not require that time se-quence tk : k ∈ ℵ or the corresponding changes of S(t) be predictable orprescribed or known a priori or modeled in any way. Instead, the time se-quence and the corresponding changes of sij(t) (where j = 1, · · · , q) in the ithrow of matrix S(t) are detectable instantaneously by (and only by) the ithsystem, and the cooperative control ui reacts to the changes by taking theavailable information into its calculation. Specifically, the cooperative controlshould be of general form

ui = Ui(si1(t)(y1 − yi), · · · , siq(t)(yq − yi)), (5.10)

which requires only a selective set of feedback on relative output measure-ments. In addition to topology changes, the sensing/comminucation networkmay also have latency, which is the subject of Section 6.6.1.

5.2.2 Linear Cooperative Control

A general class of linear relative-output-feedback cooperative controls are ofform: for i = 1, · · · , q,


ui =1

q∑

η=1

wiη(t)siη(t)

q∑

j=1

wij(t)Kc[sij(t)(yj − yi)], (5.11)

where wij(t) > 0 are generally time-varying weights, sij(t) are piecewise-constant as defined in (5.9), and Kc is the matrix contained in Ai. It followsthat, if sij(t) = 0, ui does not depend on yj and hence is always implementablewith all and only the available information. To facilitate the derivation ofclosed-loop dynamics, we rewrite Control 5.11 as

ui =

q∑

j=1

Gij(t)yj − Kcyi= Gi(t)y − Kcyi, (5.12)

where ni = lim, l =∑q

i=1 li, n =∑q

i=1 ni = lm, y = [yT1 · · · yT

q ]T ∈ ℜqm

is the overall output, Gi(t)=[

Gi1(t) · · · Giq(t)]

is the feedback gain matrix,and its elements are defined by

Gij(t)=

wij(t)sij(t)q∑

η=1

wiη(t)siη(t)

Kc.

Combining (5.8) and (5.11) for all i and exploring the special structure ofmatrices Ai, Bi, Ci yields the following closed-loop and overall system:

x = [−I + A + BG(t)C]x = [−I + D(t)]x, (5.13)

whereA = diag(A1 − B1Kc), · · · , (Aq − BqKc) ∈ ℜn×n,

B = diagB1, · · · , Bq ∈ ℜn×(mq), C = diagC1, · · · , Cq ∈ ℜ(mq)×n,

G(t) =[

GT1 (t) · · · GT

q (t)]T ∈ ℜ(mq)×(mq), and D(t) ∈ ℜn×n is given by

D(t) =

⎡

⎢

⎣

D11(t) · · · D1q(t)...

......

Dq1(t) · · · Dqq(t)

⎤

⎥

⎦,

⎧

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎩

Dii(t) =

[

0 I(li−1)×(li−1) ⊗ Im×m

Gii(t) 0

]

,

Dij(t) =

[

0 0Gij(t) 0

]

, if i = j.

(5.14)It is obvious that matrix D(t) is piecewise-constant (as matrix S(t)) and alsorow-stochastic. The following lemma establishes the equivalence of structuralproperties among matrices S(t), G(t) and D(t). Combining the lemma withTheorem 5.4 yields Theorem 5.6.


Lemma 5.5. The following statements are true:

(a) Consider matrix E ∈ ℜ(qm)×(qm)+ with sub-blocks Eij ∈ ℜm×m

+ . Then,matrix

E =

⎡

⎢

⎢

⎢

⎣

0 Im×m 0 · · · 0E11 0 E12 · · · E1q

......

......

Eq1 0 Eq2 · · · Eqq

⎤

⎥

⎥

⎥

⎦

is irreducible (or lower triangularly complete in its canonical form) if andonly if E is irreducible (or lower triangularly complete in its canonicalform).

(b) Given matrices S ∈ ℜq×q+ and F ∈ ℜm×m

+ with F being irreducible androw-stochastic, matrix S⊗F is irreducible (or lower triangularly completein its canonical form) if and only if S is irreducible (or lower triangularlycomplete in its canonical form).

(c) Matrix D(t) in (5.14) is irreducible (or lower triangularly complete) if andonly if S(t) is irreducible (or lower triangularly complete).

Proof:

(a) To show that E is irreducible if E is irreducible, consider vectors z1, α ∈ℜm

+ , z2 ∈ ℜ(q−1)m+ , z =

[

zT1 zT

2

]T, z =

[

zT1 αT zT

2

]T, and z′ =

[

αT zT2

]T. It follows that, for any γ ≥ 1, inequality γz ≥ Ez is equivalent

toγz1 ≥ α, γz′ ≥ Ez, (5.15)

which implies thatγ2z ≥ γz′ ≥ Ez.

If E is irreducible, we know from Corollary 4.3 and from the above in-equality that z > 0 and Ez > 0. It follows from (5.15) that z > 0 and, byCorollary 4.3, E is irreducible.On the other hand, if E is reducible, there is non-zero and non-positivevector z such that γz ≥ Ez for some γ ≥ 1. Inequality γz ≥ Ez implies

γz′ ≥ Ez′, where vector z′ =[

zT1 zT

1 zT2

]Tis also non-zero and non-

positive. Hence, by Corollary 4.3, E must be reducible.Matrix E can be permutated into

E′=

⎡

⎢

⎢

⎢

⎢

⎢

⎣

0 E11 E12 · · · E1q

Im×m 0 0 · · · 00 E21 E22 · · · E2q

......

......

0 Eq1 Eq2 · · · Eqq

⎤

⎥

⎥

⎥

⎥

⎥

⎦

.

It is apparent that, if E is permutated to be lower triangular (either

complete or incomplete), E′has the same property.


(b) The proof is parallel to that of (a) but uses the facts that, for any z ∈ ℜmq+ ,

inequality z′⊗1m ≤ z ≤ z′′⊗1m holds for some z′, z′′ ∈ ℜq+ and that, for

any z ∈ ℜq+, γ(z ⊗ 1m) ≥ (S ⊗ F )(z ⊗ 1m) holds if and only if γz ≥ Sz.

(c) Matrix G(t) in (5.13) and (5.12) has the property that

B(G(t)) = B(S(t) ⊗ Kc),

where B(·) is defined in (4.22). The conclusion can be made by invoking(b) and then by using (a) inductively with respect to relative degrees ofli and by applying appropriate permutations.

Theorem 5.6. System 5.8 is uniformly asymptotically cooperatively stableunder Cooperative Control 5.11 if and only if the sensing/communication ma-trix S(t) in (5.9) has the property that S(tk) : k ∈ ℵ is uniformly sequen-tially complete.

5.2.3 Conditions of Cooperative Controllability

The analysis leading to Theorem 5.6 illustrates that, for a group of heteroge-neous linear systems, their cooperative controllability calls for two conditions:(a) the individualized property that each and every system in the group isinput-output dynamically feedback linearizable and has input-to-state stableinternal dynamics, and (b) the network property that S(tk) : k ∈ ℵ issequentially complete.

The requirement of individual systems being dynamic feedback linearizablewill be relaxed in Chapter 6. In many applications, outputs of the systemsare physically determined, and a cooperative behavior of system outputs isdesired. In these cases, the following output cooperative stability should bepursued.

Definition 5.7. Systems of (5.16) are said to be asymptotically output coop-eratively stable if all the states are Lyapunov stable and if limt→∞ y(t) = c1.

To show sufficiency of feedback linearization and to demonstrate the flex-ibility in choosing self-feedback controls for achieving output cooperative sta-bility, consider the ith system in its original state equation as

zi = Fi(zi) + Bi(zi)wi, yi = h(zi), (5.16)

where zi ∈ ℜni and yi, wi ∈ ℜm. Since the system is input-output feedbacklinearizable, there exists a transformation wi = αi(zi) + βi(zi)ui such that

y(li)i = −Ki

[

yTi · · ·

(

y(li−1)i

)T]T

+ ui.

Gain matrix Ki can be chosen to match the above equation with the canonicalform of (5.8). For output cooperative stability, there are many choices of Ki

and ui, and they are summarized into the following theorem.


Theorem 5.8. Systems in the form of (5.16) are uniformly asymptoticallyoutput cooperatively stable under Cooperative Control 5.11 if the followingconditions hold:

(a) There exists a mapping wi = αi(zi)+βi(zi)ui such that the dynamics fromoutput yi to input ui are linear and positive and that, under the controlui = Kc(c1 − yi), yi converges to the steady state of c1.

(b) Sensing/communication matrix S(tk) : k ∈ ℵ is uniformly sequentiallycomplete.

As shown in Section 4.4.4, the network condition of S(tk) : k ∈ ℵ beinguniformly sequentially complete is both sufficient and necessary for achiev-ing uniform asymptotic cooperative stability. A simple way of monitoringuniform sequential completeness is to compute the binary products of sen-sor/communication matrices sequence S(tk) over certain consecutive time in-tervals. That is, for any sub-sequence kη : η ∈ ℵ, the cumulative effect ofnetwork topology changes is determined by the cumulative exchange of in-formation, and the latter over time interval [tkη

, tkη+1) is described by thecomposite matrix

SΛ(η)= S(tkη+1−1)

∧

S(tkη+1−2)∧

· · ·∧

S(tkη), (5.17)

where∧

denotes the operation of generating a binary product of two binarymatrices. Hence, sequence S(tk) is sequentially complete if SΛ(η) is lowertriangularly complete for all η ∈ ℵ, that is, there exists a permutation matrixT (η) under which

T T (η)SΛ(η)T (η) =

⎡

⎢

⎢

⎢

⎣

S′Λ,11(η) 0 · · · 0

S′Λ,21(η) S′

Λ,22(η) · · · 0...

.... . .

...S′

Λ,p1(η) S′Λ,p2(η) · · · S′

Λ,pp(η)

⎤

⎥

⎥

⎥

⎦

, (5.18)

where p(η) > 0 is an integer, S′Λ,ii(η) are square and irreducible, and

S′Λ,ij(η) = 0 for some j < i. Once kη is known, kη+1 should inductively be

found such that lower triangular completeness of (5.18) holds. If the compos-ite matrix in (5.18) would be observable real-time, sensing/communicationprotocols could be devised and implemented to allocate resources, to im-prove information exchange, and to achieve the lower triangular complete-ness. Of course, such a monitoring and optimization scheme would requiremore than a truly decentralized, local-information-sharing, and memorylessnetwork. The following examples provide and illustrate sequentially completesensor/communication sequences.

Example 5.9. Consider a sensor/communication sequence S(tk), k ∈ ℵ de-fined by S(tk) = S1 for k = 4η + 3, S(tk) = S2 for k = 4η + 2, S(tk) = S3 fork = 4η + 1, and S(tk) = S4 for k = 4η, where η ∈ ℵ,


S1 =

⎡

⎣

1 0 01 1 00 0 1

⎤

⎦ , S2 =

⎡

⎣

1 1 00 1 00 0 1

⎤

⎦ ,

S3 =

⎡

⎣

1 0 00 1 01 0 1

⎤

⎦ , and S4 =

⎡

⎣

1 0 00 1 00 0 1

⎤

⎦ . (5.19)

It follows that

SΛ(η)= S1

∧

S2

∧

S3

∧

S4 =

⎡

⎣

1 1 01 1 01 0 1

⎤

⎦

=

[

S′Λ,11 ∅

S′Λ,21 1

]

,

from which sequential completeness of sequence S(tk), k ∈ ℵ becomes ap-parent.

Similarly, it can be shown that every sequence generated from patterns Si

is sequentially complete as long as the sequence contains infinite entries of allthe patterns of S1, S2, S3 and S4. Thus, for the purpose of simulations andverification, one can generate a sequentially complete sequence by randomlyselecting the next time instant tk and then by randomly choosing S(tk) fromS1 up to S4. Unless mentioned otherwise, the following choices are made in allthe subsequent simulation studies: tk+1−tk = 1 is set, and a random integer isgenerated under uniform distribution to select S(tk) from S1 up to S4. Thatis, the sensing/communication sequences generated in the simulations haveprobability 1 to be uniformly sequentially complete.

Example 5.10. A lower triangularly complete composite sensor/communicationmatrix SΛ(·) may arise from lower triangularly incomplete matrices S(tk) ofdifferent block sizes. For example, consider sequence S(tk) : k ∈ ℵ definedby S(tk) = S1 if k = 2η and S(tk) = S2 if k = 2η + 1, where η ∈ ℵ,

S1 =

⎡

⎢

⎢

⎢

⎢

⎣

1 0 0 0 01 1 1 0 00 1 1 0 00 0 0 1 00 0 0 0 1

⎤

⎥

⎥

⎥

⎥

⎦

, and S2 =

⎡

⎢

⎢

⎢

⎢

⎣

1 0 0 0 00 1 0 0 00 0 1 1 00 0 0 1 10 0 1 0 1

⎤

⎥

⎥

⎥

⎥

⎦

.

It follows that

SΛ(η)= S2

∧

S1 =

⎡

⎢

⎢

⎢

⎢

⎣

1 0 0 0 01 1 1 0 00 1 1 1 00 0 0 1 10 1 1 0 1

⎤

⎥

⎥

⎥

⎥

⎦

,

which implies SΛ(η) : η ∈ ℵ is sequentially lower-triangularly complete.

Example 5.11. A lower triangularly complete composite sensor/communicationmatrix SΛ(·) may also arise from lower triangularly incomplete matrices S(tk)


whose canonical forms require different permutation matrices. For instance,consider sequence S(tk) : k ∈ ℵ defined by S(tk) = S1 if k = 3η, S(tk) = S2

if k = 3η + 1, and S(tk) = S3 if k = 3η + 2, where η ∈ ℵ,

S1 =

⎡

⎢

⎢

⎢

⎢

⎣

1 0 0 0 00 1 0 0 00 0 1 0 00 1 1 1 00 0 0 0 1

⎤

⎥

⎥

⎥

⎥

⎦

, S2 =

⎡

⎢

⎢

⎢

⎢

⎣

1 0 1 0 00 1 0 1 01 0 1 0 00 1 0 1 00 0 0 0 1

⎤

⎥

⎥

⎥

⎥

⎦

,

and

S3 =

⎡

⎢

⎢

⎢

⎢

⎣

1 0 0 0 00 1 0 0 00 0 1 0 00 0 0 1 00 0 1 0 1

⎤

⎥

⎥

⎥

⎥

⎦

.

It is apparent that matrices S1, S2 and S3 are all reducible and lower triangu-larly incomplete. Their canonical forms are of different sizes on the diagonaland require different permutation matrices: Ts(t3η) = Ts(t3η+2) = I, and

Ts(t3η+1)=

⎡

⎢

⎢

⎢

⎢

⎣

0 1 0 0 00 0 0 1 01 0 0 0 00 0 1 0 00 0 0 0 1

⎤

⎥

⎥

⎥

⎥

⎦

.

Nonetheless, sequence S(k), k ∈ ℵ is sequentially complete because an infi-nite lower-triangularly complete sub-sequence SΛ(η) : η ∈ ℵ can be con-

structed as SΛ(η)= S(t3η+2)

∧

S(t3η+1)∧

S(t3η) = S3

∧

S2

∧

S1 and itscanonical lower triangular form is

T TΛ SΛ(η)TΛ =

⎡

⎢

⎢

⎢

⎢

⎣

1 1 0 0 01 1 0 0 01 0 1 1 01 0 1 1 01 1 0 0 1

⎤

⎥

⎥

⎥

⎥

⎦

,

where TΛ = Ts(t3η+1).

5.2.4 Discrete Cooperative System

Parallel to the study of Continuous System 5.1, discrete-time cooperativesystem

x(k + 1) = Pkx(k), x0 given, k ≥ 0, (5.20)

can be investigated, the following theorem is a restatement of Theorem 4.53,and Theorem 5.12 corresponds to Theorem 5.4.


Theorem 5.12. System 5.20 is uniformly asymptotically cooperatively stableif and only if Pk : k ∈ ℵ is uniformly sequentially complete and uniformlydiagonally positive.

For discrete-time heterogeneous systems, their canonical form is given by

xi(k + 1) = [cdI + (1 − cd)Ai]xi(k) + (1 − cd)Biui(k), yi(k) = Cixi(k),ϕi(k + 1) = gi(k, ϕi(k), xi(k)),

(5.21)where Ai, Bi, Ci are the same as those in (5.8), and 0 < cd < 1 is a designconstant. Then, under the same cooperative control ui(k) given by (5.11)except that t is replaced by k everywhere therein, the overall closed-loopsystem becomes (5.20), where Pk = cdI + (1 − cd)D(k) and matrix D(k)is structurally the same as D(t) in (5.13). Thus, applying Lemma 5.5 andTheorem 5.12 yields Theorem 5.13 (which corresponds to Theorem 5.6).

Theorem 5.13. System 5.21 is uniformly asymptotically cooperatively stableunder Cooperative Control 5.11 (after replacing t with k) if and only if thecorresponding sensing/communication matrix S(k) : k ∈ ℵ is uniformlysequentially complete.

5.3 Applications of Cooperative Control

In this section, Cooperative Control 5.11 is applied to several typical problemsand its performance is illustrated by simulation results. Unless stated other-wise, topology changes in the simulations are chosen according to Example5.9, i.e., the sensing/communication matrix is randomly switched among theseed matrices in (5.19). It is assumed that, by utilizing the hierarchical controlstructure described in Section 1.4, individual systems have been mapped intothe canonical form of (5.8) for the purpose of cooperative control design.

5.3.1 Consensus Problem

The consensus problem is to ensure that x(t) → c1 for some c ∈ ℜ, it isthe final outcome of asymptotic cooperative stability, and hence CooperativeControl 5.11 can directly be applied. As an extension of the simple alignmentproblem in Section 5.1.3, consider that dynamics of three agents are describedby (5.8) with m = 2 and li = 1. Then, the consensus problem is to align bothvelocity components along the two primary axes in the two-dimensional space.Cooperative Control 5.11 is simulated with the choices of wij = 1 and

Kc =

[

0 11 0

]

,

which is irreducible and row-stochastic. Consensus of the velocity componentsis shown in Fig. 5.2, where the initial velocities of the three agents are set tobe [0.2 − 0.3]T , [0.6 0.2]T , and [−0.3 0.5]T , respectively.

5.3 Applications of Cooperative Control 211

0 5 10 15 20 25 30 35 40 45 50Ŧ0.3

Ŧ0.2

Ŧ0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

Time (sec)

x11

x21

x31

(a)

0 5 10 15 20 25 30 35 40 45 50Ŧ0.3

Ŧ0.2

Ŧ0.1

0

0.1

0.2

0.3

0.4

0.5

Time (sec)

x12

x22

x32

(b)

Fig. 5.2. Velocity consensus


5.3.2 Rendezvous Problem and Vector Consensus

Vector consensus is to ensure that x → 1 ⊗ c for some vector c ∈ ℜm. Thesimplest approach to achieve the behavior of vector consensus is to decouplethe output channels of the systems by setting Kc = Im×m in which case,by Theorem 5.6, asymptotic cooperative stability is concluded per outputchannel. For example, let us relax the velocity consensus problem in Section5.3.1 so that velocities vxi

along the x axis reach a consensus for all i whilevyi

along the y axis also have a (different) consensus. This relaxation gives usback the alignment problem (as their orientation angles θi = vxi

/vyihave a

consensus) but without assuming a constant linear absolute velocity. In thiscase, we can keep all the settings as before except for Kc = I2×2, and thesimulation results are shown in Fig. 5.3.

The rendezvous problem is to ensure that all the agents converge to thesame point in either 2-D or 3-D space. This goal can be achieved using eitherconsensus or vector consensus. For instance, we can apply the same coopera-tive controls (used in the aforementioned problems of velocity consensus andalignment) to the position alignment problem, and the simulation results areshown in Fig. 5.4 for initial positions of [8 1]T , [1 6]T , and [4 − 1]T , respec-tively. Specifically, Fig. 5.4(a) shows consensus of all the coordinates underthe single-consensus cooperative control, while Fig. 5.4(b) shows the generalcase of rendezvous under the vector-consensus cooperative control.

5.3.3 Hands-off Operator and Virtual Leader

A cooperative system described by (5.1) is autonomous as a group (or groups).To enable the interaction between a hands-off operator and the cooperativesystem, we can model the operator as a virtual leader. Communication be-tween the virtual leader and the systems in the cooperative system can also beintermittent and local, depending upon availability of the operator and themeans of communication. Thus, we can introduce the following augmentedsensor/communication matrix and its associated time sequence tk : k ∈ ℵas: S(t) = S(tk) for all t ∈ [tk, tk+1) and

S(t) =

⎡

⎢

⎢

⎢

⎣

1 s01(t) · · · s0q(t)s10(t)

...sq0(t)

S(t)

⎤

⎥

⎥

⎥

⎦

∈ ℜ(q+1)×(q+1), (5.22)

where s0i(t) = 1 if xi is known to the operator at time t and s0i(t) = 0 ifotherwise, and si0(t) = 1 if command y0(t) is received by the ith system attime t and si0(t) = 0 if otherwise. Accordingly, cooperative control is modifiedfrom (5.11) to the following version: for i = 1, · · · , q,


0 5 10 15 20 25 30 35 40 45 50Ŧ0.3

Ŧ0.2

Ŧ0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

Time (sec)

x11

x21

x31

(a)

0 5 10 15 20 25 30 35 40 45 50Ŧ0.3

Ŧ0.2

Ŧ0.1

0

0.1

0.2

0.3

0.4

0.5

Time (sec)

x12

x22

x32

(b)

Fig. 5.3. Vector consensus: convergence per channel


Ŧ1 0 1 2 3 4 5 6 7 8 9

Ŧ1

0

1

2

3

4

5

6

7

8

9

agent 1

agent 2

agent 3

(a)

Ŧ1 0 1 2 3 4 5 6 7 8 9

Ŧ1

0

1

2

3

4

5

6

7

8

9

agent 1

agent 2

agent 3

(b)

Fig. 5.4. Rendezvous under cooperative controls


ui(t) =1

q∑

η=0

wiηsiη(t)

q∑

j=0

wijKc[sij(t)(yj − yi)]. (5.23)

Evolution of operator’s command can be modeled as

y0 =

(

1 −q∑

i=1

ǫis0i(t)

)

(u0 − y0) +

q∑

i=1

ǫis0i(t)(yi − y0), (5.24)

where y0(t) ∈ ℜm, ǫi ≥ 0 are constants with∑q

i=1 ǫi ≤ 1, and u0 is a constant(or piecewise-constant) vector specifying the desired cooperative behavior. Itwill be shown in Section 5.8 that, under sequential completeness of S(tk),a single consensus behavior can be achieved under the choices of u0 = cd

01 forsome cd

0 ∈ ℜ and Kc being irreducible, and vector consensus behaviors canbe ensured under u0 = c ∈ ℜm and Kc = I. In general, both u0 and initialcondition of y0(t0) affect the cooperative behavior(s).

Virtual Leader 5.25 generates filtered terms of yi, and hence CooperativeControl 5.23 is a combination of Cooperative Control 5.11 and its filtered ver-sion. To see analogous derivations in standard control theory, consider scalarsystem

z = a + v,

where a is an unknown constant, and its adaptive control with a leakageadaptation law [172] is

v = −z + a, ˙a = −a + z.

Hence, Cooperative Control 5.23 can be viewed as an adaptive cooperativecontrol or integral cooperative control based on (5.11).

In the special case of ǫi = 0 for all i and y0(t0) = u0, Dynamics 5.24 ofthe virtual leader reduce to

y0 = 0, (5.25)

with which the overall Sensing/communication Matrix 5.22 becomes

S(t) =

⎡

⎢

⎢

⎢

⎣

1 0 · · · 0s10(t)

...sq0(t)

S(t)

⎤

⎥

⎥

⎥

⎦

. (5.26)

In this case, we know from Theorem 5.6 that, if S(tk) is sequentially com-pleted, the desired cooperative behavior of yi → u0 is achieved. Without thevirtual leader, the cooperative behavior of yi = yj can be achieved but notnecessarily the desired behavior of yi = u0. While yi → u0 can be cast as astandard control problem (as (yi − u0) → 0), the cooperative control design


Ŧ1 0 1 2 3 4 5 6 7 8 9

Ŧ1

0

1

2

3

4

5

6

7

8

9

agent 1

agent 2

agent 3

Fig. 5.5. Rendezvous to the desired target position

framework makes it possible to handle both intermittent delivery of command(or the reference input) and intermittent feedback.

As an example, reconsider the rendezvous problem in Section 5.3.2 andassume that the desired target position is [4 3]T and that the first systemreceives the target information from the virtual leader once a while. Thecorresponding sensor/communiction matrix is given by (5.26) where S(t) isthe same as before, s10 is a binary value randomly assigned over time, ands20 = s30 = 0. Under the same initial conditions, Cooperative Control 5.23with wij = 0.9 for i = j and wii = 0.1 is simulated together with VirtualLeader 5.25, and the result in Fig. 5.5 demonstrates convergence to the desiredtarget position.

In addition to commanding the desired cooperative behavior and ensuringits convergence, the virtual leader can also play critical roles in enhancing ro-bustness and maintaining network connectivity. These topics will be discussedin Sections 5.7 and 5.4, respectively.

5.3.4 Formation Control

To illustrate the options in designing cooperative formation control, considerwithout loss of any generality the second-order dynamics: i = 1, · · · , q,


ψi = vi,

where ψi ∈ ℜm is the output, and vi is the control. Suppose that the formationis presented by the following vertices in the moving frame Fd(t) that movesalong a contour2 described by time function ψd(t):

pdi (t) =

m∑

j=1

αijej(t), i = 1, · · · , q,

where basis vectors ej are calculated from ψd(t) and according to the discus-sions in Section 3.4.1. As illustrated in Section 1.4, we can define individualbehavior yi = ψi − ψd(t) − pd

i and choose vehicle-level control to be

vi = ui + ψd(t) + pdi − yi = ui − 2ψi − 2ψd(t) + ψd(t) + pd

i − 2pdi , (5.27)

under which the systems are mapped into the canonical form of (5.8) withli = 2 and Kc = 1:

yi = −yi + zi, zi = −zi + yi + ui.

In this case, the cooperative control to achieve and maintain the formation isgiven by (5.11), that is,

ui =1

si1(t) + · · · + siq(t)

q∑

l=1

sil(t)(yl − yi)

= −ψi +1

si1(t) + · · · + siq(t)

q∑

l=1

sil(t)

⎛

⎝ψl +

m∑

j=1

(αij − αlj)ej(t)

⎞

⎠ (5.28)

= −ψi + ψd

i ,

in which ψdi is in the form of (3.99). Nonetheless, compared to the tracking

formation control design in Section 3.4.1, the above cooperative formationcontrol design of vi does not requires time derivatives of ψd

i , and hence it onlyinvolves dynamics of the ith system but not dynamics of any other systems.

As an example, consider the case of three agents whose formation contouris given by ψd(t) = [0.2 0.2]T and whose formation vertices in the movingframe are pd

1 = [ 1√2

1√2]T , pd

2 = [− 1√2

1√2]T , and pd

3 = [ 1√2

− 1√2]T . Simu-

lation is done under the randomly-switched sensing/communication sequencespecified earlier in this section and for the initial positions ψ1(0) = [4 2.5]T ,ψ2(0) = [5 2]T and ψ3(0) = [3 1]T . The simulation results in Fig. 5.6 verifyperformance of cooperative formation control.

2 The formation is supposed to move along the contour or in the phase portrait ofψd(t), but it may not necessarily track ψd(t) closely. If ψd(t) is to be tracked, avirtual leader can be introduced as described in Section 5.3.3.


0 2 4 6 8 10 120

2

4

6

8

10

12

agent 1agent 2agent 3

(a) Phase portrait

0 5 10 15 20 25 30 35 40 45 50Ŧ1.2

Ŧ1

Ŧ0.8

Ŧ0.6

Ŧ0.4

Ŧ0.2

0

0.2

0.4

Time (sec)


(b) Horizontal velocities

Fig. 5.6. Performance under Cooperative Formation Control 5.28


0 5 10 15 20 25 30 35 40 45 50Ŧ1.6

Ŧ1.4

Ŧ1.2

Ŧ1

Ŧ0.8

Ŧ0.6

Ŧ0.4

Ŧ0.2

0

0.2

0.4

Time (sec)


(c) Vertical velocities

Fig. 5.6 (continued)

To reduce further the information required for formation control, considerthe case that pd

i (t) is piecewise-constant3 and that ψd(t) is not available to allthe vehicles. In this case, the formation can be decomposed into geometricalrelationship with respect to any neighboring vehicle l (with the virtual leaderas vehicle 0), ψd(t) is dropped explicitly from the individual behavior (butinstead is injected through the sensing/communication network by the virtualleader), and hence the vehicle-level control in (5.27) reduces to

vi = ui − 2ψi, pdi =

m∑

j=1

αijej(tk) ≈m∑

j=1

α′ilje

′ilj(tk),

where e′ilj are the orthonormal basis vectors of moving frame Fl(t) at thelth vehicle, α′

ilj are the coordinates of the desired location for the ith vehicle

with respect to the desired formation frame Fdl (t). That is, whenever sil(t) =

1, the ith system can observe (or receive information from) the lth systemabout its position ψl(t) (or simply [ψl(t)−ψi(t)] and its current heading basisvectors e′ilj(tk) (which can also be estimated if not directly observed). Hence,Cooperative Formation Control 5.28 can be modified to be

3 This will be relaxed in Section 5.8 by using the design of dynamical cooperativecontrol.


ui =1

si0(t) + si1(t) + · · · + siq(t)

q∑

l=0

sil(t)

⎛

⎝ψl − ψi +m∑

j=1

α′ilj e

′ilj(t)

⎞

⎠ ,

(5.29)which is of form (3.102).

To illustrate the above estimation-based formation control, let us consideragain the case of three entities whose formation motion contour is given byψd(t) = [0.2 0.2]T with formation vertices in the moving frame with respect toψd(t) being [1 0]T , [0 1]T , and [0 − 1]T , respectively. The desired formationcan be decomposed into desired geometric relationship between any pair oftwo entities. For instance, consider the neighboring structure specified by thefollowing augmented sensing/communication matrix

S =

⎡

⎢

⎢

⎣

1 0 0 01 1 0 00 1 1 00 1 0 1

⎤

⎥

⎥

⎦

.

It follows that relative coordinates of α′ilj and current heading basis vectors

of the neighbors are

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

α′101 = 1, α′

102 = 0

e′101 = e′101 =

[

1√2

1√2

]

e′102 = e′102 =

[

− 1√2

1√2

]

,

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

α′211 = −1, α′

212 = 1

e′211 =

⎡

⎣

ψ11√ψ2

11+ψ212

ψ12√ψ2

11+ψ212

⎤

⎦

e′212 =

⎡

⎣

− ψ12√ψ2

11+ψ212

ψ11√ψ2

11+ψ212

⎤

⎦

,

and⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

α′311 = −1, α′

312 = −1

e′311 =

⎡

⎣

ψ11√ψ2

11+ψ212

ψ12√ψ2

11+ψ212

⎤

⎦

e′312 =

⎡

⎣

− ψ12√ψ2

11+ψ212

ψ11√ψ2

11+ψ212

⎤

⎦

,

where ψ11 and ψ12 are the two elements of vector ψ1 (and they can also beestimated online). The corresponding simulation results are provided in Fig.5.7, and they have similar performance as those in Fig. 5.6. For the generalcase of changing topologies, it is straightforward to derive the expressions ofα′

ilj and e′ilj(t) for all the pairs of (i, l), based on which Cooperative FormationControl 5.29 can be implemented.


0 2 4 6 8 10 120

2

4

6

8

10

12


(a) Phase portrait

0 5 10 15 20 25 30 35 40 45 50Ŧ1.5

Ŧ1

Ŧ0.5

0

0.5

1

Time (sec)


(b) Horizontal velocities

Fig. 5.7. Performance under Cooperative Formation Control 5.29


0 5 10 15 20 25 30 35 40 45 50Ŧ1.5

Ŧ1

Ŧ0.5

0

0.5

1

Time (sec)


(c) Vertical velocities


5.3.5 Synchronization and Stabilization of Dynamical Systems

Closely related to vector consensus is the so-called synchronization problem.Consider the following linear systems of identical dynamics: for i = 1, · · · , q,

xi = Axi + Bui, (5.30)

where xi ∈ ℜn and ui ∈ ℜm. Then, the systems are said to be synchronized if‖xi(t) − xj(t)‖ → 0 as t → ∞.

A time-invariant synchronizing control is

ui = K

q∑

j=1

gij(xj − xi),

where K is a gain matrix to be chosen, and G = [gij ] ≥ 0 is the couplingmatrix. It follows that the overall closed-loop system is

x = (Iq×q ⊗ A)x + [(−H + G) ⊗ (BK)]x,

where x =[

xT1 · · · xT

q

]Tand H = diag∑q

j=1 gij. If G is irreducible, ma-trix (−H + G) with zero row sum has its eigenvalues at λi where Re(λ1) ≤Re(λ2) ≤ · · · < λq = 0. Therefore, there exists a matrix T (whose last columnis 1) such that

5.4 Ensuring Network Connectivity 223

T−1(−H + G)T = J,

where J is the Jordan canonical form with diagonal elements of λi. Applyingstate transformation z = (T ⊗ In×n)x yields

z = (Iq×q ⊗ A)z + [J ⊗ (BK)]z.

If K is chosen such that A+λiBK is Hurwitz for i = 1, · · · , (q−1), the abovesystem has the limit that zi → 0 for all i except that zq(t) = eAtzq(0). Hence,

x(t) = Tz(t) → 1⊗ zq(t),

which establishes synchronization. Compared to cooperative control with con-stant network topology, synchronization under constant coupling does notrequire that matrix A be Metzler.

In the case that the synchronization control is chosen to be time-varyingas

ui = K

q∑

j=1

gij(t)(xj − xi),

the above eigenvalue analysis is no longer applicable. Should matrix G(t) beknown, one can attempt to analyze synchronization by applying Lyapunovdirect method and derive the Riccati-equation-like conditions [276, 277] sincefinding the Lyapunov function is generally very difficult. In the general casethat the synchronization control is implemented through a network, matrixG(t) is not known a priori but nonetheless synchronization can be concludedusing Theorem 5.8 by first designing an individual control law for every systemsuch that matrix A becomes Metzler.

If matrix A is also Hurwitz (or simply stabilizable), the above synchroniza-tion problem reduces to the stabilization problem of interconnected systems.General conditions on heterogeneous systems and their connectivity topologycan be derived.

5.4 Ensuring Network Connectivity

The preceding analysis of cooperative systems requires that topological changesof their sensing/communication network is uniformly sequentially completeover time. This requirement of network connectivity is both necessary andsufficient and, based on this requirement, cooperative control is designed forcooperative motion behaviors. Depending upon the network which acquiresand transmits the information, motion behaviors could have significant im-pact on network connectivity since efficiency and accuracy of sensing andcommunication are dependent on distance and direction.

If feedback information needed to implement cooperative control is trans-mitted (for instance, by wireless communication), a proper communication


protocol should be implemented to manage power and bandwidth in such away that network connectivity is ensured over time. Should it become neces-sary, one way to maintain and enhance network connectivity is for the hands-off operator to act as one virtual leader. Typically, the command center hasthe best equipment and communication capacity so that the virtual leader canseek out each and every system in the group and its dynamics in (5.24) playthe role of implicit information relay. Specifically, we know that, by broadcast-ing (i.e., si0(t) = 1) individually to and demanding response (i.e., s0i(t) = 1)from every system repeatedly over time, S(tk) in (5.22) would become se-quentially complete even if S(tk) would never be sequentially complete. Inthe case that the virtual leader only broadcasts to some of the systems butreceives nothing, sensing/communication matrix is prescribed by (5.26), andthen S(tk) is sequentially complete if and only if S(tk) is sequentiallycomplete. This also implies that, if any of the systems acts maliciously asa fake leader (i.e., by only broadcasting but not reacting), it can cause thewhole group of systems to oscillate unless corrective measures can be taken.

If all the feedback information needed to implement cooperative controlis acquired by the sensors onboard any of the individual vehicles, cooperativecontrol must be designed to generate motions that enable the sensors to con-tinue data acquisition and hence maintain network connectivity. In this case,motion and network connectivity cannot be separated, and network connec-tivity needs to be maintained by properly choosing weights in cooperativecontrol law, that is, wij(t) > 0 in (5.11). To illustrate the process, considerthe one-dimensional six-agent rendezvous problem in which all the particleshave the same sensing radius of Rs > 0 and their dynamics are given by

zi(k + 1) = ui(k), i = 1, · · · , 6,

where cooperative control u(k) is chosen to be in the form of (5.11) as

ui(k) =1

∑6l=1 wil(k)sil(k)

6∑

j=1

wij(k)sij(k)zj(k). (5.31)

Assume that z1(0) = z2(0) = 0, z3(0) = Rs, z4(0) = 2Rs, and z5(0) = z6(0) =3Rs. Clearly, the sensing network is connected at k = 0 but could becomedisconnected at k = 1 and beyond. Indeed, once the sensing network becomesdisconnected, it may never recover in this case that only passive sensors offixed range are employed. Hence, maintaining network connectivity is the keyto ensure cooperative behavior of rendezvous. In what follows, two designcases of Cooperative Control 5.31 are elaborated and compared.

The first case is that all the weights are set to be identical and hencewij(k) = 1. It follows that

u1(0) = u2(0) =1

3Rs, u3(0) =

3

4Rs, u4(0) = 2

1

4Rs, u5(0) = u6(0) = 2

2

3Rs,

which yields

5.4 Ensuring Network Connectivity 225

z1(1) = z2(1) =1

3Rs, z3(1) =

3

4Rs, z4(1) = 2

1

4Rs, z5(1) = z6(1) = 2

2

3Rs.

Hence, at k = 1, the sensing network is no longer connected since z4(1) −z3(1) = 1.5Rs. As k increases, the network stays disconnected and, for allk ≥ 2,

z1(k) = z2(k) = z3(k) =17

36Rs, z4(k) = z5(k) = z6(k) = 2

19

36Rs.

Intuitively, breakup of the sensing network is primarily due to the movementof agents 3 and 4. At k = 0, each of these two agents has two other agentsbarely within and at both ends of their sending ranges and, since either knowshow its neighbors will move, both of agents 3 and 4 should not move at k = 1as any movement could break up network connectivity.

The second case is to choose the weights according to the so-called circum-center algorithm: for each agent, find its neighbors in the sensing range, calcu-late the circumcenter location of its neighbors (i.e., the center of the smallestinterval that includes all its neighbors), and move itself to the circumcenter.Obviously, the algorithm reflects the intuition aforementioned. For the six-agent rendezvous problem, let the circumcenter location for the ith agent atstep k be denoted by ξi(k). Applying the algorithm, we know that

ξ1(0) = ξ2(0) = Rs, ξ3(0) = Rs, ξ4(0) = 2Rs, ξ5(0) = ξ6(0) = 2Rs,

andξ1(1) = ξ2(1) = ξ3(1) = ξ4(1) = ξ5(1) = ξ6(1) = 1.5Rs.

It follows that, at k = 0, the corresponding non-zero weights for implementingCooperative Control 5.31 are

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎩

w11(0) = w12(0) = ǫ, w13(0) = 1 − 2ǫ;w21(0) = w22(0) = ǫ, w23(0) = 1 − 2ǫ;w31(0) = w32(0) = ǫ, w33(0) = 1 − 3ǫ, w34(0) = ǫ;w43(0) = ǫ, w44(0) = 1 − 3ǫ, w45(0) = w46(0) = ǫ;w54(0) = 1 − 2ǫ, w55(0) = w56(0) = ǫ;w64(0) = 1 − 2ǫ, w65(0) = w66(0) = ǫ;

andwij(k) = 1, k ≥ 2,

where ǫ > 0 is a small constant. It can be easily verified that, as long asǫ is small, Cooperative Control 5.31 can maintain network connectivity. Asǫ approaches zero, Cooperative Control 5.31 becomes the circumcenter algo-rithm which provides the best convergence speed and is, strictly speaking, anon-linear cooperative control. Nonetheless, the circumcenter algorithm pro-vides useful guidance in selecting the weights of Control 5.31 for the systemsequipped with a bidirectional sensing/communication network. Non-linear co-operative systems will be studied in Chapter 6, and the circumcenter algorithmwill be analyzed further in Section 6.4.


The circumcenter algorithm or its linear approximation for Control 5.31can be extended to arbitrary dimensions if each agent has the knowledge ofa common frame and if its sensing range is rectangular (or contains a non-trivial rectangular sub-set). However, typical sensing ranges in the 2-D and3-D spaces are a limited sector and a limited cone, respectively. In those cases,maintaining network connectivity is not difficult if a simple chain of leader-follower motion pattern can be imposed.

5.5 Average System and Its Properties

It is shown by Theorems 5.6 and 5.13 as well as by the discussions in Section5.2.3 that asymptotic cooperative stability of System 5.1 is determined bycumulative information exchange over time. Another way to quantify explicitlycumulative effects of topological changes is to develop the so-called averagesystem (introduced in Section 2.4.2) for Cooperative System 5.1. Specifically,Lemma 5.14 provides the properties on an average cooperative system, and ittogether with Theorem 5.6 yields Theorem 5.15.

Lemma 5.14. Exponential functions of non-negative matrices have the fol-lowing properties:

(a) Defineξ(α) = [eAαeBτ ]

αα+τ ,

where α ≥ 0 is the scalar argument, A, B ∈ ℜn×n+ are constant matrices,

and τ > 0 is a constant. Then, [ξ(α)− I] is a non-negative matrix for anysufficiently small values of α > 0.

(b) Consider a pair of non-negative and row-stochastic matrices Di and Dj.If B[Di + Dj] = B[(Di + Dj)

2] where B(·) is defined in (4.22), there is anon-negative and row-stochastic matrix Da,ij such that, for any τi, τj > 0

eDa,ij(τi+τj) = eDiτieDjτj .

(c) Given any row-stochastic matrix Da = 1cT with c ∈ ℜn+, Da = D2

a and

e−(I+Da)t → Da as t → ∞.

Proof:

(a) It follows that ξ(0) = I and that, since ln C is well defined for any invert-ible matrix C,

dξ(α)

dα=

∂eAαeBτ

∂α· [eAαeBτ ]

αα+τ

−1

+∂ α

α+τ

∂α· ln[eAαeBτ ] · [eAαeBτ ]

αα+τ

=

A +τ

(α + τ)2ln[eAαeBτ ]

ξ(α).

5.5 Average System and Its Properties 227

Therefore, the Taylor series expansion of ξ(α) for sufficiently small andpositive values of α is

ξ(α) = I + [A + B]α + h.o.t., (5.32)

where h.o.t. stands for high-order terms. By repeatedly taking higher-order derivatives, one can show that their values at α = 0 introducenew non-negative matrix product terms while, except for scalar multipli-ers, non-positive matrix terms are the same as those non-negative matrixterms already appeared in the lower-order derivatives. For example, itfollows that

d2ξ(α)

dα2=

A +τ

(α + τ)2ln[eAαeBτ ]

2

ξ(α)

+

− 2τ

(α + τ)3ln[eAαeBτ ] +

τ

(α + τ)2A

ξ(α),

which implies that the second-order term in Taylor Expansion 5.32 is

d2ξ(α)

dα2

∣

∣

∣

∣

α=0

α2 =

[A + B]2 − 2

τB +

1

τA

α2.

In the above expression, non-negative terms in terms of products ABand BA appear, the only non-positive term is −2α2B/τ and this term isdominated by αB which is one of the non-negative terms already appearedin the first-order Taylor expansion. By induction, one can claim that, inthe Taylor series expansion in (5.32), all non-positive terms appeared inh.o.t. are dominated by the non-negative terms, which proves (a).

(b) It follows from Baker-Campbell-Hausdorff formula [213] that

Da,ij(τi + τj) = Diτi + Djτj +1

2adDiτi

(Djτj) +1

12adDiτi

(adDiτiDjτj)

+adDjτj(adDiτi

Djτj)

+ · · · ,

where adAB = AB − BA is the Lie bracket. Since Di and Dj are row-stochastic, [adDiτi

(Djτj)]1 = 0 and so do the higher-order Lie brackets.Hence, matrix Da,ij is row-stochastic. To show that Da,ij is non-negative,let αi = τi/β for any chosen positive integer β > 1 and note that

eDa,ij(τi+τj) =

[

β∏

k=1

eDiαi

]

eDjτj =

[

β−1∏

k=1

eDiαi

]

eH(αi+τj),

where eH(αi+τj) = eDiαieDjτj . On the other hand, we know from Taylor

expansion

[

eH(αi+τj)]

αiαi+τi

= eHαi = I + Hαi +1

2H2α2

i + h.o.t.


and from (a) that, if B(H) = B(H2), H is non-negative for all sufficientlysmall αi > 0 and by induction matrix Da,ij is non-negative. Comparingthe above expansion with (5.32) yields the conclusion of (b).

(c) Since Da = 1cT , D2a = Da and hence

limt→∞

e−(I+Da)t = limt→∞

e−t∞∑

j=0

1

j!Dj

atj = lim

t→∞e−t

[

I + Da(et − 1)]

= Da,

which completes the proof.

Theorem 5.15. The following statements are equivalent:

(a) System 5.1 is uniformly asymptotically cooperatively stable.(b) For all η ∈ ℵ, composite sensing/communication matrices SΛ(η) defined

in (5.17) are lower triangularly complete, and the lengths (tkη− tkη−1) of

their composite time intervals are uniformly bounded.(c) For any k ∈ ℵ, there exists κ > k such that System 5.1 has the following

average system:

xa = [−I + Da(κ : k)]xa, t ∈ [tk, tκ), (5.33)

where terminal conditions are xa(tκ) = x(tκ) and xa(tk) = x(tk), andDa(κ : k) is a non-negative and row-stochastic matrix satisfying

e[−I+Da(κ:k)](tκ−tk) = e[−I+D(tκ−1)](tκ−tκ−1)e[−I+D(tkκ−2)](tκ−1−tκ−2)

· · · e[−I+D(tk)](tk+1−tk). (5.34)

Statement (b) of Lemma 5.14 spells out the condition under which an av-erage system can be found for System 5.1 over any two consecutive intervals.In general, the average system always exists over a composite interval of suf-ficient length, which is stated by Theorem 5.15 and is due to (c) of Lemma5.14. The following example illustrates some of the details.

Example 5.16. Control 5.11 with m = li = wij = 1 renders the follow-ing matrices for the Sensing/communication Sequence 5.19 in Example 5.9:D(tk) = D1 for k = 4η + 3, D(tk) = D2 for k = 4η + 2, D(tk) = D3 fork = 4η + 1, and D(tk) = D4 for k = 4η, where η ∈ ℵ,

D1 =

⎡

⎣

1 0 00.5 0.5 00 0 1

⎤

⎦ , D2 =

⎡

⎣

0.5 0.5 00 1 00 0 1

⎤

⎦ , D3 =

⎡

⎣

1 0 00 1 0

0.5 0 0.5

⎤

⎦ , D4 =

⎡

⎣

1 0 00 1 00 0 1

⎤

⎦ .

Assume that tk − tk−1 = 1 for all k. It follows that

Da(3 : 2) =

⎡

⎣

0.6887 0.3113 00.1888 0.8112 0

0 0 1.0000

⎤

⎦

5.6 Cooperative Control Lyapunov Function 229

is the average matrix of D2 and D1 as

e2[−I+Da(3:2)] = e−I+D1e−I+D2 =

⎡

⎣

0.6065 0.3935 00.2387 0.7613 0

0 0 1

⎤

⎦ .

However, non-negative matrix Da(3 : 0) does not exists to be the averagematrix of D4 up to D1 since B[(Da(3 : 2) + D3)

2] = B[Da(3 : 2) + D3], while

e−I+D1e−I+D2e−I+D3e−I+D4 = e−I+Da(3:2)e−I+D3

=

⎡

⎣

0.6065 0.3935 00.2387 0.7613 00.3935 0 0.6065

⎤

⎦ .

Nonetheless, for any α ∈ ℵ, matrix

Da((4α + 4 ∗ 32) : (4α)) =

⎡

⎣

0.3775 0.6225 00.3775 0.6225 0

0.5 0 0.5

⎤

⎦

is the matrix of the average system over interval [4α, 4α + 128) (or extendedinterval [4α, t) for any t > 4α + 128) as

e32[−I+Da((4α+4∗32):(4α))] =[

e−I+D1e−I+D2e−I+D3e−I+D4]32

=

⎡

⎣

0.3775 0.6225 00.3775 0.6225 00.3775 0.6225 0

⎤

⎦ .

Clearly, matrix Da(128 : 0) corresponds SΛ(η) defined by (5.17) with tkη= kη

and kη = 128η, where η ∈ ℵ.

5.6 Cooperative Control Lyapunov Function

The previous analysis of cooperative stability is based on the solution ofpiecewise-constant linear networked systems and convergence of the resultingmultiplicative matrix sequence. However, an exact solution cannot be analyti-cally found for networked systems of non-linear dynamics. In order to analyzeand design non-linear networked systems, we need to apply non-linear toolsthat do not require explicit solution. One prominent candidate is the Lya-punov direct method which is universally applicable to linear and non-linearsystems. In this section, Lyapunov function is sought for Cooperative System5.8, and the results build the foundation for non-linear analysis in Chapter 6.

Should System 5.8 be uniformly asymptotically cooperatively stable, thecorresponding Relative-error System 5.4 must be uniformly asymptoticallystable, and hence existence of a Lyapunov function is guaranteed by Lyapunovconverse theorem. The subsequent discussion provides a Lyapunov functionthat is in the form for concluding cooperative stability and is also strictlydecreasing over time.


Definition 5.17. Vc(x(t), t) is said to be a cooperative control Lyapunovfunction (CCLF) for system x = F(x, t) if Vc(x, t) is uniformly boundedwith respect to t, if Vc(x, t) = V (eci

, t) is positive definite with respect toeci

for some or all i ∈ 1, · · · , n where eciis defined in (5.3), and if

Vc(x(t′), t′) > Vc(x(t), t) along the solution of the system and for all t > t′ ≥ t0unless x(t′) = c1.

In the sequel, cooperative control Lyapunov function is found first fora linear cooperative system of fixed topology and then for one of varyingtopologies.

5.6.1 Fixed Topology

Consider the following time-invariant system

x = −(I − D)x= −Ax, (5.35)

where x ∈ ℜn, D ≥ 0 is row-stochastic and constant, and hence A is asingular M-matrix. It has been shown in Section 4.3.5 that, if A is irreducible,a Lyapunov function of the following simple form can be found:

V (x) = xT Px =

n∑

i=1

λix2i , P = diagλ1, · · · , λn, (5.36)

where λ =[

λ1 · · · λn

]Tis the unity left eigenvector of D and associated with

eigenvalue ρ(D) = 1, that is,

DT λ = λ, λT 1 = 1. (5.37)

Nonetheless, Lyapunov Function 5.36 is not a control Lyapunov function forSystem 5.35 because V is positive definite with respect to x but V is only neg-ative semi-definite and because, as shown in Example 4.35, such a Lyapunovfunction does not exist if A is reducible. Hence, we need to find a cooperativecontrol Lyapunov function first for any singular irreducible M-matrix and thenfor a singular reducible M-matrix. The following lemma and theorem providea constructive way of finding such a cooperative control Lyapunov function.

Lemma 5.18. For any constants λ1 up to λn and along the trajectory ofSystem 5.35, the time derivative of

Vc =n∑

i=1

n∑

j=1

λiλj(xj − xi)2 =

n∑

i=1

λieTci

GTi PGieci

, (5.38)

can be expressed as

Vc = −2

n∑

i=1

λieTci

Qcieci

, (5.39)


where P is that in (5.36), eciis defined by (5.3), Gi ∈ ℜn×(n−1) is the resulting

matrix after eliminating the ith column from In×n, and

Qci

= GT

i [P (I − D) + (I − D)T P ]Gi

= (GTi PGi)(I − GT

i DGi) + (I − GTi DGi)

T (GTi PGi). (5.40)

Proof: To verify the expression of Vc for function Vc in (5.38), consider firstthe time derivative of term (xi − xj)

2. It follows that, for any pair of i, j ∈1, · · · , n,

1

2

d(xi − xj)2

dt= (xi − xj)[xi − xj ]

= (xi − xj)

[

−xi +n∑

k=1

dikxk + xj −n∑

k=1

djkxk

]

= (xi − xj)

⎧

⎨

⎩

⎡

⎣−(1 − dii)xi +

n∑

k =j, k =i, k=1

dikxk + (1 − djj)xj

−n∑

k =i, k =i, k=1

djkxk

⎤

⎦+ dijxj − djixi

⎫

⎬

⎭

. (5.41)

Since D is row-stochastic, we have

⎧

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎩

dij = 1 − dii −n∑

k =i, k =j, k=1

dik

dji = 1 − djj −n∑

k =j, k =j, k=1

djk

.

Substituting the above two expressions of dij and dji into (5.41) yields

1

2

d(xi − xj)2

dt

= (xi − xj)

⎡

⎣−(2 − dii − djj)xi +n∑

k =j, k =i, k=1

dik(xk − xj) + (2 − dii − djj)xj

+n∑

k =i, k =j, k=1

djk(xi − xk)

⎤

⎦

= −(1 − dii)(xi − xj)2 − (1 − djj)(xj − xi)

2 +n∑

k =i, k=1

dik(xi − xj)(xk − xj)


+n∑

k =j, k=1

djk(xi − xj)(xi − xk). (5.42)

It follows from (5.38) and (5.42) that

1

2Vc = −

n∑

i=1

n∑

j=1

λiλj(1 − dii)(xi − xj)2 −

n∑

i=1

n∑

j=1

λiλj(1 − djj)(xj − xi)2

+

n∑

i=1

n∑

j=1

⎡

⎣

n∑

k =i, k=1

λiλjdik(xi − xj)(xk − xj)

⎤

⎦

+

n∑

i=1

n∑

j=1

⎡

⎣

n∑

k =j, k=1

λiλjdjk(xi − xj)(xi − xk)

⎤

⎦

= −2

n∑

i=1

n∑

j=1

λiλj(1 − dii)(xi − xj)2

+2

n∑

i=1

n∑

j=1

⎡

⎣

n∑

k =j, k=1

λiλjdjk(xi − xj)(xi − xk)

⎤

⎦ . (5.43)

On the other hand, it follows from (5.40) and P that Qciis the resulting

square matrix after removing the ith row and column from matrix

Q = P (I − D) + (I − D)T P

=

⎡

⎢

⎢

⎢

⎣

2λ1(1 − d11) −λ1d12 − λ2d21 · · · −λ1d1n − λndn1

−λ2d21 − λ1d12 2λ2(1 − d22) · · · −λ2d2n − λndn2

......

......

−λndn1 − λ1d1n −λndn2 − λ2d2n · · · 2λn(1 − dnn)

⎤

⎥

⎥

⎥

⎦

.

Hence, we have

−n∑

i=1

λixTci

Qcixci

= −n∑

i=1

λi

⎧

⎨

⎩

n∑

j =i, j=1

2λj(1 − djj)(xj − xi)2

−n∑

j =i, j=1

⎡

⎣

n∑

k =i, k =j, k=1

(λjdjk + λkdkj)(xj − xi)(xk − xi)

⎤

⎦

⎫

⎬

⎭

= −n∑

i=1

n∑

j=1

2λiλj(1 − djj)(xj − xi)2


+n∑

i=1

n∑

j=1

⎡

⎣

n∑

k =j, k=1

λi(λjdjk + λkdkj)(xj − xi)(xk − xi)

⎤

⎦

= −2n∑

i=1

n∑

j=1


+n∑

i=1

n∑

j=1

⎡

⎣

n∑

k =j, k=1

λiλjdjk(xj − xi)(xk − xi)

⎤

⎦

+n∑

i=1

n∑

j=1

⎡

⎣

n∑

k =j, k=1

λiλkdkj(xj − xi)(xk − xi)

⎤

⎦

= −2n∑

i=1

n∑

j=1


+2n∑

i=1

n∑

j=1

⎡

⎣

n∑

k =j, k=1

λiλjdjk(xj − xi)(xk − xi)

⎤

⎦ ,

which is the same as (5.43). Hence, Matrix Expression 5.39 is established.

Theorem 5.19. If matrix D is irreducible, function Vc in (5.38) is a cooper-ative control Lyapunov function for System 5.35 by choosing λ according to(5.37).

Proof: It follows from Theorem 4.8 that λ > 0 and hence Vc is positivedefinite with respect to eci

. To show Qciis positive definite and hence Vc is

negative definite, recall that Qci= GT

i QGi, where Q = P (I−D)+(I−D)T P .According to (d) of Theorem 4.27, Q is positive semi-definite and hence so isQci

. Furthermore, since D is irreducible, PD and in turn (PD+DT P ) as wellas 2(maxλiI−P )+(PD+DTP ) are irreducible and, by (a) of Theorem 4.27,matrix Q = 2P − (PD+DT P ) = 2 maxλiI− [2(maxλiI−P )+(PD+DTP )]is of rank n − 1. Thus, xT Qx > 0 for all x = c1 with c ∈ ℜ, and 1T Qx =xT Q1 = 0. In particular, x = xi1 + Gieci

, x = c1 if and only if eci= 0, and

hencexT Qx = eT

ciGT

i QGieci

is positive definite with respect to eci.

The above proof also suggests a close relationship between Lyapunov Func-tion 5.36 and Cooperative Control Lyapunov Function 5.38. Specifically, itfollows that

V = − 1

n

n∑

i=1

xT [P (I − D) + (I − D)T P ]x


= − 1

n

n∑

i=1

[(Gieci+ xi1)T [P (I − D) + (I − D)T P ](Gieci

+ xi1)]

= − 1

n

n∑

i=1

eTci

Qcieci

,

which is negative definite with respect to eci. On the other hand, it follows

that

V = xT Px

= (Gieci+ xi1)T P (Gieci

+ xi1)

= eTci

(GTi PGi)eci

+ 2xi1T PGieci

+ x2i

= eTci

(GTi PGi)eci

+ 2xiO − x2i ,

where O is the stationary center of mass studied in (5.2). Clearly, V is notpositive definite with respect to eci

, but function Vc is and it can be rewrittenas

Vc=

n∑

i=1

λieTci

(GTi PGi)eci

=n∑

i=1

λiV − 2On∑

i=1

λixi +n∑

i=1

λix2i

= 2V − 2O2.

Taking the time derivative on both sides of the above expression, invoking theexpression of V and recalling the property of O yield

Vc = 2V = − 2

n

n∑

i=1

eTci

Qcieci

,

which is an alternative expression equivalent to (5.39).If matrix A is reducible, Cooperative Control Lyapunov Function 5.38

can still be found but Lyapunov function in the form of (5.36) may not exist.Theorem 5.20 provides a general solution, while Corollary 5.21 gives a simplerversion (and its proof is a special case of Vc = V2 where V2 is defined by (5.47)).

Theorem 5.20. Consider a non-negative, reducible and row-stochastic matrixD, and let F\ be its lower-triangular canonical form in (4.1). Then, System5.35 has a cooperative control Lyapunov function if and only if F\ is lowertriangularly complete, and the Lyapunov function is of form

Vc =

n∑

i=1

n∑

k=1

pμ,k(xμ − xk)2, (5.44)


where pμ,k are positive constants. If F\ is not lower triangularly complete,

pμ,k > 0 exist such that Vc ≤ 0 and Vc is negative definite with respect to(xμ − xk) for all μ, k corresponding to the same lower triangularly completeblock.

Proof: To show sufficiency, assume without loss of any generality that p = 2(if otherwise, the subsequent proof can be extended to p > 2 by induction).Since p = 2, we have that

x =(

−I + F\

)

x, x =

[

yz

]

, F\ =

[

F11 0F21 F22

]

, (5.45)

where F11 ∈ ℜr1×r1 and F22 ∈ ℜr2×r2 are irreducible, and r1 + r2 = n.It follows from Theorem 5.19 that there exist positive constants λ1 up to

λr1 such that, along the trajectory of y = (−I + F11)y, the time derivative of

V1 =

r1∑

i=1

r1∑

j=1

λiλj(yj − yi)2 (5.46)

is

V1 = −2

r1∑

i=1

λiyTci

Q1,ciyci

,

where yci= GT

1,i(y − yi1), G1,i is the same as Gi in Lemma 5.18 except forits dimension, and Q1,ci

are all positive definite. On the other hand, let

xci=

[

yci

zci

]

,

⎧

⎪

⎪

⎨

⎪

⎪

⎩

yci= GT

1,i(y − yi1), zci= z − yi1,

if i ∈ 1, · · · , r1;yci

= y − zi−r11, zci= GT

2,(i−r1)(z − zi−r11),

if i ∈ r1 + 1, · · · , r1 + r2;

where G2,j is the same as Gj in Lemma 5.18 except for its dimension. Then,for any i ∈ 1, · · · , r1, dynamics of z can be expressed as

zci= [F21Gi + 1eT

i (−I + F11)Gi]yci+ (−I + F22)zci

,

where ei ∈ ℜr1 is the vector with zero as its elements except that its ithelement is 1. Since F21 = 0, it follows from Lemma 4.38 and Theorem 4.31 thatthere exists a positive definite and diagonal matrix P2 of diagonal elementsλr1+1 up to λn such that

Q2= P2(I − F22) + (I − F22)

T P2

is positive definite. Now, consider Lyapunov function

V2 = α1V1 +

r1∑

i=1

zTci

P2zci, (5.47)


where α1 > 0 is a positive constant to be determined. It follows that

V2 = −2α1

r1∑

i=1

λiyTci

Q1,ciyci

−r1∑

j=1

zTci

Q2zci

+2

r1∑

i=1

zTci

P2[F21Gi + 1eTi (−I + F11)Gi]yci

,

which is negative definite by simply choosing

α1 > max1≤i≤r1

λ2max(P2)‖F21Gi + 1eT

i (−I + F11)Gi‖2

2λiλmin(Q1,ci)λmin(Q2)

.

Choose the following candidate of cooperative control Lyapunov function:

Vc = V3 + α2V1 + α3V2,

where α2 > 0 and α3 ≥ 0 are to be chosen shortly, V1 is defined by (5.46), V2

is given by (5.47), and

V3 =

n∑

i=1

n∑

j=1

λiλj(xj − xi)2, λi ≥ 0.

It follows from Lemma 5.18 that

V3 = −2

n∑

i=1

λixTci

Q′ci

xci,

where xci= GT

i (x − xi1), P = diagλ1, · · · , λn, and Q′ci

= GTi [P (I − F\) +

(I − F\)T P ]Gi. Therefore, we know that

Vc = −2

n∑

i=1

λixTci

Q′ci

xci+ α3V2,

where Q2,ci= GT

2,(i−r1)Q2G2,(i−r1) for r1 < i ≤ r1 + r2,

Q′ci

=

⎧

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎩

[

(1 + α2)Q1,ci−GT

1,iF12

−FT12G1,i Q2

]

if 1 ≤ i ≤ r1,

[

Q1 −F12G2,(i−r1)

−GT2,(i−r1)

FT12 Q2,ci

]

if r1 + 1 ≤ i ≤ r1 + r2.

Since Q1,ciand Q2 are positive definite, matrices Q′

ciwith 1 ≤ i ≤ r1 can

be made positive definite by simply increasing α2. Since Q1 is only positivesemi-definite, matrices Q′

ciwith r1 + 1 ≤ i ≤ r1 + r2 cannot be guaranteed to

be positive definite. Nonetheless, all the indefinite terms are from the product


of 2zTci

GT2,(i−r1)F

T12(y − zi1), and they all are of form (xj − xi)(xk − xi) for

r1 + 1 ≤ i, j ≤ r1 + r2 and 1 ≤ k ≤ r1. Noting that matrices Q2,ciare positive

definite and they are associated with (xj − xi) and that V2 has been shownabove to be negative definite with respect to (xk − xi), we can conclude thatVc can be made negative definite by increasing both α2 and α3.

To show necessity, note that matrix F\ must be block diagonal if F\ isnot lower triangularly complete and that both sub-matrices [−I + F11] and[−I + F22] have one of their eigenvalues at 0. As a result, their block-systemsare decoupled and hence they together cannot be asymptotically cooperativelystable. In other words, Vc cannot be negative definite, which shows necessity.

In the case that F21 = 0, it follows from the expression of matrix Q′ci

that it is positive semi-definite. Thus, Vc is negative semi-definite by settingVc = V3.

Corollary 5.21. Consider a non-negative, reducible and row-stochastic ma-trix D, and let F\ be its lower-triangular canonical form in (4.1). Then,

Vc(x) =

n∑

i=1

∑

k∈Ω1

pμ,k(xμ − xk)2 (5.48)

is a cooperative control Lyapunov function for System 5.35 if and only if F\ islower triangularly complete, where pμ,k are positive constants, and Ω1 is theset of the indices corresponding to those indices of F11 in (4.1).

5.6.2 Varying Topologies

In the presence of topology changes, the cooperative control Lyapunov func-tion can be sought as before for Piecewise-Constant System 5.1 within eachand every interval. Over time, a cooperative control Lyapunov function canalso be found for its average system. Combining Theorems 5.15, 5.19 and 5.20as well as Lemmas 5.14 and 5.22 yields the following theorem.

Lemma 5.22. Cooperative control Lyapunov function for system x = [−I +1cT ]x with c ∈ ℜn

+ can be chosen to be Vc in (5.44) in which pμ,k > 0 arearbitrary.

Proof: It follows from x = [−I + 1cT ]x that xi − xj = −(xi − xj), whichcompletes the proof.

Theorem 5.23. System 5.1 is uniformly asymptotically cooperatively stableif and only if its average system over any sufficiently-long time interval has aCooperative Control Lyapunov Function in the form of (5.44).

Example 5.24. Consider the following sensing/communication matrices Si andtheir corresponding cooperative matrices Di: over two consecutive intervals[tk, tk+1] and [tk+1, tk+2),


S(tk) =

⎡

⎣

1 0 01 1 00 0 1

⎤

⎦ , D(tk) =

⎡

⎣

1 0 00.5 0.5 00 0 1

⎤

⎦ ,

S(tk+1) =

⎡

⎣

1 0 00 1 01 0 1

⎤

⎦ , and D(tk+1) =

⎡

⎣

1 0 00 1 0

0.5 0 0.5

⎤

⎦ .

Assume that tk+2 − tk+1 = tk+1 − tk for simplicity. It follows from D(tk) andD(tk+1) being commutative that the average system over interval [tk, tk+2) isdefined by matrix

Da((k + 2) : k) =

⎡

⎣

1 0 00.25 0.75 00.25 0 0.75

⎤

⎦ .

Since Da is lower triangularly complete, Cooperative Control Lyapunov Func-tion 5.44 exists for the average system, and it can be found by either followingthe proof of Theorem 5.20 or through simple computation shown below.

It follows from

Vc = p1(x1 − x2)2 + p2(x1 − x3)

2 + p3(x2 − x3)2

that, along the trajectory of xa = (−I + Da)xa,

Vc = −0.25p1(x1 − x2)2 − 0.25p2(x1 − x3)

2 − 0.25p3(x2 − x3)2.

Hence, all choices of Vc with pi > 0 are cooperative control Lyapunov functionsover the composite interval [tk, tk+2).

It should be noted that, in the presence of topological changes, common(cooperative control) Lyapunov function may not exist. As such, the averagesystem and its (cooperative control) Lyapunov function are used to describecumulative characteristics of a linear (cooperative) system over time.

5.7 Robustness of Cooperative Systems

To analyze robustness, consider first the ideal cooperative system

z(k + 1) = P (k)z(k), (5.49)

which represents either the continuous-time version of (5.5) or the discrete-time version of (5.20). It is assumed that, if all the feedbacks are accurate,System 5.49 is (uniformly) asymptotically cooperatively stable; that is, Pk isnon-negative and row-stochastic, and the system has the limit that, for someconstant vector c ∈ ℜn,

limk→∞

Φ(k + 1, 0) = 1cT

5.7 Robustness of Cooperative Systems 239

where

Φ(k + 1, j) =

k∏

η=j

P (η) = P (k)P (k − 1) · · ·P (j). (5.50)

In practice, the information available for feedback is often inaccurate due tomeasurement errors, communication noises, and disturbances. Unless relativemeasurements are sufficient, a universal coordinate is needed to interpret ab-solute measurements, which may also induce errors for different entities. Toaccount for these errors, Cooperative Model 5.49 has to be changed into

z(k + 1) = P (k)z(k) +

⎡

⎢

⎣

∑nj=1 P1j(k)w′

1j(k)...

∑nj=1 Pnj(k)w′

nj(k)

⎤

⎥

⎦, (5.51)

where w′ij(k) is the feedback error of the ith entity about the jth entity at

time k. Since matrix P (k) is row-stochastic, (5.51) can be simplified as

z(k + 1) = P (k)z(k) +

⎡

⎢

⎣

w1(k)...

wn(k)

⎤

⎥

⎦

= P (k)z(k) + w(k), (5.52)

where wi(k) is the lumped error at the ith entity and has the property that

|wi(k)| ≤ maxj

|w′ij(k)|. (5.53)

The solution to System 5.52 is

z(k + 1) = Φ(k + 1, 0)z(0) +

k−1∑

i=0

Φ(k + 1, i + 1)w(i) + w(k). (5.54)

Property 5.53 implies that, no matter how many entities a cooperative systemcontains, the lumped error at one of the entities is no worse than the worst ofits individual errors of acquiring information only from one of its neighboringentities. While Property 5.53 is nice, the presence of w(k) may degrade oreven destroy cooperativeness (cooperative stability). In what follows, impactsof three different types of errors are investigated, and strategies are developedto enhance robustness of cooperative systems.

Sector-Bounded Errors

The first case is that lumped error w(k) satisfies the linear growth condition,that is, w(k) is proportional to z(k) as

w(k) = W (k)z(k), (5.55)


where W (k) is a matrix of bounded values. Substituting (5.55) into (5.52)yields

z(k + 1) = [P (k) + W (k)]z(k)= Pd(k)y(k). (5.56)

In general, the presence of W (k) prevents matrix Pd(k) from keeping theproperties of P (k). That is, matrix Pd(k) may no longer be row-stochastic,it may not even be non-negative, and its structural property may be altered.The following example shows that, in either of these three cases, cooperativestability becomes void.


W (k) =

⎡

⎣

w11 0 0w21 w22 0w31 w32 w33

⎤

⎦ , and P (k) =

⎡

⎣

1 0 00.5 0.5 00 0.1 0.9

⎤

⎦ .

If wij = 0.01 in the above matrix W (k), matrix Pd(k) is non-negative butnot row-stochastic. It is straightforward to verify that zi(k) → ∞ for all i and|zi(k) − zj(k)| → ∞ for all i = j as k → ∞.

If wij = 0 except w31 = −0.01 in the above matrix W (k), matrix Pd(k) isnot non-negative or row-stochastic. It follows from

limk→∞

[P (k) + W (k)]k =

⎡

⎣

1 0 01 0 0

0.9 0 0

⎤

⎦

that System 5.56 is not asymptotically cooperatively stable.If wij = 0 except w32 = −0.1 and w33 = 0.1 in the above matrix W (k),

matrix Pd(k) is still non-negative and row-stochastic. However, unlike P (k),matrix Pd(k) is no longer lower triangularly complete. Hence,

limk→∞

[P (k) + W (k)]k =

⎡

⎣

1 0 01 0 00 0 1

⎤

⎦ ,

which implies the loss of asymptotic cooperative stability for System 5.56.

Example 5.25 shows that, in the presence of sector-bounded errors, asymp-totic cooperative stability cannot be maintained unless matrix W (k) has theproperties that W (k)1 = 0, |wij(k)| = 0 if pij(k) = 0, and |wij(k)| < pij(k)if pij(k) > 0. These properties ensure that matrix Pd(k) is non-negative, row-stochastic and structurally the same as matrix P (k). However, for unknownerror matrix W (k), these properties are too restrictive to be satisfied.

Uniformly Bounded Errors

Consider the case that lumped error w(k) is uniformly bounded as |wi(k)| ≤cw for all k and for some constant cw. In this case, approximate cooperative-ness can be maintained but Lyapunov stability is typically lost. To quantify


robustness better, the following definitions are introduced, and Lemma 5.27is the basic result.

Definition 5.26. System 5.52 is said to be robust cooperative in the presenceof w(k) with |wi(k)| ≤ cw if

limη→∞

supk≥η

maxi,j

|yi(η) − yj(η)| ≤ γ1(cw),

where γ1(·) is a strictly monotone increasing function of its argument.

Lemma 5.27. Suppose that System 5.49 is asymptotically cooperatively stableand that, for some integer κ > 0, its state transition matrix Φ(k + 1, i + 1) isscrambling for all (k − i) ≥ κ and uniformly with respect to k, i. Then, in thepresence of uniformly bounded error w(k), System 5.52 is robust cooperative.

Proof: It follows from |wi(k)| ≤ cw, from the definition of δ(·) in (4.15), andfrom Solution 5.54 that

maxi,j

|zi(k + 1) − zj(k + 1)|

≤ δ(Φ(k + 1, 0))maxi

|zi(0)| + 2δ

(

k−1∑

i=0

Φ(k + 1, i + 1)

)

cw + 2cw

≤ δ(Φ(k + 1, 0))maxi

|zi(0)| + 2

k−1∑

i=0

δ (Φ(k + 1, i + 1)) cw + 2cw. (5.57)

On the other hand, Φ(k + 1, i + 1) being uniformly scrambling impliesλ(Φ(k + 1, i + 1)) ≤ 1− ǫφ, where λ(·) is defined by (4.15) and ǫφ > 0. Hence,it follows from Inequality 4.18 that, for any positive integer l and for any ksatisfying (l − 1)κ + 1 ≤ k < lκ + 1,

k−1∑

i=0

δ (Φ(k + 1, i + 1)) =

k−1∑

i=(l−1)κ+1

δ (Φ(k + 1, i + 1)) +

(l−1)κ∑

i=0

δ (Φ(k + 1, i + 1))

≤ κ +

(l−1)κ∑

i=0

δ (Φ((l − 1)κ + 2, i + 1))

≤ κ +

l−1∑

j=1

jκ∑

i=(j−1)κ

δ (Φ((l − 1)κ + 2, i + 1))

≤ κ + κ

l−1∑

j=1

δ (Φ((l − 1)κ + 2, jκ + 1))

≤ κ + 1 + κl−2∑

j=1

l−2∏

i=j

δ (Φ((i + 1)κ + 1, iκ + 1))


≤ κ + 1 + κ

l−2∑

j=1

(1 − ǫφ)l−1−j → κ + ǫφ

ǫφas k, l → ∞.

The conclusion can be drawn by substituting the above result into (5.57) andby noting that δ(Φ(k + 1, 0)) → 0 as k → ∞.

Recall that Φ(k + 1, i + 1) being scrambling uniformly for all (k − i) ≥κ implies exponential convergence of System 5.49 to its consensus. Lemma5.27 remains to be the same without requiring the exponential convergencesince its proof can be generalized to the case that Φ(k + 1, i + 1) becomesscrambling for sufficient large (k − i). The following result shows that, if theerrors are vanishing, robust cooperativeness as well as asymptotic cooperativeconvergence can be maintained.

Definition 5.28. System 5.52 is said to be robust cooperatively convergentin the presence of w(k) with |wi(k)| ≤ cw if

limk→∞

maxi,j

|zi(η) − zj(k)| = 0.

Lemma 5.29. Suppose that System 5.49 is asymptotically cooperatively sta-ble. Then, if cumulative error magnitude series

∑ki=0 maxj |wj(i)| are Cauchy

sequences as limk,l→∞∑k

i=l maxj |wj(i)| = 0, System 5.52 is robust coopera-tively convergent. Furthermore, if error series wj(k) are absolutely convergentseries as

∑∞k=0 maxj |wj(k)| < ∞, System 5.52 is robust cooperatively conver-

gent and uniformly bounded.

Proof: Since System 5.49 is asymptotically cooperatively stable, limk→∞ Φ(k+1, 0)z(0) = c11 for some c1 ∈ ℜ. Hence, it follows from Solution 5.54 that Sys-tem 5.52 is robust cooperatively convergent if and only if, for some c2 ∈ ℜ,

limk→∞

[

k−1∑

i=0

Φ(k + 1, i + 1)w(i) − c2(k)1

]

= 0.

For robust cooperative convergence and uniform boundedness, c1 is a constant(for any given series of w(i)).

Let l = k/2 if k is even or l = (k + 1)/2 if otherwise. Then,

k−1∑

i=0

Φ(k + 1, i + 1)w(i) =

k−1∑

i=l

Φ(k + 1, i + 1)w(i) +

l∑

i=0

Φ(k + 1, i + 1)w(i).

It follows from limk,l→∞∑k

i=l maxj |wj(i)| = 0 and from Φ(k + 1, i + 1) beingrow-stochastic that


limk→∞

∣

∣

∣

∣

∣

k−1∑

i=l

Φ(k + 1, i + 1)w(i)

∣

∣

∣

∣

∣

≤ limk→∞

k−1∑

i=l

Φ(k + 1, i + 1) |w(i)|

≤ limk→∞

k−1∑

i=l

maxj

|wj(i)|1

= 0,

where | · | is the operation defined in Section 4.1. On the other hand, it followsfrom System 5.49 being asymptotically cooperatively stable that, for someη ∈ ℜn

+ with ηT1 = 1,

limk→∞

Φ(k + 1, l + 2) = 1ηT ,

and hence

limk→∞

l∑

i=0

Φ(k + 1, i + 1)w(i) = limk→∞

Φ(k + 1, l + 2)

l∑

i=0

Φ(l + 2, i + 1)w(i)

= 1 liml→∞

l∑

i=0

ηT Φ(l + 2, i + 1)w(i)

= 1 lim

k→∞c2(k),

from which robust cooperative convergence is concluded.If∑∞

k=0 maxj |wj(k)| < ∞, we have

∣

∣

∣

∣

∣

liml→∞

l∑

i=0

ηT Φ(l + 2, i + 1)w(i)

∣

∣

∣

∣

∣

≤ liml→∞

l∑

i=0

maxj

|wj(i)| < ∞,

from which c2(k) is uniformly bounded and convergent. Therefore, robustcooperative convergence and uniform boundedness are concluded.

Robust cooperative convergence requires only [zi(k) − zj(k)] → 0 and,unless zi(k) converges to a finite consensus, it does not imply that the system isuniformly bounded. Due to the presence of uniformly bounded errors, the statelikely becomes unbounded because the unperturbed System 5.49 is merelyLyapunov stable (i.e., not asymptotically stable or exponentially stable) andLyapunov stability is not robust in general, and robust cooperativeness is thebest achievable. The following example shows a few simple but representativecases.


P (k) =

⎡

⎣

1 0 00.5 0.5 00 0.5 0.5

⎤

⎦ .


It follows that, for any k > i,

Φ(k, i) =

⎡

⎣

1 0 01 − 0.5k−i 0.5k−i 0

1 − (k − i + 1)0.5k−i (k − i)0.5k−i 0.5k−i

⎤

⎦ .

Case (1): w(k) =[

w1 w2 w3

]Twhere w1 = 0. In this case, the system is

subject to constant errors, its state z(t) is not uniformly bounded, but it isrobust cooperative because

limk→∞

[

k−1∑

i=0

Φ(k + 1, i + 1) − k1[

1 0 0]

]

=

⎡

⎣

0 0 0−1 1 02 2 1

⎤

⎦ .

Case (2): w(k) =[

0 w2 w3

]Twhere w2w3 = 0. In this case, System 5.52

is also subject to constant errors; nonetheless its state z(t) is both uniformlybounded and robust cooperative because z1(k) is identical to that of System5.13.

Case (3): w(k) = 1/(k+1). In this case, System 5.52 is subject to vanishingerrors whose series are Cauchy. It can be verified that limk→∞[z(k)−H(k)1−z1(0)1] = 0, where H(k)

=∑k

i=11i is the so-called harmonic number. Hence,

the system is robust cooperatively convergent but not uniformly bounded.Case (4): w(k) = 1ǫk

w for 0 ≤ ǫw < 1. In this case, System 5.52 is subjectto exponentially vanishing errors (whose series are convergent). It can beverified that limk→∞ z(k) = 1

1−ǫw1+z1(0)1. Hence, the system is both robust

cooperatively convergent and uniformly bounded.

Robustification via Virtual Leader

Case (2) in Example 5.30 tells us that uniform boundedness of the state canbe ensured if the system topology has certain special property. The followinglemma provides such a result, and its proof can be completed by simply in-voking Theorem 5.13 and Lemmas 5.27 and 5.29. Recall from Section 5.3.3that the system structural condition of (5.58) can be achieved by introducinga virtual leader and its corresponding Sensing/communication Matrix 5.26.

Lemma 5.31. Suppose that System 5.49 is asymptotically cooperatively stableand that System 5.52 has the structure that, for some integer n1 > 0,

P (k) =

[

P11(k) 0P21(k) P22(k)

]

, P11(k) ∈ ℜn1×n1 , w1(k) = · · · = wn1(k) = 0.

(5.58)Then, System 5.52 is robust cooperative and uniformly bounded if w(k) is

uniformly bounded, and it is robust cooperatively convergent if∑k

i=0 |wj(i)|are Cauchy sequences as limk,l→∞

∑ki=l |wj(i)| = 0 for all j.


0 2 4 6 8 10 12Ŧ2

Ŧ1

0

1

2

3

4

5

6

7agent 1agent 2agent 3

Fig. 5.8. Rendezvous in the presence of measurement bias

As an illustration of Lemma 5.31, consider the rendezvous problem studiedin Sections 5.3.2 and 5.3.3. Here, we assume that w(t) = 0.01 is a small con-stant bias incurred in measuring neighbors’ positions. In the presence of thisbias, the system is robust cooperative but not robust cooperatively conver-gent; the response is shown in Fig. 5.8 for time interval [0, 50] and afterwardsthe state variables continue their motion in the upper right direction towardinfinity. In comparison, the virtual leader introduced to make the state con-verge to a specific location also keeps the state from going to infinity, whichis shown in Fig. 5.9(a). Robustfication is obvious from comparing Fig. 5.9(a)and the corresponding bias-free response in Fig. 5.9(b).

Robustification can also be achieved by designing a non-linear cooperativecontrol, which will be discussed in Section 6.3.1.

Robustification Using Deadzone on Relative Measurement and

Relative Motion

In the absence of disturbance, a cooperative system can also be made robustagainst bounded measurement errors by only taking relative measurement andby only making relative movement when the feedback exceeds certain errorlevel. To motivate this, reconsider Ideal System 5.49. It follows from P (k)being row-stochastic that, for any i = 1, · · · , n,


0 1 2 3 4 5 6 7 8 9Ŧ2

Ŧ1

0

1

2

3

4

5

6


(a) With measurement bias

0 1 2 3 4 5 6 7 8 9Ŧ2

Ŧ1

0

1

2

3

4

5

6


(b) Bias free

Fig. 5.9. Robustification via a virtual leader


zi(k + 1) − zi(k) =n∑

j=1

Pij(k)zj(k) − zi(k) =n∑

j=1,j =i

Pij(k)∆ij(k),

in which relative positions ∆ij(k)= [zj(k) − zi(k)] are measured whenever

sij(k) = 1 (i.e., Pij(k) > 0), and zi(k + 1) − zi(k) are the relative motionsexecuted at the kth step. Since only the measurement errors are consideredin this case, the corresponding system reduces to

zi(k + 1) − zi(k) =

n∑

j=1,j =i

P1j(k)∆ij(k)

=n∑

j=1,j =i

P1j(k)∆ij(k) +n∑

j=1,j =i

Pij(k)w′ij(k), (5.59)

where ∆ij(k) is the measurement of ∆ij(k), and w′ij(k) is the measurement

error uniformly bounded as |w′ij(k)| ≤ c′w. Note that System 5.59 is in the

same form as (5.51) and (5.52) except that the lumped error wi(k) is nowbounded as

|wi(k)| ≤ ǫc′w,

where ǫ = 1 − minj Pjj(k) and hence ǫ ∈ [0, 1). According to Example 5.30,Motion Protocol 5.59 may render unbounded state zi(k). In order to ensurerobustness of the state being bounded, we can simply introduce the deadzonefunction

Dc′w [η] =

0 if |η| ≤ c′wη if |η| > c′w

into (5.59) and obtain the following result.

Lemma 5.32. Suppose that System 5.49 is asymptotically cooperatively sta-ble. Then, the relative motion protocol

zi(k + 1) − zi(k) =n∑

j=1,j =i

P1j(k)Dc′w [∆ij(k)]

=

n∑

j=1,j =i

P1j(k)Dc′w [∆ij(k) + w′ij(k)], i = 1, · · · , n, (5.60)

ensures that the state z(k) is both uniformly bounded and robust cooperative.

Proof: Whenever |∆ij(k)| > c′w, System 5.60 is identical to (5.59) and (5.52).Hence, robust cooperativeness is readily concluded by invoking Lemma 5.27.

To show that z(k) is uniformly bounded, it is sufficient to show that, withrespect to k, maxi zi(k) is non-increasing and mini zi(k) is non-decreasing.Suppose that, at time instant k, zi∗ = maxi zi(k). Then, we have

∆i∗j(k)= zj(k) − zi∗(k) ≤ 0


for all j, which implies that

Dc′w [∆i∗j(k) + w′ij(k)] ≤ 0.

Substituting the above result into (5.60) yields

zi∗(k + 1) − zi∗(k) ≤ 0,

which shows that maxi zi(k) is non-increasing. Similarly, it can be shown thatmini zi(k) is non-decreasing. This concludes the proof.

Note that System 5.60 is non-linear but nonetheless Lemma 5.32 is es-tablished. Non-linear cooperative systems will be systematically studied inChapter 6.

5.8 Integral Cooperative Control Design

The virtual leader introduced in Sections 5.3.3 and 5.3.4 allows the hands-offoperator(s) to adjust the group behavior of consensus or formation, but thediscussion is limited to the case that the operator’s output is constant. InSection 5.7, the presence of a virtual leader is also shown to increase robust-ness of the group behavior. To account for possible changes in the operator’soutput, a multi-layer cooperative control design is presented below, and thedesign renders an integral-type cooperative control law.

Consider first the simple case that the sub-systems are given by

zi = ui, i = 1, · · · , q, (5.61)

and that the virtual leader is given by

z0 = u0, (5.62)

where ui ∈ ℜ is the control input to the ith sub-system, and u0 ∈ ℜ is theoperator control input. Assuming that u0 = v0 for a (piecewise-) constantvelocity command v0, we have

z0 = v0, v0 = 0.

In this case, cooperative control vi can be chosen to be of the integral form

ui = upi+ vi, vi = uvi

, (5.63)

where upiis a relative-position feedback cooperative control law, and uvi

is arelative-velocity feedback cooperative control law. For instance, in the spiritof Cooperative Control 5.11, upi

and uvican be chosen to be

5.8 Integral Cooperative Control Design 249

upi=

1q∑

η=0

siη(t)

q∑

j=0

sij(t)(zj − zi) and uvi=

1q∑

η=0

siη(t)

q∑

j=0

sij(t)(vj − vi),

respectively. It follows that the overall closed-loop system can be written as

z = [−I + D(t)]z + v, v = [−I + D(t)]v, (5.64)

where

z =

⎡

⎢

⎢

⎢

⎣

z0

z1

...zq

⎤

⎥

⎥

⎥

⎦

, S(t) =

⎡

⎢

⎢

⎢

⎣

1 0 · · · 0s10(t)

...sq0(t)

S(t)

⎤

⎥

⎥

⎥

⎦

, D(t) =

[

Sij(t)∑q

η=0 siη(t)

]

, v =

⎡

⎢

⎢

⎢

⎣

v0

v1

...vq

⎤

⎥

⎥

⎥

⎦

.

Closed-loop System 5.64 consists of two cascaded position and velocity sub-systems. It follows from Theorem 5.6 that, under uniform sequential com-pleteness of S(t), v → cv1 for some constant cv. Defining ez = z − cvt1, wecan rewrite the position sub-system in (5.64) as

ez = [−I + D(t)]ez + (v − cv1),

whose convergence properties can be concluded using Lemmas 5.27 and 5.29.Specifically, if S(t) is uniformly sequentially complete, convergence of v →cv1 is exponential, and hence convergence of z → (z0(t0) + cvt)1 is in turnestablished.

As an illustration on the performance under Integral Control 5.63, considerthe consensus problem of three entities defined by (5.61) with q = 3, let thevirtual leader be that in (5.62) and with u0 = v0 = 1, and set the augmentedsensor/communication matrix to be

S =

⎡

⎢

⎢

⎣

1 0 0 01 1 1 00 0 1 10 1 0 1

⎤

⎥

⎥

⎦

.

For the initial conditions of [z0(0) z1(0) z2(0) z3(0)] = [0 6 2 4], the simulationresults are given in Fig. 5.10 .

Obviously, System 5.61 can be extended to those in the form of (5.8), andthe above design process remains. On the other hand, Virtual Leader 5.61 canalso be extended to be that of (piecewise-) constant acceleration command,in which case the overall system under the integral cooperative control designbecomes

z = [−I + D(t)]z + v, v = [−I + D(t)]v + a, a = [−I + D(t)]a,

for which cooperative stability can be concluded in the same fashion. Moregeneralized operations by virtual leader, such as Models 5.24 and 5.22, canalso be analyzed and designed.


0 20 40 60 80 100 120 140 160 180 2000

20

40

60

80

100

120

140

160

180

200

Time (sec)

z0

z1

z2

z3


0 20 40 60 80 100 120 140 160 180 200Ŧ6

Ŧ5

Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

Time (sec)

z1Ŧz

2z

3Ŧz

2z

1Ŧz

3

(b) Consensus of positions

Fig. 5.10. Integral cooperative control


0 20 40 60 80 100 120 140 160 180 200Ŧ1.5

Ŧ1

Ŧ0.5

0

0.5

1

1.5

Time (sec)

u0

u1

u2

u3

(c) Control inputs and consensus of velocities



Combined into the hierarchical control structure in Section 1.4, analysis oflinear cooperative systems in this chapter makes it possible to design lin-ear cooperative controls for autonomous vehicles operating in a dynamicallychanging and uncertain environment and to exhibit not only certain groupbehavior but also their individual behaviors.

Early work on distributed robotics is based on artificial intelligence meth-ods [167], behavior-based control paradigm [7], and probabilistic exploration[69]. For example, decentralized architecture, task allocation, mapping build-ing, coordination and control algorithms are pursued for multi-vehicle teams[5, 17, 71, 184]; rule-based formation behaviors are defined and evaluatedthrough simulations [13]; heuristic rules are used to make mobile robots toform certain geometric patterns [248] or to converge to a point [4, 252]. In-deed, by observing animal behaviors and experimenting through computeranimation, biologically inspired rules are obtained, for instance, cooperativerules [115], the rules of cohesion, separation and alignment in a flock [217],and avoidance, aggregation and dispersion [149].

The alignment problem [263] represents a truly local-feedback cooperativecontrol problem, and the nearest neighbor rule is the solution initially derivedfrom experimentation. Using the graph theoretical approach, convergence of


the nearest neighbor rule is analyzed for a connected and undirected graph[98], for a directed graph with a spanning tree [214], for a directed graph witha globally reachable node [135]. Many of these analyses are done for agentsof relative degree one; an extension to second-order dynamics is possible butinvolved [255, 256]. A recent account on convergence of multi-agent coordina-tion, consensus and flocking can be found in [24].

Complementary to the graph theoretical approach, the matrix theoreticalmethod is adopted in this chapter. This approach is standard in systems andcontrol theory, and the lower triangular form and lower triangular complete-ness arise naturally due to their physical meanings [196, 198, 199, 201, 200].As shown in Section 4.2, all the results presented have their graphical ex-planations and can be visually as explicit as a graph. In terms of analysisand design, the matrix theoretical approach has the advantage that heteroge-neous systems of high relative-degree can be handled [202], control Lyapunovfunction can be found [205], and robustness can be analyzed [209].

In the simpler case that the topology of sensing and communication net-work is time-invariant, analysis and control of multi-vehicle systems can becarried out using such standard approaches in control theory as non-linearcontrol, sliding mode control, and so on [50, 105, 128, 179, 182, 253, 269].Early work on asymptotic stability analysis based on connective topologycan be found in [228, 229]. Closely related is the synchronization problem ofelectric circuits [41, 277], and the Lyapunov direct method is used to deriveRiccati-equation-like condition [276]. It is shown in Section 5.3 that the co-operative control framework can be used to solve the problems of consensus,rendezvous, tracking of a virtual leader, formation control, synchronization,and stabilization of interconnected systems. For autonomous vehicles that onlyuse onboard sensors to acquire neighbors’ information, maintaining networkconnectivity is crucial, and the circumcenter algorithm based on proximitygraph [4, 43, 134, 147] or one of its variations [146] should be either directlyimplemented or used for choosing the weights in the cooperative control law.

6

Cooperative Control of Non-linear Systems

In this chapter, cooperative stability of the following networked-connectednon-linear system is investigated:

x = F c(x, D(t)), (6.1)

or equivalently element-by-element,

xi = F ci (x, Di(t)) = F c

i (di1(t)x1, di2(t)x2, · · · , din(t)xn),

where x ∈ ℜn is the state, D(t) = [dij(t)] ∈ ℜn×n+ is a non-negative and

piecewise-constant matrix (but, unlike in the preceding analysis, D(t) is notrequired in general to be row-stochastic), Di(t) is the ith row of D(t), andF c

i (·) is the closed-loop dynamics of the ith state variable. In the case that xi

and xj are state variables of the same physical sub-system in the network, thevalues of dij(t) and dji(t) characterize the structural property of dynamicscoupling. If xi and xj are state variables of different physical sub-systems,dij(t) and dji(t) capture connectivity property of the sensing/communicationnetwork between the two sub-systems. While the structural property of sub-systems are typically fixed, the network topology often changes. Hence, matrixD(t) is generally time-varying, and changing values of its entries are not as-sumed to be known a priori in either stability analysis or cooperative controldesign in order to consider arbitrary topology changes.

Cooperative stability of linear systems has been studied in Chapter 5 andbased on convergence of multiplicative row-stochastic matrix sequences. How-ever, non-linear systems do not have a piecewise-constant matrix solution.The standard method to overcome this difficulty and handle non-linear sys-tems is to use the Lyapunov direct method. It will be shown in Section 6.1that, if the network has either bidirectional or balanced irreducible topologies,a common and known cooperative control Lyapunov function is available toderive cooperative stability conditions. If the network has sequentially com-plete but otherwise arbitrary topology changes, common Lyapunov function

254 6 Cooperative Control of Non-linear Systems

does not exist in general and, as shown in Section 5.6, a successful coopera-tive control Lyapunov function has to be solved based upon the topologicalchanges to be experienced by the network and hence cannot be found a pri-ori. This marks the major distinction between Lyapunov stability analysis ofnon-linear systems and cooperative stability analysis of networked non-linearsystems, and it is also the main difficulty in analyzing non-linear cooperativesystems. To overcome the difficulty, it is observed that cooperative controlLyapunov functions found in Section 5.6 all have the same and known com-ponents. In Section 6.2, a multi-variable topology-based comparison theoremand its extensions are introduced so that the knowledge of cooperative con-trol Lyapunov function is no longer required and that asymptotic cooperativestability can instead be analyzed using the known components of cooperativecontrol Lyapunov function. In Section 6.3, cooperative control is designed forseveral classes of systems including cascaded systems, systems in the feedbackform, affine systems, etc. Discrete non-linear cooperative systems, cooperativecontrol of non-holonomic systems, and cooperative and pliable behaviors aresubsequently discussed in Sections 6.4, 6.5, and 6.6, respectively.

6.1 Networked Systems with Balanced Topologies

The Lyapunov direct method can be readily applied to conclude asymptoticcooperative stability if a common cooperative control Lyapunov function canbe found for all the topologies under consideration. One such case is that thenetwork topology is restricted to be balanced in the sense that both D(t) andDT (t) are row-stochastic. As shown in Chapter 5, D(t) being row-stochasticcan be satisfied. Should the network be bidirectional, D(t) = DT (t), and thenetwork is always balanced. The following lemma summarizes the conditionson both system dynamics and network topology.

Lemma 6.1. Consider Non-linear Networked System 6.1 and suppose that thenetwork changes are always balanced. Then, System 6.1 is uniformly asymp-totically cooperatively stable if D(t) is row-stochastic, piecewise-constant anduniformly sequentially complete and if the following inequality holds along itstrajectory and for all i and k:

d

dt(xi − xk)2 ≤ −2(xi − xk)2 + 2

n∑

j=1

(xi − xk)[dij(t) − dkj(t)]xj . (6.2)

Proof: System 6.1 includes as its special case the linear networked systemx = [−I + D(t)]x for which the time derivative of (xi − xk)2 is

d

dt(xi − xk)2 = −2(xi − xk)2 + 2

n∑

j=1

(xi − xk)[dij(t) − dkj(t)]xj . (6.3)

6.1 Networked Systems with Balanced Topologies 255

Consider the following candidate of common cooperative control Lyapunovfunction:

Vc(x) =n∑

i=1

n∑

k=1

(xi − xk)2,

which is the special case of Cooperative Control Lyapunov Function 5.38 withλi = 1. It follows from D(t) being balanced that DT1 = 1 and hence Lemma5.18 holds with λi = 1 and P = I. Hence, comparing the two expressions of(6.3) and (6.2) and then following the proof of Lemma 5.18, we conclude thatthe time derivative of Vc(x) along any trajectory of System 6.1 is

Vc(x) ≤ −2

n∑

i=1

eTci

Qci(t)eci

, (6.4)

where Gi is the matrix defined in Lemma 5.2, eci= GT

i (x − xi1), Qci(t) =

GTi Q(t)Gi, and

Q(t) = 2I − D(t) − DT (t).

It follows from Q(t)1 = 0, from (d) of Theorem 4.27 and from Q(t) beingsymmetric that Q(t) is a singular M-matrix and also positive semi-definite.Thus, we know from (6.4) and (6.2) that (xi−xk)2 are both uniformly boundedand uniformly continuous.

Assume without loss of any generality that, at some time t, non-negativeD(t) has the following lower triangular canonical form: for some permutationmatrix T (t) and for some integer p(t) ≥ 1,

T T (t)D(t)T (t) =

⎡

⎢

⎢

⎢

⎣

E11 0 · · · 0E21 E22 · · · 0...

.... . .

...Ep1 Ep2 · · · Epp

⎤

⎥

⎥

⎥

⎦

= E\(t).

It follows from Lemma 4.44 that, if D(t) is lower triangularly complete, [2I +D(t) + DT (t)]k ≥ [I + D(t)]k + [I + DT (t)]k > 0 for sufficiently large integerk. Hence, [D(t) + DT (t)] as well as Q(t) are irreducible. By (a) of Theorem4.27, matrix Q(t) is of rank (n− 1) and, since Q(t)1 = 0, zT Q(t)z > 0 for allz = c1 and therefore matrix Qci

(t) is positive definite. In this case, asymptoticcooperative stability can be concluded from (6.4).

In general, D(t) may not be lower triangularly complete at any time t,but the cumulative topology changes over a finite period of time are lowertriangularly complete, and it follows from (6.4) that

∫ ∞

t0

[

n∑

i=1

eTci

Qci(t)eci

]

dt < ∞. (6.5)

In this case, the preceding analysis applies to lower-triangularly completesub-blocks of D(t) at every instant of time. That is, if k1, k2 are two indices


corresponding to any lower-triangularly complete sub-matrix of E\(t) givenabove and at any time t, there is a positive definite term of form (xk1 − xk2)

2

in the left hand of Inequality 6.5. Since the cumulative topology changes overa finite period of time are lower triangularly complete, all combinations ofquadratic terms (xk1 − xk2)

2 appear in the left hand of Inequality 6.5. Sincethe sensing/communication sequence is uniformly sequentially complete, everyterm of (xk1 −xk2)

2 in (6.5) is integrated over an infinite sub-sequence of finitetime intervals and, by Lemma 2.6, (xk1 − xk2) → 0 uniformly for all k1 andk2. That is, x → c1 for some c ∈ ℜ, which completes the proof.

Lemma 6.1 is limited to networked systems with balanced topologies.Without any centralized coordination, balanced topologies can only be achievedby requiring that the network be bidirectional. Clearly, Lemma 6.1 needs to begeneralized to overcome this restriction. Indeed, in many applications such asthose governed by a leader-follower teaming strategy, the sensing and commu-nication network is inherently unbalanced. In those cases, analysis and controlof cooperative systems with arbitrary topologies should be carried out, whichis the subject of the next section.

6.2 Networked Systems of Arbitrary Topologies

To analyze non-linear cooperative systems with arbitrary topologies, it is nat-ural to develop a Lyapunov-based argument since the Lyapunov direct methoddoes not require any knowledge of system solution and is a universal approachfor analyzing non-linear systems. For linear networked cooperative systems,cooperative control Lyapunov function is explicitly found as complete-squarefunction Vc in (5.44). Indeed, dynamics of linear cooperative systems aresimpler in the sense that all the sub-systems can be expressed in a canon-ical form and, since there is no time-varying or non-linear dynamics exceptfor piecewise-constant network changes, their Lyapunov functions are deter-mined primarily by the topology changes. Unlike Lyapunov functions in typ-ical control problems [108, 116, 192], cooperative control Lyapunov functionVc changes over time and is usually unknown and not differentiable becausethe network topology changes are instantaneous and not known a priori. Fornon-linear cooperative systems, finding a cooperative Lyapunov function be-comes more challenging. In this section, we present a comparative Lyapunovargument by requiring the knowledge of not the cooperative control Lyapunovfunction itself but its components. The Lyapunov comparison theorem is thenused to analyze and design several classes of non-linear cooperative systems.

6.2.1 A Topology-based Comparison Theorem

To motivate the basic concept of cooperative control Lyapunov function com-ponents and their comparative Lyapunov argument, consider first Linear Co-operative System 5.1, its Cooperative Control Lyapunov Function 5.44, and

6.2 Networked Systems of Arbitrary Topologies 257

the closely related Lyapunov Function 5.36. It follows that, as topologychanges, cooperative control Lyapunov function Vc and Lyapunov functionV change accordingly. Nonetheless, Vc and V have the same components ofx2

i and (xμ − xk)2 for any i, μ, k ∈ 1, · · · , n, and hence these componentsare called components of (cooperative control) Lyapunov functions. It followsfrom (5.1) that

d

dt

[

1

2x2

i

]

= −x2i +

n∑

l=1

dil(t)xixl, (6.6)

and that

d

dt

[

1

2(xμ − xk)2

]

= −(xμ − xk)2 +

n∑

l=1

(xμ − xk)[dμl(t) − dkl(t)]xl. (6.7)

Clearly, Linear Cooperative System 5.1 being uniformly bounded is equiva-lent to x2

i being uniformly bounded for all i, and Linear Cooperative Sys-tem 5.1 being asymptotically cooperatively stable implies and is implied bythat (xμ − xk)2 converges to zero for all μ, k. The first question arising iswhether Lyapunov stability and asymptotic cooperative stability can be di-rectly concluded from Equalities 6.6 and 6.7. Given Theorem 5.20, the an-swer to the first question should be affirmative. The second question arising iswhether Equalities 6.6 and 6.7 can be generalized so that Lyapunov stabilityand asymptotic cooperative stability of non-linear cooperative systems can beconcluded. In this regard, Lemma 6.1 provides an affirmative answer for thespecial case of balanced topologies. In what follows, a Lyapunov argument isprovided for systems of arbitrary topologies. To this end, Equalities 6.6 and6.7 are generalized into the following two conditions.

Definition 6.2. A time function α(t) is said to be strictly increasing (orstrictly decreasing) over a time interval [t1, t2] if α(t1) < α(t2) (or α(t1) >α(t2)) and if, for any [t′1, t

′2] ⊂ [t1, t2], α(t′1) ≤ α(t′2) (or α(t′1) ≥ α(t′2)).

Condition 6.3 System 6.1 is said to be amplitude dominant on the diagonalif, for all i, differential inequality

d

dtVi(xi) ≤ −ξi(|xi|) + ηi(xi)

n∑

l=1

dil(t)[βi,l(xl) − βi,l(xi)], (6.8)

ord

dtVi(xi) ≤ −ξi(|xi|) + ηi(xi)

n∑

l=1

dil(t)βi,l(xl − xi), (6.9)

holds for some positive definite, radially unbounded and differentiable func-tion Vi(·), piecewise-constant entries dil(t) of non-negative matrix D(t), non-negative function ξi(·), and strictly monotone increasing functions η(·) andβ(·) with η(0) = β(0) = 0.


Condition 6.4 System 6.1 is said to be relative amplitude dominant on thediagonal if, for any index pair μ, k, the following differential inequalityholds:

d

dtLμ,k(xμ − xk) ≤ [η′

μ,k(xμ) − η′μ,k(xk)]

n∑

l=1

dμl(t)[β′μ,k,l(xl) − β′

μ,k,l(xμ)]

−[η′μ,k(xμ) − η′

μ,k(xk)]n∑

l=1

dkl(t)[β′′μ,k,l(xl) − β′′

μ,k,l(xk)]

−ξ′μ,k(|xμ − xk|), (6.10)

or

d

dtLμ,k(xμ − xk) ≤ η′

μ,k(xμ − xk)n∑

l=1

[dμl(t)β′μ,k,l(xl − xμ)

−dkl(t)β′′μ,k,l(xl − xk)] − ξ′μ,k(|xμ − xk|), (6.11)

where Lμ,k(·) is a positive definite, radially unbounded and differentiable func-tion, piecewise-constant time functions dil(t) are the entries of non-negativematrix D(t), scalar function ξ′μ,k(·) is non-negative, and scalar functionsη′

μ,k(·), β′μ,k,l(·), and β′′

μ,k,l(·) are strictly monotone increasing functions andpass through the origin.

Both Conditions 6.3 and 6.4 contain two inequalities, and the two inequal-ities have the same implications but are not the same unless the functions onthe right hand sides are linear. In the conditions, matrix D(t) is not requiredto be row-stochastic. Should Vi(s) = 0.5s2, βi,l(s) = ηi(s) = s, and matrixD(t) be row-stochastic (i.e.,

∑

l dil(t) = 1), Inequality 6.8 becomes

d

dtVi(xi) ≤ −ξi(|xi|) + xi

n∑

l=1

dil(t)[xl − xi],

= −ξi(|xi|) − x2i +

n∑

l=1

dil(t)xixl,

≤ −x2i +

n∑

l=1

dil(t)xixl, (6.12)

which includes (6.6) as a special case. Similarly, Inequality 6.10 includes (6.7)as a special case. Based on Conditions 6.3 and 6.4, the following theoremprovides a straightforward Lyapunov argument on Lyapunov stability andasymptotic cooperative stability without requiring the knowledge of any spe-cific Lyapunov function or cooperative control Lyapunov function.


Theorem 6.5. Consider Networked System 6.1 satisfying the following con-ditions:

(a) Matrix sequence of D(t) over time is uniformly sequentially complete(which, by the structural property of each sub-system and through con-trol design, is true if and only if the sensor/commnunication sequence ofS(t) is uniformly sequentially complete over time). Furthermore, wheneverits element dij(t) = 0, it is uniformly bounded from below by a positiveconstant.

(b) Conditions 6.3 and 6.4 hold.

Then System 6.1 is both uniformly Lyapunov stable and uniformly asymptot-ically cooperatively stable.

Proof: Let Ω = 1, · · · , n be the set of indices on state variables. Then, thefollowing three index sub-sets of Ω are introduced to describe instantaneousvalues of the state variables: at any instant time t,

Ωmax(t) = i ∈ Ω : xi = xmax , Ωmid(t) = i ∈ Ω : xmin < xi < xmax ,

andΩmin(t) = i ∈ Ω : xi = xmin .

wherexmax(t) = max

j∈Ωxj(t), and xmin(t) = min

j∈Ωxj(t).

It is apparent that, unless xi = xj for all i and j, xmin < xmax and set Ω ispartitioned into the three mutually disjoint sub-sets of Ωmax, Ωmid and Ωmin.In addition, the index set of maximum magnitude is defined as

Ωmag(t) = i ∈ Ω : |xi| = xmag , xmag(t) = maxj∈Ω

|xj(t)|.

Thus, if i ∈ Ωmag(t), either i ∈ Ωmax(t) or i ∈ Ωmin(t) but not both unlessxj are identical for all j ∈ Ω. On the other hand, for each state variable xi,we can define the set of its neighbors as

Θi(t) = j ∈ Ω : j = i and dij > 0 .

In addition, the set of its neighbors with distinct values is characterized by

Θ′i(t) = j ∈ Ω : j = i, dij > 0, and xj = xi .

In particular, the set of the neighbors for those variables of maximum magni-tude is denoted by

Θ′i,mag(t) = i ∈ Ωmag, j ∈ Ω : j = i, dij > 0, and j ∈ Ωmag .

Among all the state variables, the maximum relative distance is defined as


δmax(t) = maxμ,k∈Ω

|xμ(t) − xk(t)|.

It is obvious that δmax(t) = xmax(t) − xmin(t).The proof is completed in six steps. The first step is regarding Lyapunov

stability, and the rest are about convergence and asymptotic cooperativeasymptotic stability. It is worth noting that, if xmin = xmax at some instantof time t, Ω = Ωmax = Ωmin while Ωmid is empty and that, by Step 3, thesystem is already asymptotically cooperatively stable. Thus, in the analysisafter Step 3, xmin < xmax is assumed without loss of any generality.

Step 1: Lyapunov Stability: to show Lyapunov stability, it is sufficient todemonstrate that the maximum magnitude of the state variables does notincrease over time. Suppose without loss of any generality that, at time t,i∗ ∈ Ωmag(t). It follows from (6.8) or (6.9), the definition of Ωmag(t), andD(t) being non-negative that

d

dtVi∗(|xi∗ |2) ≤ −ξi∗(|xi∗ |)

−

⎧

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎩

ηi∗(|xi∗ |)n∑

l=1

di∗l(t)[βi∗,l(|xi∗ |) − βi∗,l(xl)sign(xi∗)]

ηi∗(|xi∗ |)n∑

l=1

di∗l(t)βi∗,l(|xi∗ | − |xl|)

≤ −ξi∗(|xi∗ |) ≤ 0, (6.13)

from which xmag(t) is known to be non-increasing over time.Step 2: Strictly decreasing magnitude of xmag(t) over any time interval over

which, for all i ∈ Ωmag(t), the corresponding index sets Θ′i,mag(t) do

not remain empty: this conclusion is established by showing that, if|xi(t)| = xmag(t) and Θ′

i,mag(t) is non-empty, |xi(t)| is strictly decreasing.By definition, we know from Θ′

i,mag(t) being non-empty that there existssome j = i such that dij > 0 and j ∈ Ωmag. In this case, Inequality 6.13is strict and hence |xi(t)| is strictly decreasing. Furthermore, xmag(t) isstrictly monotone decreasing if Θ′

i,mag(t) are non-empty for all t and forall i ∈ Ωmag. In addition, since dij(t) is uniformly bounded away fromzero whenever dij(t) = 0, it follows from the strict version of (6.13) that,if xmax(t) is decreasing, the decreasing is uniformly with respect to time.

Step 3: Maximum distance δmax(t) being non-increasing: recall that η′μ,k(·),

β′μ,k,l(·) and β′′

μ,k,l(·) have the strictly monotone increasing property.Hence, we have that, for any μ∗ ∈ Ωmax and k∗ ∈ Ωmin,⎧

⎪

⎪

⎨

⎪

⎪

⎩

η′μ∗,k∗(xμ∗) − η′

μ∗,k∗(xk∗) > 0, η′μ∗,k∗(xμ∗ − xk∗) > 0

if Ωmax ∩ Ωmin = ∅,η′

μ∗,k∗(xμ∗) − η′μ∗,k∗(xk∗) = 0, η′

μ∗,k∗(xμ∗ − xk∗) = 0

if otherwise,


⎧

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎩

n∑

l=1

dμ∗l(t)[β′μ∗,k∗,l(xl) − β′

μ∗,k∗,l(xμ∗)] ≤ 0,

n∑

l=1

dμ∗l(t)β′μ∗,k∗,l(xl − xμ∗)] ≤ 0,

(6.14)

and⎧

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎩

n∑

l=1

dk∗l(t)[β′′μ∗,k∗,l(xl) − β′′

μ∗,k∗,l(xk∗)] ≥ 0,

n∑

l=1

dk∗l(t)β′′μ∗,k∗,l(xl − xk∗) ≥ 0.

(6.15)

Substituting the above inequalities into (6.10) and (6.11) that

d

dtLμ∗,k∗(xμ∗ − xk∗) ≤ −ξ′μ∗,k∗(|xμ∗ − xk∗ |) ≤ 0,

from which δmax(t) being non-increasing over time can be concluded.Step 4: Maximum distance δmax(t) being uniformly and strictly monotone

decreasing as long as D(t) is irreducible: based on the derivations in Step 3,we need only show that at least one of the inequalities in (6.14) and (6.15)is strict. To prove our proposition by contradiction, let us assume thatboth (6.14) and (6.15) be equalities. It follows that, unless xmin = xmax,

(6.15) being equalities =⇒ dk∗l(t) = 0 if l ∈ Ωmid ∪ Ωmax and k∗ ∈ Ωmin,

and that

(6.14) being equalities =⇒ dμ∗l(t) = 0 if l ∈ Ωmid ∪ Ωmin and μ∗ ∈ Ωmax.

Recall that, as long as xmin < xmax, index sets Ωmin, Ωmid and Ωmax

are mutually exclusive, and Ωmin ∪ Ωmid ∪ Ωmax = Ω. This means that,unless xmin = xmax, there is permutation matrix P (t) under which

P (t)D(t)PT (t) =

⎡

⎣

E11 0 00 E22 0

E31 E32 E33

⎤

⎦

= E(t), (6.16)

where Eii are square blocks, row indices of E11 correspond to those inΩmin, row indices of E22 correspond to those in Ωmax, and row indicesof E33 correspond to those in Ωmid. Clearly, the structure of matrix E(t)contradicts the knowledge that D(t) is irreducible. Hence, we know thatone of Inequalities 6.14 and 6.15 must be a strict inequality and henceδmax is strictly monotone decreasing. Again, the decrease is uniform withrespect to time since, whenever dij(t) = 0, dij(t) is uniformly boundedaway from zero.


Step 5: Maximum distance δmax(t) being uniformly and strictly monotonedecreasing if D(t) may not be irreducible but remains to be lower trian-gularly complete: the proof by contradiction is identical to that of Step 4because the structure of matrix E(t) in (6.16) cannot be lower triangularlycomplete.

Step 6: Maximum distance δmax(t) being strictly decreasing over an infinitesub-sequence of time intervals if D(t) may not be lower triangularly com-plete over any of the intervals but is sequentially complete: assume thatδmax(t0) > 0. Then the conclusion is established by showing that, givenany time instant t1, there exists a finite duration ∆t such that

δmax(t1 + ∆t) < δmax(t1), (6.17)

where ∆t > 0 depends upon changes of D(t) over [t1, t2) and the value ofδmax(t1).Consider index sets Ωmax(t1) and Ωmin(t1). It follows that δmax(t1) =xμ∗(t1) − xk∗(t1), where μ∗ ∈ Ωmax(t1) and k∗ ∈ Ωmin(t1). Evolution ofδmax(t) after t = t1 has two possibilities. The first case is that, for everyμ∗ ∈ Ωmax(t1), there exists k∗ ∈ Ωmin(t1) such that index pair μ∗, k∗belongs to the same lower-triangularly-complete block in the lower trian-gular canonical form of D(t1). In this case, it follows from Steps 5 and 3that δmax(t) is strictly decreasing at time t = t1 and non-increasing after-wards. Therefore, we know that, for any ∆t > 0, Inequality 6.17 holds.The second and more general case is that, at time t = t1 as well as ina finite interval afterwards, some of the indices in Ωmax(t) correspondto different diagonal block in the lower triangular canonical form of D(t)than those for all the indices of Ωmin(t). In this case, Steps 4 and 5 are nolonger applicable, while Step 3 states that δmax(t) is non-increasing for allt ≥ t1. Nonetheless, the sequence of matrix D(t) over time is uniformlysequentially complete and hence we know that, for any index i ∈ Ωmag,either i ∈ Ωmax(t) or i ∈ Ωmin(t), and set Θ′

i,mag cannot be non-emptyexcept over some finite intervals. It follows from Step 2 that xmag(t) isstrictly monotone decreasing over (possibly intermittent) time intervalsand hence there exists a finite length ∆t such that

xmag(t1 + ∆t) < 0.5[xmax(t1) − xmin(t1)]. (6.18)

Recalling xmag(t) = max|xmax(t)|, |xmin(t)|, we know from (6.18) that,for any μ ∈ Ωmax(t1 + ∆t) and k ∈ Ωmin(t1 + δt),

δmax(t1 + ∆t) = xmax(t1 + ∆t) − xmin(t1 + ∆t)

≤ 2xmag(t1 + ∆t)

< xmax(t1) − xmin(t1),

which establishes Inequality 6.17. In essence, we know that, while xmag(t)(i.e., max|xmax(t)|, |xmin(t)|) decreases, the value of [xmax(t)−xmin(t)]


could remain unchanged but only temporarily (and at latest till the in-stant that xmax(t) = −xmin(t)), and afterwards δmax(t) must decrease asxmag(t) does. Since t1 is arbitrary, strictly decreasing of δmax(t) over aninfinite sub-sequence of time intervals is shown.

It is now clear from Steps 3 to 6 that, if D(t) over time is uniformlysequentially complete, δmax(t) is uniformly asymptotically convergent to zero.Hence, asymptotic cooperative stability is concluded.

6.2.2 Generalization

As has been shown in Chapter 5, Item (a) of Theorem 6.5 is a topologicalrequirement on the system network, and it is both necessary and sufficientfor uniform asymptotic cooperative stability. On the other hand, Conditions6.3 and 6.4 can be generalized to the following two conditions on NetworkedDynamical System 6.1.

Condition 6.6 System 6.1 is said to be amplitude dominant on the diagonalif, for all i, inequality

xiFci (x, D(t)) ≤ 0

holds for all values of x(t) with |xi(t)| = max1≤j≤n |xj(t)|.

Condition 6.7 System 6.1 is said to be relative amplitude dominant on thediagonal if, for any index pair μ, k, inequality

(xi − xk)[F ci (x, D(t)) − F c

k (x, D(t))] ≤ 0, (6.19)

holds for all x(t) with xi(t) = max1≤j≤n xj(t) and xk(t) = min1≤j≤n xj(t).Furthermore, System 6.1 is called relative-amplitude strictly dominant on thediagonal if Inequality 6.19 holds and it becomes strict whenever D(t) is lowertriangularly complete and maxi xi(t) = minj xj(t).

The diagonal dominance conditions of 6.6 and 6.7 not only generalize Con-ditions 6.3 and 6.4 but also are more intuitive to understand and easier toverify. It is worth recalling that the proof of Theorem 6.5 is essentially a com-bination of amplitude dominance on the diagonal, relative-amplitude domi-nance on the diagonal, and their relationship to topological condition in termsof non-negative and piecewise-constant matrix D(t). Thus, the following the-orem on asymptotic cooperative stability can analogously be concluded, andit is based on Conditions 6.6 and 6.7.

Theorem 6.8. System 6.1 is both uniformly Lyapunov stable and uniformlyasymptotically cooperatively stable if the following conditions are satisfied:

(a) Non-negative and piecewise-constant matrix sequence of D(t) over timeis uniformly sequentially complete. Furthermore, whenever its elementdij(t) = 0, it is uniformly bounded from below by a positive constant.


(b) System 6.1 is both amplitude dominant on the diagonal and relative-amplitude strictly dominant on the diagonal.

Without the properties of amplitude/relative-amplitude dominance on thediagonal, Comparison Theorems 6.5 and 6.8 are not expected to hold, neitheris the Lyapunov argument in terms of cooperative Lyapunov function com-ponents. This is because, as shown in Sections 2.3.4 and 2.4.2, any transientovershoot could induce instability under switching and hence must be explic-itly accounted for in a Lyapunov argument but, if a cooperative system hasany overshoot, the overshoot cannot be calculated or estimated without theexact knowledge of the switching topologies.

To illustrate further Conditions 6.6 and 6.7 in terms of physical proper-ties of System 6.1, let us first introduce the following two concepts. Then,Lemma 6.11 provides necessary and sufficient conditions on whether a systemis minimum-preserving and/or maximum-preserving, and Lemma 6.12 estab-lishes the relationship between amplitude/relative-amplitude dominance onthe diagonal and minimum/maximum preservation.

Definition 6.9. Dynamical System 6.1 is said to be minimum preserving if,for any initial condition x(t0) ≥ cmin1, its trajectory preserves the minimumas x(t) ≥ cmin1.

Definition 6.10. System 6.1 is said to be maximum-preserving if, for anyinitial condition x(t0) ≤ cmax1, its trajectory preserves the maximum asx(t) ≤ cmax1.

Lemma 6.11. System 6.1 is minimum-preserving if and only if, for any j ∈1, · · · , n, inequality

F cj (x, D(t)) ≥ 0

holds for all values of x(t) with xj(t) = min1≤k≤n xk(t). System 6.1 ismaximum-preserving if and only if, for any j ∈ 1, · · · , n, inequality

F cj (x, D(t)) ≤ 0

holds for all values of x(t) with xj(t) = max1≤k≤n xk(t).

Proof: For any initial condition x(t0) ≥ cmin1 with xj(t0) = cmin, it followsfrom the property of F c

j (·) that xj(t0) ≥ 0 and hence xj(t) ≥ cmin. Since jis arbitrary, we know that x(t) ≥ cmin1. On the other hand, it is apparentthat, if F c

j (x, D(t)) < 0 for some j and for some x(t0) with xj(t0) = cmin,xj(t0) < 0 and hence xj(t0 + δt) < cmin for some sufficiently small δt > 0.Since t0 is arbitrary, the minimum-preserving is shown.

Necessity and sufficiency of a maximum-preserving system can be shownin a similar fashion.


Lemma 6.12. If System 6.1 is both minimum-preserving and maximum-preserving, it is both amplitude and relative-amplitude dominant on the di-agonal.

Proof: Let |xi(t)| = max1≤k≤n |xk(t)| ≥ 0, and there are two cases. The firstcase is that xi = |xi| ≥ 0 and hence xi(t) = max1≤k≤n xk(t), it follows fromLemma 6.11 that F c

i (x, D(t)) ≤ 0. The second case is that xi = −|xi| ≤ 0and hence xi(t) = min1≤k≤n xk(t), it follows again from Lemma 6.11 thatF c

i (x, D(t)) ≥ 0. Combining the two case yields Condition 6.6.Let xi(t) = max1≤j≤n xj(t) and xk(t) = min1≤j≤n xj(t). Then, (xi−xk) ≥

0. It follows from Lemma 6.11 that Condition 6.7 holds.

It is worth noting that, if System 6.1 is both amplitude and relative-amplitude dominant on the diagonal, it is maximum-preserving but may notbe minimum-preserving. This observation can be verified by considering thespecial case of F c

j (x, D(t)) = −η(xj), where η(·) is a strictly increasing func-tion with η(0) = 0. Indeed, with exception of the minimum preservation prop-erty, the aforementioned properties are all invariant under the introduction ofsuch term −η(xj) into F c

j (x, D(t)).Combining Theorem 6.8 and Lemma 6.12 yields the following corollary.

Corollary 6.13. System 6.1 is both uniformly Lyapunov stable and uniformlyasymptotically cooperatively stable if the following conditions are satisfied:

(a) Matrix sequence of D(t) over time is non-negative, piecewise-constantand uniformly sequentially complete. Furthermore, whenever its elementdij(t) = 0, it is uniformly bounded from below by a positive constant.

(b) System 6.1 is both minimum-preserving and maximum-preserving and, in(6.19), the inequality of (xi − xk)[F c

i (x, D(t)) − F ck (x, D(t))] < 0 holds

uniformly whenever maxi xi(t) = minj xj(t) and D(t) is lower triangularlycomplete.

An application of Theorem 6.5 or 6.8 typically involves two aspects: di-rectly check Conditions 6.6 and 6.7 (or Conditions 6.3 and 6.4), and verify thatmatrix D(t) has the same reducibility/irreducibility property as matrix S(t).It follows from Lemma 6.12 that Lemma 6.11, if applicable, can be used tosimplify the first aspect and then Corollary 6.13 provides the stability results.

Example 6.14. Consider the following networked non-linear system:

θμ =

q∑

j=1

sμj(t)[γμ(θj) − γμ(θμ)],

where μ = 1, · · · , q for some positive integer q, γμ(·) are strictly mono-tone increasing function with γμ(0) = 0, and sμj(t) are the entries of sens-ing/coomunication matrix S(t) ∈ ℜq×q. It is straightforward to verify thatthe system is minimum-preserving, maximum-preserving, and also relative-amplitude strictly dominant on the diagonal. Thus, the networked system


is globally uniformly asymptotically cooperatively stable provided that sens-ing/coomunication matrix S(t) is uniformly sequentially complete.

For the networked system

θμ =

q∑

j=1

sμj(t)γμ(θj − θμ), μ = 1, · · · , q,

global asymptotic cooperative stability can similarly be concluded. The sameconclusion can also be drawn for Aggregation Model 1.3 and SynchronizationModel 1.7.

Proof of Theorem 6.5 implies that local Lyapunov stability and localasymptotic cooperative stability can readily be concluded if Conditions 6.6and 6.7 (or Conditions 6.3 and 6.4) hold only in a neighborhood around theorigin.

Example 6.15. Consider the Kuramoto model:

θμ =

q∑

j=1

eμj(t) sin(θj − θμ),

where μ = 1, · · · , q,eμj(t) =

wμjsμj(t)∑q

i=1 wμisμi(t),

and wμi > 0 are weighting constants. Since sin(·) is locally strictly increasingwith sin 0 = 0 and since matrix E(t) = [eμj ] ∈ ℜq×q has the same property ofirreducibility/reducibility as matrix S(t), the Kuramoto model is locally uni-formly asymptotically cooperatively stable if sensing/coomunication matrixS(t) is uniformly sequentially complete.

For the non-linear version of Vicsek model:

θμ = − tan(θμ) +1

cos(θμ)

q∑

j=1

eμj(t) sin(θj),

the same conclusion can be drawn.

To achieve cooperative stability, an appropriate cooperative control shouldbe designed for every sub-system in the group and, based on the controlsdesigned and on structures of individual systems, matrix D(t) can be foundin terms of S(t). In the next section, specific classes of systems and theircooperative control designs are discussed.

6.3 Cooperative Control Design

In this section, the problem of designing cooperative control to achieve asymp-totic cooperative stability is discussed for several classes of non-linear systems.

6.3 Cooperative Control Design 267

Specifically, cooperative control design is carried out first for relative-degree-one systems, then for systems in the feedback form, and also for the classof affine systems. The design process for non-affine systems will be outlined.Other classes of systems such as non-holonomic systems will be illustrated inSections 6.5 and 6.6.

6.3.1 Systems of Relative Degree One

Consider the following systems:

yi = fi(yi) + vi, (6.20)

where i = 1, · · · , q, yi ∈ ℜm is the output of the ith system, and vi is thecontrol to be designed. System 6.20 is feedback linearizable. Without loss ofany generality, assume that control vi consists of an individual self feedbackpart and a cooperative control part, that is,

vi(t) = −ki(yi)yi + ui(y, S(t)) + wi(t), (6.21)

where cooperative control part ui can simply be set to be (5.11), S(t) is thesensing and communication matrix, wi(t) presents the noises from measure-ment and communication channels, and self feedback gain ki(·) is to be chosensuch that inequalities

yTi [fi(yi) − k(yi)yi] ≤ 0

(yi − yj)T [fi(yi) − ki(yi)yi − fj(yj) + kj(yj)yj ] ≤ 0

(6.22)

hold. It is straightforward to verify that, with the inequalities in (6.22) andwith wi = 0, Conditions 6.3 and 6.4 are met. Hence, by Theorem 6.5, System6.20 under Control 6.21 is uniformly asymptotically cooperatively stable ifthere is no noise and if matrix S(t) is uniformly sequentially complete overtime.

There are numerous choices for self feedback gain ki(yi) to satisfy theinequalities in (6.22). The simplest choice is ki(yi)yi = fi(yi), in which case theoverall closed-loop system reduces to a linear cooperative system. If yT

i [fi(yi)−ki(yi)yi] ≤ −ǫ1‖yi‖2+ǫ2 for some constants ǫl > 0, the overall closed-loopsystem is input-to-state stable with respect to noise wi(t). One such choice iski(yi)yi = fi(yi) + yi‖yi‖2, which ensures robustness.

To compare the resulting linear and non-linear cooperative systems, con-sider the case that m = 1, q = 3, and fi(yi) = 0, that the initial conditionis [−0.6 0.3 0.9]T , that the “noise” if present is a bias as w(t) = 0.02 × 1,and that S(t) is randomly generated as specified in Example 5.9. If ki(yi) = 0is chosen, the resulting cooperative system is linear, and its performance isshown in Fig. 6.1. If ki(yi) = y2

i is chosen, the resulting cooperative system isnon-linear, and its performance is shown in Fig. 6.2. Clearly, the non-linearfeedback term improves convergence rate but more importantly guarantees


0 5 10 15 20 25 30 35 40 45 50Ŧ0.6

Ŧ0.4

Ŧ0.2

0

0.2

0.4

0.6

0.8

1

Time (sec)

y1

y2

y3

(a) Noise free response

0 5 10 15 20 25 30 35 40 45 50Ŧ0.6

Ŧ0.4

Ŧ0.2

0

0.2

0.4

0.6

0.8

1

Time (sec)

y1

y2

y3

(b) Response under the noise

Fig. 6.1. Performance of linear cooperative system


0 5 10 15 20 25 30 35 40 45 50Ŧ0.6

Ŧ0.4

Ŧ0.2

0

0.2

0.4

0.6

0.8

1

Time (sec)

y1

y2

y3

(a) Noise free response

0 5 10 15 20 25 30 35 40 45 50Ŧ0.6

Ŧ0.4

Ŧ0.2

0

0.2

0.4

0.6

0.8

1

Time (sec)

y1

y2

y3

(b) Response under the noise

Fig. 6.2. Performance of non-linear cooperative system


0 1 2 3 4 5 6 7 8 9 10

x 104

Ŧ0.6

Ŧ0.4

Ŧ0.2

0

0.2

0.4

0.6

0.8

1

Time (sec)

y1

y2

y3

Fig. 6.3. Extended response of the non-linear cooperative system in Fig. 6.2(a)

uniform boundedness and robustness, while Linear Networked System 5.1 in-herently has limited robustness in the sense that its state will become un-bounded in the presence of persistent uncertainties.

On the other hand, the introduction of self feedback gain ki(yi) = y2μ

prevents the ith system from having the minimum-preserving property. Con-sequently, any non-zero consensus reached by the overall system slowly bleedsto zero. As shown in Fig. 6.3, the system is indeed asymptotically stable.Thus, to achieve both asymptotic cooperative stability (but not asymptoticstability) and robustness, the better choice of ki(yi) is

ki(yi) =

‖yi‖2 if y ∈ Ωf

0 else, (6.23)

where Ωf ⊂ ℜmq is a set containing the origin, and it specifies that admissibleequilibria of cooperative stability are c1 with c1 ∈ Ωf .

6.3.2 Systems in the Feedback Form

Consider first the following group of heterogeneous systems:⎧

⎪

⎪

⎨

⎪

⎪

⎩

yi = xi1, xi1 = ui, i = 1, · · · , q − 1,

yq = xq1,

⎧

⎨

⎩

xq1 = ζq1(γq1(xq2) − γq1(xq1)),xq2 = ζq2(γq2(xq3) − γq2(xq2)),xq3 = uq,

(6.24)


where yj ∈ ℜ are the system outputs, γqi(·) and ζqi(·) are strictly monotoneincreasing functions with γqi(0) = 0 and ζqi(0) = 0, and uj ∈ ℜ are thecontrol inputs to be designed. While the first (q−1) systems are linear, the qthsystem is cascaded and non-linear, and the cascaded structure and functionaldependence of its dynamics can be represented by the non-negative matrix

Aq =

⎡

⎣

1 1 00 1 10 0 0

⎤

⎦ .

For this group of q systems, non-linear self-state-feedback and neighbor-output-feedback cooperative control can be chosen to be

ui =

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎪

⎩

q∑

j=1

sij(t)ζi1(γi1(yj) − γi1(yi))

i = 1, · · · , q − 1,

ζq3(γq3(yq) − γq3(xq3))

q∑

j=1

sqj(t) +

q∑

j=1

sqj(t)ζq3(γq3(yj) − γq3(yq))

i = q,(6.25)

where sij(t) are the entries of sensing/communication matrix S(t), and γi1(·),γq3(·), ζi1(·), and ζq3(·) are odd and strictly monotone increasing functions.Substituting the controls into the system yields an overall networked systemin the form of (6.1) in which the combined system structure and networkmatrix is

D(t) =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎣

s11 · · · s1(q−1)(t) s1q(t) 0 0...

......

......

s(q−1)1(t) · · · s(q−1)(q−1) s(q−1)q(t) 0 00 · · · 0 1 1 00 · · · 0 0 1 1

sq1(t) · · · sq(q−1)(t) sqq 0 1

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎦

.

The above matrix can best be seen from the simplest case of ζq3(τ) = τ andγij(τ) = τ . It follows from (a) of Lemma 5.5 that the above matrix has thesame property of irreducibility/reducibility as matrix S(t). For cooperativestability analysis, the following lemma is introduced.

Lemma 6.16. Consider the following scalar differential equation:

zl = ζ(γ(zi) − γ(zl))∑

j

λj +∑

j

λjζ(γ(zj) − γ(zi)),

where zk ∈ ℜ, ζ(·) and γ(·) are odd and non-decreasing functions, and λj ≥ 0.Then, the differential equation of zl or simply zl is both minimum-preservingand maximum-preserving.



zl =∑

j

λj [ζ(γ(zi) − γ(zl)) + ζ(γ(zj) − γ(zi))].

If zl = maxk zk, we have

γ(zi) − γ(zl) ≤ γ(zi) − γ(zj),

which implies

ζ(γ(zi) − γ(zl)) + ζ(γ(zj) − γ(zi)) = ζ(γ(zi) − γ(zl)) − ζ(γ(zi) − γ(zj)) ≤ 0.

Similarly, if zl = mink zk, it can be shown that

ζ(γ(zi) − γ(zl)) + ζ(γ(zj) − γ(zi)) ≥ 0.

The proof is completed by summarizing the two cases.

It is apparent that, for System 6.24 under Control 6.25, differential equa-tions of x11 up to xq2 are both minimum-preserving and maximum-preserving.And, by Lemma 6.16, differential equation of xq3 is both minimum-preservingand maximum-preserving as well. It is straightforward to verify that theclosed-loop system is also relative-amplitude strictly dominant on the diag-onal. Therefore, by Corollary 6.13, System 6.24 under Cooperative Control6.25 is uniformly asymptotically cooperatively stable if and only if S(t) isuniformly sequentially complete.

As an example, consider System 6.24 and its Cooperative Control 6.25with q = 3,

ζ11(w) = ζ21(w) = ζ31(w) = ζ32(w) = ζ33(w) = w,

and

γ11(w) = γ21(w) = w5, γ31(w) = w3, γ32(w) = w13 , γ33(w) = w5.

Simulation is carried out with initial condition [−1.5 2 0.3 0.4 1]T and withS(t) being randomly generated as specified in Example 5.9, and the simulationresult is shown in Fig. 6.4.

It is straightforward to extend the above design and analysis to those sys-tems in the feedback form. Specifically, consider a group of q systems definedby

⎧

⎪

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎪

⎩

xi1 = ζi12(γi12(xi2) − γi12(xi1)),

xij =

j−1∑

k=1

ai,jkζik(j+1)(γik(j+1)(xik) − γik(j+1)(xij))

+ζij(j+1)(γij(j+1)(xi(j+1)) − γij(j+1)(xij)),xini

= ui,

(6.26)


0 20 40 60 80 100Ŧ1.5

Ŧ1

Ŧ0.5

0

0.5

1

1.5

2

Time (sec)

y1

y2

y3

x32

x33

Fig. 6.4. Performance of System 6.24 under Cooperative Control 6.25

where i = 1, · · · , q, yi = xi1 is the output, ai,jk are binary constants, and allthe functions have similar properties as those in (6.24). The structure of thenon-linear ith system can be represented by the non-negative matrix

Ai =

⎡

⎢

⎢

⎢

⎢

⎢

⎣

1 1 0 0 0ai,21 1 1 0 0

.... . .

. . .. . .

...ai,(ni−1)1 ai,(ni−1)2 · · · 1 1

0 0 0 · · · 0

⎤

⎥

⎥

⎥

⎥

⎥

⎦

∈ ℜni×ni

+ ,

whose closed-loop version is always irreducible under any control ui that con-tains yi. Thus, cooperative control can be chosen to be

ui =

ni−1∑

k=2

ai,nikζikni(γikni

(xik) − γikni(xini

))

+ζini1(γini1(yi) − γini1(xini))

q∑

j=1

sij(t)

+

q∑

j=1

sij(t)ζini1(γini1(yj) − γini1(yi)), (6.27)


where ai,nik are binary constants that can be arbitrarily chosen. Similar to theprevious analysis, one can conclude that System 6.26 under Control 6.27 isuniformly asymptotically cooperatively stable if and only if S(t) is uniformlysequentially complete.

6.3.3 Affine Systems

Among the class of affine systems, the following systems are guaranteed tobe minimum-preserving and maximum-preserving: for i = 1, · · · , q and forj = 1, · · · , ni,

xij =

ni∑

k=1

ai,jkρijk(xi)ζik(j+1)(γik(j+1)(xik) − γik(j+1)(xij)) + bijui, (6.28)

where xi = [xi1 · · · xini]T , ρijk(xi) > 0, ζik(j+1)(·) and γik(j+1)(·) are strictly

monotone increasing functions passing through the origin, ai,jk are binaryconstants with ai,ji = 1, and bij = 0 except for bini

= 1. The internal structureof System 6.28 is captured by the binary matrix

Ai = [ai,jk] ∈ ℜni×ni

+ . (6.29)

Now, consider the following neighbor-output-feedback cooperative control:

ui =

q∑

j=1

sij(t)ζ′i(γ

′i(yj) − γ′

i(xini)), (6.30)

where i = 1, · · · , q, and ζ′i(·) and γ′i(·) are strictly monotone increasing func-

tions passing through the origin. Substituting Control 6.30 into System 6.28yields the overall networked system which is in the form of (6.1) and has thecombined structural and networking matrix

D(t) =

⎡

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎣

A10 0

s12(t) 00 0

s13(t) 0· · · 0 0

s1q(t) 00 0

s21(t) 0A2

0 0s23(t) 0

· · · 0 0s2q(t) 0

.... . .

. . .. . .

...0 0

sq1(t) 00 0

sq2(t) 0· · · 0 0

sq(q−1)(t) 0Aq

⎤

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎦

,

where matrices Ai are those given by (6.29). It follows from the precedinganalysis and from Corollary 6.13 that the following theorem can be concluded.

Theorem 6.17. Suppose that matrices Ai in (6.29) are all irreducible. Then,System 6.28 under Control 6.30 is uniformly asymptotically cooperatively sta-ble if and only if S(t) over time is uniformly sequentially complete.


As an example, consider the following group of non-linear affine systems:

⎧

⎨

⎩

x11 = x712 − x7

11

x12 = arctan(x13 − x12)x13 = x3

11 − x313 + ui

,

⎧

⎨

⎩

x21 = arctan(x322 − x3

21)x22 = x3

21 − x322 + x23 − x22

x23 = x721 − x7

23 + u2

,

and⎧

⎨

⎩

x31 = x332 − x3

31 + sinh(x33 − x31)x32 = x31 − x32 + sinh(x3

33 − x332)

x33 = x331 − x3

33 + x32 − x33 + u3

.

Apparently, the above systems are in the form of (6.28), their internal dynam-ical interconnections are captured by matrices

A1 =

⎡

⎣

1 1 00 1 11 0 1

⎤

⎦ , A2 =

⎡

⎣

1 1 01 1 11 0 1

⎤

⎦ , A3 =

⎡

⎣

1 1 11 1 11 1 1

⎤

⎦ ,

and these structural matrices are all irreducible. Thus, Control 6.30 withζ′i(w) = w and γ′

i(w) = w5 can be applied. Its simulation results with initialcondition [−1.5 2 0.1 2 0.4 1 1 − 0.2 2]T are shown in Fig. 6.5, and allthe state variables converge to 1.8513.

6.3.4 Non-affine Systems

To illustrate the process of designing cooperative control for non-affine sys-tems, consider heterogeneous non-linear systems:

xi = Fi(xi, vi) yi = Cixi, i = 1, · · · , q, (6.31)

where xi ∈ ℜni is the state, yi ∈ ℜ is the output, vi ∈ ℜ is the control input,and Ci =

[

1 0 · · · 0]

. Requiring only real-time knowledge of Si(t) (the ithrow of sensing/communication matrix S(t)), control vi at the ith system canbe chosen to be of form

vi = −Ki(xi)xi + ui(xi, y, Si(t)), (6.32)

where Ki(xi) is the self-feedback control gain matrix, and cooperative controlpart ui(·) has a similar expression as (6.30). As such, ui has the propertiesthat ui(1,1, Si(t)) = 0 and that, if sij(t) = 0 for all j = i, ui(xi, y, Si(t)) = 0.Thus, for the overall system to exhibit cooperative behavior, it is necessarythat the isolated dynamics of the ith system, xi = Fi(xi,−Ki(xi)xi), beasymptotically cooperatively stable by themselves. Accordingly, Control 6.32should generally be synthesized in two steps:

Step 1: Find Ki(xi) such that isolated dynamics of xi = Fi(xi,−Ki(xi)xi) beuniformly asymptotically cooperatively stable.

Step 2: Determine ui(·) in a way parallel to that in (6.27) or (6.30).


0 2 4 6 8 10Ŧ1.5

Ŧ1

Ŧ0.5

0

0.5

1

1.5

2

Time (sec)

x11

x21

x31

(a) Trajectories of xi1

0 2 4 6 8 10Ŧ0.5

0

0.5

1

1.5

2

Time (sec)

x12

x22

x32

(b) Trajectories of xi2

Fig. 6.5. Performance under Cooperative Control 6.30


0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Time (sec)

x13

x23

x33

(c) Trajectories of xi3


In both steps, Theorem 6.8 or Corollary 6.13 can be applied.

Definition 6.18. Dynamical system z = F (z, w) is said to be positive if, forany non-negative initial condition x(t0) ≥ 0 and for any non-negative inputw(t) ≥ 0, its trajectory is also non-negative, i.e., x(t) ≥ 0.

It is apparent that any minimum-preserving closed-loop system is alwayspositive and that Lemma 6.11 can be applied to conclude whether a non-linear system is positive. An asymptotically cooperatively stable system maylikely be positive but is not always; for instance, System 6.20 under Control6.21 but with its self feedback term −ki(yi)yi redesigned (in a way parallel to(6.23)) to make admissible equilibrium set Ωf contain only c1 where c < 0. IfCorollary 6.13 is successfully applied to System 6.31 under Control 6.32, thesystem is both asymptotically cooperatively stable and positive.

6.3.5 Output Cooperation

In the preceding subsections, cooperative control is designed to make all thestate variables exhibit the same cooperative behavior denoted by x → c1.As such, system dynamics are required to meet stringent conditions. In manyapplications, what need to be cooperative are system outputs rather than thevectors of their whole state. In these cases, cooperative control can be designed


by combining an output consensus algorithm and a standard tracking control.Such a combination is conceptually simpler, easier to design and implement,and requires less restrictive conditions on system dynamics.

To be specific, reconsider the collection of the systems in (6.31), and theoutput cooperation problem is to ensure yi(t) → c for all i. Assume that,given any set of initial conditions of system outputs yi(t0), i = 1, · · · , q, thedesired output cooperative behavior be generated by the following discreteaverage algorithm:

ydi (tk+1) =

1

si1(t) + · · · + siq(t)

q∑

j=1

sij(tk+1)ydj (tk), yd

i (t0) = yi(t0), (6.33)

where tk : k ∈ ℵ is the sequence of time instants at which there are changesin sensing/communication matrix S(t). It follows from the discussion in Sec-tion 5.2.4 that, as long as matrix S(t) over time is uniformly sequentiallycomplete, average consensus of the desired outputs is ensured in the sensethat

ydi (tk+1) → [y1(t0) + · · · + yq(t0)]/q.

Upon prescribing Discrete Algorithm 6.33, the output cooperation problemof Systems 6.31 reduces to the problem of designing a tracking control ui =ui(xi, y

di ) under which output tracking yi → yd

i is achieved for any piecewise-constant desired trajectory yd

i . Such a tracking control can be designed usingone of the standard techniques such as input-output feedback linearizationand the backstepping design, and these techniques have been summarized inChapter 2.

6.4 Discrete Systems and Algorithms

In general, a discrete non-linear networked system can be expressed as, givenx(k) = [x1(k) · · · xn(k)]T ,

xi(k + 1) = F ci (x(k), D(k)), (6.34)

where x(k) ∈ ℜn is the state, D(k) = [dij(k)] ∈ ℜn×n+ is a non-negative and

diagonally positive matrix, and F ci (·) is the closed-loop dynamics of the ith

state variable. Then, Discrete System 6.34 can be analyzed using componentsof cooperative control Lyapunov function in a way parallel to the analysis ofContinuous-time System 6.1 in Section 6.2. Specifically, diagonal dominanceconditions as well as minimum-preserving and maximum-preserving propertiescan analogously be defined, and Theorem 6.21 can be concluded in the sameway as Theorem 6.8. It is straightforward to apply Theorem 6.21 to discretenon-linear systems such as Flocking Model 1.4. Also, Theorem 6.21 should beused to design discrete cooperative controls for various classes of networkedsystems (as done in Section 6.3).

6.4 Discrete Systems and Algorithms 279

Condition 6.19 System 6.34 is called amplitude dominant on the diagonalor, equivalently, maximum preserving if, for all i, inequality

F ci (x(k), D(k)) − xi(k) ≤ 0

holds for all values of x(k) with |xi(k)| = max1≤j≤n |xj(k)|.

Condition 6.20 System 6.34 is called minimum preserving if, for all i, in-equality

F ci (x(k), D(k)) − xi(k) ≥ 0

holds for all values of x(k) with |xi(k)| = min1≤j≤n |xj(k)|. Furthermore,System 6.34 is called relative-amplitude strictly dominant on the diagonal ifinequality

F ci (x(k), D(k)) − F c

j (x(k), D(k)) − [xi(k) − xj(k)] ≤ 0 (6.35)

holds for all x(k) with xi(k) = max1≤l≤n xl(k) and xj(k) = min1≤l≤n xl(k)and if it becomes strict whenever maxl xl(k) = minl xl(k) and D(k) is lowertriangularly complete.

Theorem 6.21. System 6.34 is both uniformly Lyapunov stable and uni-formly asymptotically cooperatively stable if the following conditions are sat-isfied:

(a) Non-negative and diagonally positive matrix sequence of D(k) over timeis uniformly sequentially complete. Furthermore, whenever its elementdij(k) = 0, it is uniformly bounded from below by a positive constant.

(b) System 6.34 is both amplitude dominant on the diagonal and relative-amplitude strictly dominant on the diagonal.

Non-linear cooperative systems arise naturally if one of non-linear consen-sus algorithms is applied. To focus upon non-linear consensus algorithms, letus assume for simplicity that the collection of agents have scalar and lineardynamics as

xi(k + 1) = ui(k), i = 1, · · · , q. (6.36)

Consider the so-called modified circumcenter algorithm: given any value ofsensing/communication matrix S(t) = [sij(t)] and for some scalar weight 0 ≤β ≤ 1,

xi(k + 1) = ui(k)= β max

j:sij(k) =0xj(k) + (1 − β) min

j:sij (k) =0xj(k), (6.37)

which specifies the location of a shifted circumcenter. If β = 0.5, Algorithm6.37 reduces to the standard circumcenter algorithm, that is,

xi(k + 1) =1

2

[

maxj:sij (k) =0

xj(k) + minj:sij(k) =0

xj(k)

]

.


If β = 1, it reduces to the maximum algorithm:

xi(k + 1) = maxj:sij(k) =0

xj(k);

and, if β = 0, it becomes the minimum algorithm:

xi(k + 1) = minj:sij(k) =0

xj(k).

The modified circumcenter algorithm has several useful properties as summa-rized in the following lemma.

Lemma 6.22. Consider Modified Circumcenter Algorithm 6.37. Then, itsconvergence properties are given as follows:

(a) If β ∈ (0, 1), System 6.37 is uniformly asymptotically cooperatively stableif and only if its associated sensing/communication matrix is uniformlysequentially complete.

(b) If β ∈ (0, 1), System 6.37 is uniformly asymptotically cooperatively stableand its convergence to a consensus is in a finite time if its associatedsensing/communication matrix S(k) is symmetrical and the cumulativesensing/communication matrices over time intervals are irreducible.

(c) If β = 0 or β = 1, System 6.37 is uniformly asymptotically cooperativelystable and it converges to a consensus in a finite time if its associatedcumulative sensing/communication matrices over time intervals are irre-ducible.

(d) If β = 0 or β = 1, System 6.37 may not converge to a consensus if itsassociated sensing/communication matrix is only uniformly sequentiallycomplete or if the cumulative sensing/communication matrices over timeintervals are only lower triangularly complete (over time).

Proof: Statement (a) is established by invoking Theorem 6.21 and notingthat the dynamics of (6.37) are both minimum-preserving and maximum-preserving and that the strict inequality of (6.35) is satisfied unless

maxj:sij (k) =0

xj(k) = minj:sij(k) =0

xj(k).

Statement (b) is due to the fact that, if S(k) ∈ ℜq×q is symmetric, all thestate variables associated with the same irreducible block converge to theirmodified circumcenter in no more than (q − 1) steps.

Statement (c) is obvious because, with S(k) being irreducible and of orderq, the maximum or the minimum propagates to all the state variables within(q − 1) steps.

To show statement (d), consider the case that β = 1 and that S(t) is lowertriangularly complete as

6.5 Driftless Non-holonomic Systems 281

S(t) =

⎡

⎢

⎢

⎢

⎣

S11(t) 0 0S21(t) S22(t) 0 0

.... . . 0

Sp1 · · · Sp(p−1) Spp(t)

⎤

⎥

⎥

⎥

⎦

,

where Sii(t) ∈ ℜri×ri are irreducible, Sij(t) = 0 for some j < i, xi(t0) <maxi xi(t0) for all i ≤ r1. It is apparent that the first group of agents neverreceives the maximum information from other agents and hence the maximumconsensus cannot be reached. Indeed, given the above structural of S(t), thesystem can reach its maximum consensus only if xi(t0) = maxi xi(t0) forsome i ≤ r1. The same conclusion can be drawn for the minimum consensusalgorithm.

In Section 5.4, the circumcenter algorithm is used to select the weightingcoefficients in a linear cooperative control and to maintain network connec-tivity. In the next section, the maximum or minimum algorithm is used todesign cooperative controls for non-holonomic systems.

6.5 Driftless Non-holonomic Systems

Consider a group of heterogeneous vehicles which move in the 2-D space andwhose models are already transformed into the chained form

zi1 = ui1, zi2 = ui1zi3, · · · zi(ni−1) = ui1zini, zni

= ui2, (6.38)

where i ∈ 1, · · · , q is the index of the vehicles, zi = [zi1, · · · , zini]T ∈ ℜni

is the state of the ith vehicle, ui = [ui1, ui2]T ∈ ℜ2 is its control input,

yi = [zi1, zi2]T ∈ ℜ2 is its output, and (zi1, zi2) are the x-axis and y-axis

coordinates of the ith vehicle in a fixed world coordinate system, respectively.Vehicle motion in the 3-D space can be handled in a similar fashion.

The cooperative control design problem discussed in this section is toachieve consensus for vehicles’ outputs. Note that, as shown in Section5.3, combining consensus and geometrical displacements yields a solution tosuch real-world problems as formation control, etc. Let S(t) = [sij(t)] andtk : k ∈ ℵ denote the matrix of the sensing/communication network amongthe vehicles and the time sequence of its topology changes, respectively. Then,there are two distinct cases for the consensus problem of non-holonomic sys-tems.

6.5.1 Output Rendezvous

For any rendezvous control, ui1(t) must be vanishing over time. As shown inChapter 3, a control design for the ith vehicle alone needs to be done carefullyaccording to how ui1(t) converges zero. Here, we need not only synthesize an


appropriately convergent control ui but also ensure position consensus of yi.To this end, let us define the following auxiliary signal vector based on themaximum algorithm: for all t ∈ [tk, tk+1) and for all k ∈ ℵ,

ydi (t) = yd

i (k) =

[

ydi1(k)

ydi2(k)

]

, (6.39)

where ζ > 0 and ǫ > 0 are design parameters, β(k) is a scalar positive sequencedefined by

β(k)= ǫe−ζtk , k ∈ ℵ

and the desired rendezvous location for the ith vehicle and in (6.39) is updatedby

ydi (−1)

=

[

zi1(t0)zi2(t0)

]

,

ydi (k)

=

⎡

⎣

maxj:sij (k) =0

ydj1(k − 1) + β(k)

maxj:sij(k) =0

ydj2(k − 1)

⎤

⎦ k ∈ ℵ.(6.40)

It is obvious that, if tk+1−tk ≥ c for some constant c > 0, the sum of sequenceβ(k) is convergent as

∞∑

k=0

β(k) =

∞∑

k=0

e−ζtk ≤ e−ζt0

∞∑

k=0

e−kζc =e−ζt0

1 − e−ζc.

To simplify the rendezvous design for non-holonomic systems, ydj (k) rather

than yj(t) is the information being shared through the network and usedin the cooperative control design. Discrete Algorithm 6.40 of generating thedesired rendezvous location yd

j (k) is a combination of the maximum consensusalgorithm and bias β(k) and hence, according to Lemma 6.22, it has followingproperties:

(a) Update ydi (k) of desired location is piecewise-constant, non-decreasing and

uniformly bounded. Also, its first element ydi1(tk) (i.e., zd

i1(tk)) is strictlyincreasing with respect to k.

(b) Excluding β(k), the maximum algorithm of yi(t0) converges to the max-imum consensus in a finite time if cumulative sensing/communicationmatrix of S(t) over time intervals is irreducible. The consensus of BiasedMaximum Algorithm 6.40 is the maximum of vehicles’ initial conditionsplus the limit of

∑∞k=0 β(k).

Accordingly, the consensus error system corresponding to System 6.38 is givenby: for t ∈ [tk, tk+1),

ei1 = ui1,ei2 = ui1zi3, · · · , zi(ni−1) = ui1zini

, zini= ui2,

(6.41)

where


ei1(t) = zi1(t) − ydi1(k), ei2(t) = zi2(t) − yd

i2(k).

Choose the following control for the first sub-system in (6.41):

ui1(t) = −ζ[zi1(t) − ydi1(k)], (6.42)

under which the solution during the interval t ∈ [tk, tk+1) is

ei1(t) = e−ζ(t−tk)[zi1(tk) − ydi1(k)]. (6.43)

Recalling from (6.40) that [zi1(t0) − ydi1(0)] ≤ −β(0), we know from (6.43)

that ei1(t) < 0 for all t ∈ [t0, t1). Hence, in the interval of [t0, t1), ui1(t) > 0,zi1(t) is monotone increasing from zi1(t0), and

zi1(t1) − ydi1(0) ≤ 0.

It follows from the above inequality and again from (6.40) that

zi1(t1) − ydi1(1) ≤ yd

i1(0) − ydi1(1) ≤ −β(1).

Now, repeating the same argument inductively with respect to k, we canconclude the following facts using Solution 6.43 under Control 6.42:

(a) ui1(t) > 0 for all finite instants of time, and it is monotone decreasing fort ∈ [tk, tk+1).

(b) For t ∈ [tk, tk+1), zi1(t) is monotone increasing from zi1(tk), and thefollowing two inequalities hold:

zi1(tk) − ydi1(k) ≤ −β(k), and zi1(t) ≤ yd

i1(k).

(c) It follows that, for all t ∈ [tk, tk+1) and no matter what (finite or infinite)value tk+1 assumes,

eζtui1(t) = −eζtkζ[zi1(tk) − ydi1(k)] ≥ ζeζtkβ(k) = ζǫ > 0.

(d) Output error component ei1(t) converges to zero exponentially.

Utilizing Property (c) above, one can design feedback control component u2i(t)as that defined by (3.84) (together with (3.82), (3.83) and (3.85)) to makeoutput error component ei2(t) converge to zero. In summary, we have thefollowing result.

Lemma 6.23. Consider the group of vehicles in (6.38). Then, output ren-dezvous of (yi − yj) → 0 is ensured under Biased Maximum Algorithm 6.40,Cooperative Control 6.42, and Control 3.84 if the cumulative versions of sens-ing and communication matrix S(t) over composite time intervals are irre-ducible.

To illustrate the performance under Biased Output Maximum Algorithm6.40, consider three vehicles described by (6.38) of order ni = 3, and choose


0 5 10 15 20 25 30 35 40 45 502

3

4

5

6

7

8

9

Time (sec)

z11

z21

z31

(a) Consensus of zi1

0 5 10 15 20 25 30 35 40 45 501

1.5

2

2.5

3

3.5

4

4.5

5

5.5

Time (sec)

z12

z22

z32

(b) Consensus of zi2

Fig. 6.6. Output consensus of three vehicles


ζ = 2 and ǫ = 1. Sensing/communication matrix S(t) is randomly generatedas specified in Example 5.9. Simulation is done for the initial output conditionsof y1(t0) = [6 3]T , y2(t0) = [2 5]T , and y3(t0) = [4 1]T . As shown in Fig.6.6, output variables yi2 follow yd

i2(k) and converge to the maximum of initialvalues of zi2(t0), while output variables yi1 follow yd

i1(k) and converge to themaximum of initial values of zi1(t0) plus a positive bias. Feedback controlinputs uij corresponding to tracking yd

ij(k) are shown in Fig. 6.7.In fact, any of the linear or non-linear consensus algorithms can be used to

generate ydi2(k). It is straightforward to develop a biased minimum algorithm

to replace Biased Maximum Algorithm 6.40. The minimum or maximumalgorithm is preferred for generating yd

i1(k) since these two algorithms renderthe nice property that ui1(t) has a fixed sign. An appropriate bias shouldalways be added in the algorithm of generating yd

i1(k) since, if not, consensusfor yi2 is not achievable when yi1(t0) are all identical (in which case ui1(t) iszero under any of unbiased standard consensus algorithms).

6.5.2 Vector Consensus During Constant Line Motion

Consider the vehicles described by (6.38) and assume their non-holonomicvirtual leader is described by

z01 = u10, z02 = u01z03, z03 = u02. (6.44)

The virtual leader is to command the group of vehicles through intermittentcommunication to some of the vehicles, and it is assumed to have a constantline motion as

u10(t) = v0, u02(t) = 0, z03(t0) = 0, (6.45)

where v0 = 0 is a constant. The control objective is to ensure the vectorconsensus of (zij − zkj) → 0 for all available j (since models of differentvehicles may be of different order) and for all i, k = 0, 1, · · · , q and k = i. Dueto Chained Structure 6.38, the standard single consensus of (zij −zil) → 0 forj = l does not make any sense physically and hence is not considered.

As analyzed in Chapter 3, each system in the chained form of (6.44) and(6.38) can be divided into two sub-systems. To facilitate the developmentof vector consensus control, a special form is developed for the second sub-system. For the virtual leader, sign(u01(t)) is constant, and the following statetransformation can be defined:

w02(t) = sign(u01(t))z02(t), w03(t) = z03(t) + w02(t),

which is also one-to-one. Hence, the second sub-system of Virtual Leader 6.44is mapped into

w02 = |u01|[w03(t) − w02(t)], w03(t) = −w03(t) + w02(t), (6.46)

provided that the second command component u02(t) is chosen as


0 5 10 15 20 25 30 35 40 45 500

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Time (sec)

u11

u21

u31

(a) Inputs ui1

0 5 10 15 20 25 30 35 40 45 50Ŧ1.2

Ŧ1

Ŧ0.8

Ŧ0.6

Ŧ0.4

Ŧ0.2

0

0.2

Time (sec)

u12

u22

u32

(b) Inputs ui2

Fig. 6.7. Control inputs for the vehicles


u02(t) = −z03(t) − sign(u01(t))z02(t) − |u01(t)|z03(t) + w02(t). (6.47)

It is straightforward to verify that, given (6.45), u02(t) = 0 for all t.The first sub-system of the ith vehicle in (6.38) is simply

zi1 = ui1. (6.48)

It follows from the discussion in Section 5.8 that the corresponding cooperativecontrol component to ensure vector consensus of x-axis position zi1 and thecorresponding velocity vi should be chosen as

ui1=

1q∑

l=0

sil(t)

q∑

j=0

sij(t)[zj1 − zi1] + vi,

vi =1

q∑

l=0

sil(t)

q∑

j=0

sij(t)(vj − vi),

(6.49)

where sij(t) is the element of binary augmented sensing/communication ma-trix S(t) ∈ ℜ(q+1)×(q+1).

Control ui1(t) defined in (6.49) is piecewise-continuous with respect to thechanges of sensing/communication topology and, within each of the corre-sponding time interval [tk, tk+1), sign(u01(t)) is also piecewise-constant. Forthe second sub-system in (6.38), one can find (as for the virtual leader) apiecewise-differentiable state transformation from variables of (zi2, · · · , zini

)to those of (wi2, · · · , wini

) and a control transformation from ui2 to u′i2 such

that⎧

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎩

wi2 = |ui1|(wi3 − wi2),...wi(ni−1) = |ui1|[wini

− wi(ni−1)],wini

= −wini+ u′

i2.

(6.50)

For instance, if ni = 3, the state and control mappings are

wi2 = sign(ui1(t))zi2, wi3 = zi3 + wi2, ui2 = −wi3 − |ui1(t)|zi3 + u′i2.

Similarly, if ni = 4, the transformations are

wi2 = sign(ui1(t))zi2, wi3 = zi3 + wi2, wi4 = sign(ui1(t))zi4 + zi3 + wi3,

and

ui2 = sign(ui1(t))[−wi3 − sign(ui1(t))zi4 − ui1(t)(wi4 − wi3) + u′i2],

which are one-to-one whenever ui1(t) = 0. Then, the cooperative control com-ponent ui2 can be designed in terms of u′

i2 as


u′i2 =

1q∑

l=0

sil(t)

q∑

j=0

sij(t)wj2 . (6.51)

Having known vector consensus of zi1 and vi and applying Theorem 6.8 orCorollary 6.13 to Sub-systems 6.46 and 6.50, we can conclude the followingresult on vector consensus of non-holonomic systems.

Lemma 6.24. Consider the group of vehicles in (6.38) and suppose that theirVirtual Leader 6.44 undertakes a line motion defined by (6.45). If the aug-mented sensing/communication matrix S(t) is uniformly sequentially com-plete, state consensus of (zij −zkj) → 0 is ensured under Cooperative Controls6.49 and 6.51.

To illustrate vector consensus, consider again three chained systems inthe form of (6.38) and with ni = 3. Virtual Leader 6.44 is to undertake aconstant line motion specified by (6.45) and with v0 = 1. Simulation is donewith augmented sensor/communication matrix

S =

⎡

⎢

⎢

⎣

1 0 0 01 1 1 00 0 1 10 1 0 1

⎤

⎥

⎥

⎦

and initial conditions of z0(t0) = [0 − 1 0], z1(t0) = [6 3 0]T , z2(t0) =[2 5 0]T and z3(t0) = [4 1 0]T . Vector consensus is shown in Fig. 6.8, andthe corresponding cooperative controls are shown in Fig. 6.9.

It is not difficult to show that, if u10(t) and u20(t) are polynomial func-tions of time, integral cooperative control laws can be designed along the lineof (6.49) to achieve vector consensus for chained systems. In general, motioninputs u10(t) and u20(t) of the virtual leader may be generated by certain ex-ogenous dynamics, and the above cooperative control design can be extendedto yield vector consensus for chained systems.

6.6 Robust Cooperative Behaviors

A cooperative control design is successful if the resulting cooperative systemhas the following features:

(a) It is robust to the changes of the sensing and communication network,and it ensures asymptotic cooperative stability whenever achievable.

(b) It is pliable to physical constraints (such as non-holonomic constraints).(c) It complies with the changes in the environment.

6.6 Robust Cooperative Behaviors 289

0 20 40 60 80 1000

10

20

30

40

50

60

70

80

90

100

Time (sec)

z11

z21

z31

z01

(a) Consensus of zi1

0 20 40 60 80 100Ŧ1

0

1

2

3

4

5

Time (sec)

z12

z22

z32

z02

(b) Consensus of zi2

Fig. 6.8. Vector consensus of chained systems


0 20 40 60 80 100Ŧ3

Ŧ2.5

Ŧ2

Ŧ1.5

Ŧ1

Ŧ0.5

0

0.5

1

1.5

Time (sec)

z13

z23

z33

z03

(c) Consensus of zi3


Item (a) is achieved by applying the analysis and design tools of the matrix-theoretical approach in Chapter 4 and the topology-based Lyapunov argumentin Section 6.2. In Section 6.5, cooperative control of non-holonomic systemsis addressed. In what follows, the cooperative control design is shown to berobust to latency in the sensing/communication network and, by incorporatingthe reactive control design in Section 3.4.2, it can be also be made robust toenvironmental changes for autonomous vehicles.

6.6.1 Delayed Sensing and Communication

Consider the following group of heterogeneous non-linear systems:

zj(t) = fj(zj(t), vj(t)) yj(t) = hj(zj(t)), (6.52)

where j = 1, · · · , q, zj(t) ∈ ℜnj is the state, yj ∈ ℜ is the output, and vj ∈ ℜis the control input. Without loss of any generality, control input vj can beassumed to be of form

vj(t) = −gj(zj(t)) + uj(zj(t), sj1(t)y1(t), · · · , sj(j−1)(t)yj−1(t),

sj(j+1)(t)yj+1(t), · · · , sjq(t)yq(t)), (6.53)


0 20 40 60 80 100Ŧ1.5

Ŧ1

Ŧ0.5

0

0.5

1

1.5

Time (sec)

u11

u21

u31

u01

(a) Control inputs ui1

0 20 40 60 80 100Ŧ1.5

Ŧ1

Ŧ0.5

0

0.5

1

1.5

2

Time (sec)

u12

u22

u32

u02

(b) Control inputs ui2

Fig. 6.9. Cooperative Control 6.49 and 6.51


where gj(zj(t)) is the self-feedback control part, sjl(t) is the (j, l)th element ofsensing/communication matrix S(t), and uj(·) is the cooperative control part.Suppose that, under Control 6.53, System 6.52 is both amplitude dominant onthe diagonal and relative-amplitude strictly dominant on the diagonal as de-fined by Conditions 6.6 and 6.7. Thus, by Theorem 6.8, the closed-loop systemis uniformly asymptotically cooperatively stable if the network is uniformlysequentially complete.

Now, consider the case that there are sensing and communication delaysin the network. In this case, Control 6.53 has to be modified to be

vj(t) = −gj(zj(t)) + uj(zj(t), sj1(t)y1(t − τj1), · · · , sj(j−1)(t)yj−1(t − τj(j−1)),

sj(j+1)(t)yj+1(t − τj(j+1)), · · · , sjq(t)yq(t − τjq)), (6.54)

in which time delays τji ∈ [0, τ ] are explicitly accounted for. Withoutloss of any generality, τ is assumed to be finite (otherwise, those sens-ing/communication channels may never be connected). If Control 6.54 ensuresasymptotic cooperative stability for all τji ∈ [0, τ ], it is robust to network la-tency.

To conclude delay-independent cooperative stability, let us express theclosed-loop dynamics of System 6.52 under Control 6.54 as

xi(t) = F ci (x(t), sj1(t)y1(t − τj1), · · · , sj(j−1)(t)yj−1(t − τj(j−1)),

sj(j+1)(t)yj+1(t − τj(j+1)), · · · , sjq(t)yq(t − τjq)),

where x = [zT1 · · · zT

q ]T , and xi is an entry in sub-vector zj of state x. Notethat, in F c

i (·), xi(t) appears without any delay, that is, τjj = 0 for all j. Itfollows from Condition 6.6 that, since F c

i (·) is amplitude dominant with re-spect to |xi(t)| = maxl |xl(t)| for the case that τjl = 0 for all l, F c

i (·) mustbe amplitude dominant with respect to |xi(t)| = maxl maxτjl∈[0,τ ] |xl(t− τjl)|for the case that τjl may be non-zero. This implies that, in the presence oflatency, Condition 6.6 remains except that the norm of maxl |xl(t)| is replacedby functional norm of maxl maxτjl∈[0,τ] |xl(t − τjl)|. Similarly, Condition 6.7can be extended. Upon extending Conditions 6.6 and 6.7, Theorem 6.8 can beproven as before to conclude delay-independent cooperative stability. Exten-sions of the maximum-preserving property, the minimum-preserving propertyand Corollary 6.13 are straightforward as well. In short, one can show usingthe concepts of amplitude dominance on the diagonal and relative-amplitudestrict dominance on the diagonal that stability properties of System 6.52 un-der Control 6.53 are retained for System 6.52 under Control 6.54.

As an example, the following three systems⎧

⎨

⎩

y1 = x11, x11 = u1,y2 = x21, x21 = u2,

y3 = x31, x31 = x332 − x3

31, x32 = x1/333 − x

1/332 , x33 = y5

3 − x533 + u3,

are considered in Section 6.3.2, and its neighbor-output-feedback cooperativecontrol is selected to be


0 20 40 60 80 100Ŧ1.5

Ŧ1

Ŧ0.5

0

0.5

1

1.5

2

Time (sec)

y1

y2

y3

x32

x33

Fig. 6.10. Consensus under latency

ui(t) =1

∑

l =i sil(t)

3∑

j=1, j =i

sij(t)[y3j (t) − y3

i (t)], i = 1, 2, 3.

In the presence of network delays, the cooperative control should be

ui(t) =1

∑

l =i sil(t)

3∑

j=1, j =i

sij(t)[y3j (t − τij) − y3

i (t)], i = 1, 2, 3.

It is obvious that the overall system with delays is maximum-preserving,minimum-preserving, and relative-amplitude strictly dominant on the diago-nal. Hence, asymptotic cooperative stability is delay-independent. To demon-strate this result, simulation is done with τij = 0.5 and under the same settingas that in Section 6.3.2 and, as shown in Fig. 6.10, consensus of all the statevariables are achieved.

It is worth recalling that, using the concept of quasi diagonal dominance,delay-independent asymptotic stability is concluded for linear time-invariantsystems [132] and for time-invariant interconnected systems with non-linearinterconnections [166]. With the aforementioned functional norm, Theorem6.8 can be used to conclude delay-independent asymptotic cooperative stabil-ity for non-linear and time-varying systems. On the other hand, if the self-feedback term in Control 6.54 is subject to time delay as gj(zj(t − τjj)), the


closed-loop dynamics no longer have delay-independent stability. In this case,cooperative stability of linear cooperative systems has been analyzed usingthe Nyquist stability criterion [62, 221], and may be achieved using predictivecontrol [203].

6.6.2 Vehicle Cooperation in a Dynamic Environment

Consider a group of nq vehicles operating in the following setting:

(a) There are many static and/or moving obstacles, and each of the vehiclescan only detect those obstacles in its vicinity;

(b) The vehicles should follow a continual motion of their virtual leader ac-cording to a prescribed formation, while the augmented sensing and com-munication matrix S(t) capturing the information flow within the wholegroup is intermittent and time-varying;

(c) Control of each vehicle should ensure cooperation among the vehicles,avoid any collision with obstacles or other vehicles, and require only lo-cally available information.

Given such a dynamically changing environment, the best option is to designa cooperative and reactive control. Since the vehicles are to move continually,it follows from the discussions in Section 3.1.3 that motion equations of thevehicles can be transformed into

qi = ui, (6.55)

where qi ∈ ℜl (with either l = 2 or l = 3) is the configuration coordinate ofthe ith vehicle, i = 1, · · · , nq, and ui is the control to be designed. It followsfrom the discussions in Section 5.3.4 that the cooperative formation controlis given by ui = uc

i , where

uci =

1

si0(t) + si1(t) + · · · + sinq(t)

nq∑

l=0

sil(t)

⎛

⎝ql − qi +

m∑

j=1

α′ilj e

′ilj(t)

⎞

⎠ ,

(6.56)

e′ilj are the estimates of orthonormal basis vectors of the moving frame Fl(t)at the lth vehicle, α′

ilj are the coordinates of the desired location for the ith

vehicle with respect to the desired formation in frame Fdl (t). Note that, by

incorporating the virtual leader as a virtual vehicle, Cooperative Control 6.56is not a tracking control and does not require the knowledge of qd(t) (or qd

i (t))all the time or for all the vehicles, nor is qd or qd(t) needed. Accordingly, bysetting qd

i = 0 and ξi(·) = 0, Reactive Control 3.118 reduces to

ui = −∂Pa(qi)

∂qi+ ur

i , (6.57)

where uri is the repulsive control component defined by


uri = −

∑

j∈Ni

∂Prj(qi − qoj

)

∂qi− k

∑

j∈Ni

∂2Prj(qi − qoj

)

∂(qi − qoj)2

(qi − qoj)‖qoj

‖2

−2k∑

j∈Ni

∂Prj(qi − qoj

)

∂(qi − qoj)

qToj

qoj+ ϕi(q

∗i , qi, qi − qoj

), (6.58)

Ni is the index set of all the entities (obstacles and other vehicles) in thevicinity of the ith vehicle, Prj

is the repulsive potential field function for thejth entity listed in Ni, and ϕi(·) is a uniformly bounded term to overcomeany saddle equilibrium point. Apparently, Cooperative Control 6.56 can becombined into Reactive Control 6.57 by piecewise defining the following co-operative attractive potential field function:

Pa(qi) =1

2

nq∑

j=0

sij(t)

nq∑

l=0

sil(t)

⎛

⎝ql − qi +

m∑

j=1

α′ilj e

′ilj(t)

⎞

⎠

2

, (6.59)

where sij(t) are piecewise-constants, and so may be e′ilj(t). Therefore, coop-erative and reactive control should be chosen to be

ui = uci + ur

i , (6.60)

where uci is the cooperative control component defined by (6.56), and ur

i isthe repulsive control component defined by (6.58).

The autonomous vehicle system of (6.55) under Control 6.60 is obviouslynon-linear. It follows from the discussions in Section 5.6 and the precedingsections in this chapter that there is a cooperative control Lyapunov functionVc(q) and that a sub-set of the Lyapunov function components correspond toCooperative Potential Field Function 6.59 and hence to cooperative controlcomponent uc

i . In a way parallel to the proof of Theorem 3.21 and to thepreceding analysis, one can show that, away from the obstacles, cooperativebehavior can be ensured if S(t) is uniformly sequentially complete and that,during the transient, there is no collision among the vehicles or between avehicle and an obstacle. That is, the robust cooperative behavior is achieved.Should the obstacles form a persistent trap, the cooperative behavior of fol-lowing the virtual leader may not be achievable.

To demonstrate the performance of Cooperative Reactive Control 6.60,consider a team of three physical vehicles plus a virtual vehicle as their vir-tual leader. The virtual leader and obstacles (one static and the other moving)in the environment are chosen to be the same as those in Section 3.4.2. Ac-cordingly, all the reactive control terms are calculated using the same repulsivepotential function in (3.122) with λrj

= 5 for all the obstacles and vehicles.For avoiding collision with one of the obstacles, ρj = ǫj = 0.3. For avoidingcollision among vehicles, ρj = ǫj = 0.2. The vehicle team has initial posi-tions of q1(t0) = [3 3]T , q2(t0) = [3.5 2]T and q3(t0) = [2 1]T and is to


Fig. 6.11. Performance under Cooperative Reactive Control 6.60

follow the virtual leader in a formation while avoiding any collision. The sens-ing/communication matrix and the vehicle formation are chosen to be sameas those in the simulation of Cooperative Control 5.29 in Section 5.3.4. Thus,formation coordinates α′

ilj and basis vectors e′ijl are the same except that,since trajectory of the virtual leader is different,

⎧

⎪

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎪

⎩

α′101 = 1, α′

102 = 0,

e′101 =

[

− sin( π20 t)

cos( π20 t)

]

,

e′102 =

[

− cos( π20 t)

− sin( π20 t)

]

.

Under this setting, Cooperative Reactive Control 6.60 is simulated, the ve-hicle system response is provided in Fig. 6.11, and the corresponding vehiclecontrols are shown in Fig. 6.12. To see better vehicles’ responses, several snap-shots over consecutive time intervals are provided in Figs. 6.13 - 6.15.


The standard comparison theorem is typically used to conclude qualitativeproperties from a scalar differential inequality by solving its equality version.

Ŧ4 Ŧ3 Ŧ2 Ŧ1 0 1 2 3 4

Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

4


movingobstacle

initial formation

finalformation

static obstacle


0 20 40 60 80 100 120 140 160Ŧ8

Ŧ6

Ŧ4

Ŧ2

0

2

4

6

Time (sec)

u11

u12

(a) u1

0 20 40 60 80 100 120 140 160Ŧ5

Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

4

Time (sec)

u21

u22

(b) u2

Fig. 6.12. Cooperative Reactive Control 6.60


0 20 40 60 80 100 120 140 160Ŧ1.5

Ŧ1

Ŧ0.5

0

0.5

1

1.5

Time (sec)

u31

u32

(c) u3


This comparative argument is always an integrated part of Lyapunov directmethod and has been widely used in stability analysis and control design [108].A comparison theorem in terms of vector differential inequalities is also avail-able (as theorem 1.5.1 on page 22 of [118]), and it requires the so-called quasi-monotone property (which does not hold for cooperative systems). Along thisline, Theorems 6.5 and 6.8 enrich Lyapunov stability theory for the purposeof cooperative stability analysis of non-linear networked systems and theircooperative control designs. They can be referred to as comparison theoremsfor cooperative systems [193] and are used to design non-linear cooperativecontrols [194, 197].

Cooperative stability of discrete non-linear systems can be analyzed [161]using a combination of graph theory, convexity, and non-smooth analysis withnon-differentiable Lyapunov function V (x) = maxi xi−minj xj . The other ap-proach is to conduct cooperative stability analysis in terms of the known com-ponents of cooperative control Lyapunov function, and proof and applicationsof Theorems 6.5 and 6.8 involve only a smooth and topology-based Lyapunovargument and do not require any convexity condition on system dynamics.Despite the unpredictable changes in S(t) and hence in D(t), the correspond-ing conditions in Theorems 6.5 and 6.8 can be easily checked and also used toproceed with cooperative control design for classes of continuous and discretenon-linear systems. Indeed, Theorems 6.5 and 6.8 also provide a smooth anal-


Ŧ4 Ŧ3 Ŧ2 Ŧ1 0 1 2 3 4

Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

4

Vehicle 1Vehicle 2Vehicle 3Moving Obstacle

t=0

t=10

t=0 t=10


Ŧ4 Ŧ3 Ŧ2 Ŧ1 0 1 2 3 4

Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

4


t=10

t=30

t=10 t=30

t=20


Fig. 6.13. Responses over [0, 30] and under Cooperative Reactive Control 6.60


Ŧ4 Ŧ3 Ŧ2 Ŧ1 0 1 2 3 4

Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

4Vehicle 1Vehicle 2Vehicle 3Moving Obstacle

t=30

t=50

t=30

t=50

t=40


Ŧ4 Ŧ3 Ŧ2 Ŧ1 0 1 2 3 4

Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

4Vehicle 1Vehicle 2Vehicle 3

t=50

t=80

t=50

t=80




Ŧ4 Ŧ3 Ŧ2 Ŧ1 0 1 2 3 4

Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

4


t=80

t=110

t=80

t=110


Ŧ4 Ŧ3 Ŧ2 Ŧ1 0 1 2 3 4

Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

4Vehicle 1Vehicle 2Vehicle 3Moving Obstacle

t=110

t=130

t=110

t=130

t=120




Ŧ4 Ŧ3 Ŧ2 Ŧ1 0 1 2 3 4

Ŧ4

Ŧ3

Ŧ2

Ŧ1

0

1

2

3

4Vehicle 1Vehicle 2Vehicle 3 t=130

t=160

t=130

t=160

(c) Phase portrait for t ∈ [130, 160]


ysis and explicit conditions on whether non-differentiable Lyapunov functionV (x) = maxi xi−minj xj (or equivalently, every of the known components of acooperative control Lyapunov function) keeps decreasing along the trajectoryof a networked system.

References

1. P. Alexandroff and H. Hopf. Topologie. J. Springer, Berlin, 19352. K. T. Alligood, T. D. Sauer, and J. A. Yorke. Chaos: An Introduction to

Dynamical Systems. Springer-Verlag, New York, NY, 19973. B. D. O. Anderson and J. B. Moore. Optimal Control Linear Quadratic Meth-

ods. Prentice-Hall, Inc., Engelwood Cliffs, NJ, 19894. H. Ando, Y. Oasa, I. Suzuki, and M. Yamashita. Distributed memoryless

point convergence algorithm for mobile robots with limited visibility. IEEETransactions on Robotics and Automation, 51:818–828, 1999

5. T. Arai, E. Pagello, and L. E. Parker. Editorial: Advances in multi-robotsystems. IEEE Transactions on Robotics and Automation, 18:655–661, 2002

6. T. Arai, E. Pagello, and L.E. Parker. Editorial: Advances in multi-robot sys-tems. IEEE Transactions on Robotics and Automation, 18:655–661, 2002

7. R. Arkin. Behavior-Based Robotics. MIT Press, Cambridge, MA, 19988. Z. Artstein. Stabilization with relaxed control. Nonlinear Analysis, TMA,

7:1163–1173, 19839. A. Astolfi. Discontinuous control of nonholonomic systems. Systems & control

letters, 27:37–45, 199610. M. Athens and P. L. Falb. Optimal Control. McGraw-Hill, New York, NY,

196611. J. B. Aubin and A. Cellina. Differential Inclusions. Springer-Verlag, Berlin,

198412. A. Bacciotti. Local Stabilizability of Nonlinear Control Systems. World Scien-

tific, Singapore, 199213. T. Balch and R. C. Arkin. Behavior-based formation control for multirobot

teams. IEEE Transactions on Robotics and Automation, 14:926–939, 199814. R.B. Bapat and T.E.S. Raghavan. Nonnegative Matrices and Applications.

Cambridge University Press, Cambridge, 199715. J. Barraquand and J.-C. Latombe. Nonholonomic multibody mobile robots:

controllability and motion planning in the presence of obstacles. In Proceed-ings of the IEEE International Conference on Robotics and Automation, pages2328–2335, Sacramento, CA, April 1991

16. R. E. Bellman and S. E. Dreyfus. Applied Dynamic Programming. PrincetonUniversity Press, Princeton, NJ, 1962

304 References

17. G. Beni. The concept of cellular robot. In Proceedings of 3rd IEEE Symposiumon Intelligent Control, pages 57–61, Arlington, Virginia, 1988

18. A. Berman and R. J. Plemmons. Nonnegative Matrices in the MathematicalSciences. Academic Press, Inc., New York, 1979

19. N. Biggs. Algebraic Graph Theory. Cambridge University Press, Cambridge,UK, 1974

20. G. Birkhoff and R. S. Varga. Reactor criticality and nonnegative matrices.Journal of the Society for Industrial and Applied Mathematics, 6 (4):354–377,1958

21. A. Bloch and S. Drakunov. Stabilization and tracking in the nonholonomicintegrator via sliding modes. Systems & Control letters, 29:91–99, 1996

22. A. Bloch, M. Reyhanoglu, and N. H. McClamroch. Control and stabilizationof nonholonomic dynamic systems. IEEE Transactions on Automatic Control,37:1746–1757, 1992

23. A. M. Bloch. Nonholonomic Mechanics and Control. Springer, New York, 200324. V. D. Blondel, J. M. Hendrickx, A. Olshevsky, and J. N. Tsitsiklis. Convergence

in multiagent coordination, consensus, and flocking. In Proceedings of the 44thIEEE Conference on Decision and Control, pages 2996 – 3000, Seville, Spain,Dec 2005

25. J. D. Boissonnat, A. Cerezo, and J. Leblond. Shortest paths of bounded curva-ture in the plane. In Proceedings of IEEE International Conference on Roboticsand Automation, pages 2315–2320, Nice,France, May 1992

26. J. Borenstein and Y. Koren. The vector field histogram — fast obstacle avoid-ance for mobile robots. IEEE Transactions Robotics and Automation, 7:278–288, 1991

27. S. Boyd, L. E. Ghaoui, E. Feron, and V. Balakrishnan. Linear Matrix Inequal-ities in System and Control Theory. SIAM, 1994

28. M. S. Branicky. Multiple lyapunov functions and other analysis tools forswitched and hybrid systems. IEEE Transactions on Automatic Control, 43:475– 482, 1998

29. C. M. Breder. Equations descriptive of fish schools and other animal aggrega-tions. Ecology, 3:361–370, 1954

30. R. W. Brockett. Asymptotic stability and feedback stabilization. In DifferentialGeometric Control Theory, volume R. W. Brockett, R. S. Millman, and H. J.Sussmann, Eds, pages 181–191, 1983

31. R.A. Brooks. A robust layered control system for a mobile robot. IEEE Journalof Robotics and Automation, 2:14–23, 1986

32. L. E. J. Brouwer. Ueber abbildungen von mannigfaltigkeiten. The Mathema-tische Annalen, 71:97115, 1912

33. A. E. Bryson and Y. C. Ho. Applied Optimal Control. Hemisphere PublishingCorp., Bristol, PA, 1975

34. L. Bushnell, D. Tilbury, and S. S. Sastry. Steering three-input chained formnonholonomic systems using sinusoids: The firetruck example. In Proceedingsof European Controls Conference, pages 1432–1437, 1993

35. H. Caswell. Matrix Population Models. Sinauer Associates, Sunderland, MA,2001

36. W. W. Charles. A technique for autonomous underwater vehicle route plan-ning. IEEE Transactions of Oceanic Engineering, 3:199–204, 1990

37. B. Charlet, J. Levine, and R. Marino. On dynamic feedback linearization.System & Control Letters, 13:143151, 1989

References 305

38. B. Charlet, J. Levine, and R. Marino. Sufficient conditions for dynamic statefeedback linearization. SIAM Journal on Control and Optimization, 29:3857,1991

39. C. T. Chen. Linear System Theory and Design. Holt, Rinehart and Winston,Inc., New York, 1984

40. W. L. Chow. Uber systemen von linearren partiallen differentialgleichungenerster ordnung. The Mathematische Annalen, 117:98–105, 1939

41. L. O. Chua and D. N. Green. A qualitative analysis of the behavior of dynamicnonlinear networks: stability of autonomous networks. IEEE Transactions onCircuits and Systems, 23:355–379, 1976

42. J. Chuyuan, Z. Qu, E. Pollak, and M. Falash. A new multi-objective con-trol design for autonomous vehicles. In Proceedings of the 8th InternationalConference on Cooperative Control and Optimization, University of Florida,Gainesville, Florida, January 30 - February 1 2008

43. J. Cortes, S. Martinez, and F. Bullo. Robust rendezvous for mobile autonomousagents via proximity graphs in arbitrary dimensions. IEEE Transactions onAutomatic Control, 51:1289–1298, 2006

44. J. Cortes, S. Martinez, T. Karatas, and F. Bullo. Coverage control for mobilesensing networks. IEEE Transactions on Robotics and Automation, 20:243–255, 2004

45. I. D. Couzin, J. Krause, N. R. Franks, and S. A. Levin. Effective leadershipand decision-making in animal groups on the move. Nature, 433:513–516, 2005

46. F. Cucker and S. Smale. Emergent behavior in flocks. IEEE Transactions onAutomatic Control, 52:852–862, 2007

47. A. Czirok, A. L. Barabasi, and T. Vicsek. Collective motion of self-propelledparticles: kinetic phase transition in one dimension. Physical Review Letters,82:209–212, 1999

48. I. Daubechies and J. C. Lagaias. Sets of matrices all infinite products of whichconverge. Linear Algebra and Applications, 161:227–263, 1992

49. W. P. Dayawansa and C. F. Martin. A converse lyapunov theorem for a class ofdynamical systems which undergo switching. IEEE Transactions on AutomaticControl, 44:751–760, 1999

50. J. P. Desai, J. Ostrowski, and V. Kumar. Controlling formations of multiplemobile robots. In Proceedings of IEEE Conference on Robotics and Automa-tion, pages 2864–2869, Leuven, Belgium, May 1998

51. R. Diestel. Graph Theory. Springer-Verlag, New York, 200052. D. V. Dimarogonal and K. J. Kyriakopoulos. On the rendezvous problem

for multiple nonholonomic agents. IEEE Transactions on Automatic Control,52:916–922, 2007

53. A. W. Divelbiss and J. T. Wen. A path space approach to nonholonomicmotion planning in the presence of obstacles. IEEE Transactions on Roboticsand Automation, 13:443–451, 1997

54. R. L. Dobrushin. Central limit theorem for nonstationary markov chains, partsi and ii. Theory of Probability and Its Applications, 1, no.4:65–80 and 329–383,1956

55. B. Donald, P. Xavier, J. Canny, and J. Reif. Kinodynamic motion planning.Journal of the ACM, 40:1048–1066, 1993

56. L. E. Dubins. On curves of minimal length with a constraint on average curva-ture, and with prescribed initial and terminal positions and tangents. AmericanJournal of Mathematics, 79:497–516, 1957

306 References

57. O. Egeland, E. Berglund, and O. J. Sordalen. Exponential stabilization ofa nonholonomic underwater vehicle with constant desired configuration. InProceedings of IEEE Conference on Robotics and Automation, pages 20–25, S.Diego, CA, May 1994

58. M. Erdmann and T. Lozano-Perez. On multiple moving objects. In Proceedingsof IEEE International Conference on Robotics and Automation, pages 1419–1424, San Francisco, CA, Apr. 1986

59. H. Eren, C. F. Chun, and J. Evans. Implementation of the spline method formobile robot path control. In Proceedings of the 16th IEEE Instrumentationand Measurement Technology Conference, volume 2, pages 739–744, May 1999

60. L. Farina and S. Rinaldi. Positive Linear Systems: Theory and Applications.John Wiley & Sons, Inc., New York, 2000

61. L. Faubourg and J. B. Pomet. Control lyapunov functions for homogeneousjurdjevic-quinn systems. ESAIM: Control Optimization and Calculus of Vari-ations, 5:293–311, 2000

62. J. A. Fax and R. M. Murray. Information flow and cooperative control ofvehicle formations. IEEE Transactions on Automatic Control, 49:1465–1476,2004

63. W. Feller. An Introduction to Probability Theory and Its Applications. Wiley,New York, NY, 1957

64. C. Fernandes, L. Gurvits, and Z. Li. Near-optimal nonholonomic motion plan-ning for a system of coupled rigid bodies. IEEE Transactions on AutomaticControl, 39:450–463, 1994

65. A. F. Filippov. Differential Equations with Discontinuous Righthand Sides.Kluwer Academic Publishers, Dordrecht, Netherlands, 1988

66. P. Fiorini and Z. Shiller. Motion planning in dynamic environments usingvelocity obstacles. International Journal of Robotics Research, 17:760–772,1998

67. M. Fliess, J. Levine, P. Martin, and P. Rouchon. Flatness and defect of non-linear systems: Introductory theory and examples. International Journal ofControl, 61:1327–1361, 1995

68. T. I. Fossen. Guidance and Control of Ocean Vehicles. Wiley, New York, 199469. D. Fox, W. Burgard, H. Kruppa, and S. Thrun. A probabilistic approach to

collaborative multi-robot localization. Autonomous Robots, 8:325–344, 200070. M. Frechet. Theorie des evennments en chaine dans le cas d’um nombre fini

d’etats possible. Recherches theoriques Modernes sur le cacul des probabilites,2, 1938

71. J. Fredslund and M. J. Mataric. A general algorithm for robot formations usinglocal sensing and minimal communication. IEEE Transactions on Robotics andAutomation, 18:837–846, 2002

72. Robust nonlinear control design: state-

73. ¨

Wiss. (Berlin), pages 456–477, 191274. F. R. Gantmacher.

195975. V. Gazi and K. M. Passino. Stability of a one-dimensional discrete-time asyn-

chronous swarm. IEEE Transactions on Systems, Man, and Cybernetics: PartB, 35:834–841, 2005

R. A. Freeman and P. V. Kokotovic.space and Lyapunov techniques. Birkhauser, Boston, MA, 1996

The Theory of Matrices, vol.II. Chelsea, New York, NY,

G. Frobenius. Uber Matrizen aus nicht-negativen Elementen. S.-B. Preuss.Akad.

References 307

76. V. Gazi and K.M. Passino. Stability analysis of swarms. IEEE Transactionson Automatic Control, 48:692–697, 2003

77. J.-M. Godhavn and O. Egeland. A lyapunov approach to exponential stabiliza-tion of nonholonomic systems in power form. IEEE Transactions on AutomaticControl, 42:1028–1032, 1997

78. C. Godsil and G. Royle. Algebraic Graph Theory. Springer-Verlag, New York,2001

79. D. T. Greenwood. Classical Mechanics. Prentice Hall, Englewood Cliffs, NJ,1977

80. D. T. Greenwood. Principles of Dynamics. Prentice Hall, Engelwood Cliffs,NJ, 1988

81. L. Grune and O. Junge. A set oriented approach to global optimal control.System & Control Letters, 54:169–180, 2005

82. Y. Guo and Z. Qu. Control of friction dynamics of an one-dimensional particlearray. Automatica, 44:2560–2569, 2008

83. Y. Guo, Z. Qu, and Z. Zhang. Lyapunov stability and precise control of thefrictional dynamics of an one-dimensional particle array. Physics Review LetterB, 73, no.9, paper # 094118, March, 2006

84. L. Gurvits. Stability of discrete linear inclusion. Linear Algebra and its Appli-cations, 231:47–85, 1995

85. J. Hajnal. The ergodic properties of non-homogeneous finite markov chains.Proc. Cambridge Philos. Soc., 52:67–77, 1956

86. J. Hajnal. Weak ergodicity in non-homogeneous markov chains. Proc. Cam-bridge Philos. Soc., 54:233–246, 1958

87. J. K. Hale. Ordinary Differential Equations. Wiley Interscience, New York,NY, 1969

88. H. J. Herrmann. Spontaneous density fluctuations in granular flow and traffic.In Nonlinear Physics of Complex Systems: Current Status and Future Trends,volume 476, pages 23–34, Lecture Notes in Physics, Berlin: Springer, 1997

89. H. R. Hertz.neuem Zusammenhange dargestellt. Barth, Leipzig, 19894, English translationMacmillan, London, 1899, reprint, Dover, NY, 1956

90. R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press,Cambridge, 1985

91. L. Hou, A. N. Michel, and H. Ye. Stability analysis of switched systems. InProceedings of the 35th IEEE Conference on Decision and Control, pages 1208–1212, Kobe, Japan, December 1996

92. M. Howard, Z. Qu, K. Conrad, and J. Kaloust. A multi-objective decisionmaking and learning model for increased autonomy of unmanned vehicles. InProceedings of AUVSI’s Unmanned Systems North America, San Diego, Cali-fornia, June 2008

93. D. Hsu, R. Kindel, J-C. Latombe, and S. Rock. Randomized kinodynamicmotion planning with moving obstacles. International Journal of RoboticsResearch, 21:233–255, 2002

94. L. R. Hunt, R. Su, and G. Meyer. Global transformations of nonlinear systems.IEEE Transactions on Automatic Control, 28:24–31, 1983

95. Y. K. Hwang and N. Ahuja. A potential field approach to path planning. IEEETransactions on Robotics and Automation, 8:23–32, 1992

96. A. Isidori. Nonlinear Control Systems. 3rd ed., Springer-Verlag, Berlin, 1995

Gesammelte Werke, Band III. Die Prinzipien der Mechanik in

308 References

97. D. H. Jacobson. Extensions of Linear Quadratic Control, Optimization andMatrix Theory. Academic Press, New York, NY, 1977

98. A. Jadbabaie, J. Lin, and A.S. Morse. Coordination of groups of mobile au-tonomous agents using nearest neighbor rules. IEEE Transactions on Auto-matic Control, 48:988–1001, 2003

99. B. Jakubczyk and W. Respondek. On linearization of control systems. Bull.Acad. Pol. Sci. Ser. Sci. Math., 28:517–522, 1980

100. Z. P. Jiang. Robust exponential regulation of nonholonomic systems withuncertainties. Automatica, 36:189–209, 2000

101. D. W. Jordan and P. Smith. Nonlinear Ordinary Differential Equations. 2ndedition, Oxford University Press, New York, 1989

102. T. Kailath. Linear Systems. Prentice Hall, Englewood Cliffs, NJ, 1980103. R. E. Kalman. Contributions to the theory of optimal control. Boletin de las

sociedad Matematica Mexicana, 5:102–119, 1960104. R. E. Kalman and J. E. Bertram. Control system analysis and design via the

“second method” of lyapunov i. continuous-time systems. Transactions of theASME, Journal of Basic Engineering, 82D:371–393, 1960

105. W. Kang, N. Xi, and A. Sparks. Theory and applications of formation control ina perceptive referenced frame. In Proceedings of IEEE Conference on Decisionand Control, pages 352–357, Sydney, Australia, Dec. 2000

106. K. Kant and S. W. Zucker. Planning collision free trajectories in time-varyingenvironments: a two level hierarchy. In Proceedings of IEEE InternationalConference on Robotics and Automation, pages 1644–1649, Raleigh, NC, 1988

107. M. H. Kaplan. Modern Spacecraft Dynamics and Control. John Wiley, NewYork, 1976

108. H. K. Khalil. Nonlinear Systems. Prentice Hall, 3rd ed., Upper Saddle River,NJ, 2003

109. O. Khatib. Real-time obstacle avoidance for manipulator and mobile robots.International Journal of Robotics Research, 5:90–98, 1986

110. M. Kisielewicz. Differential Inclusions and Optimal Control. Kluwer AcademicPublishers, Dordrecht, Netherlands, 1991

111. I. Kolmanovsky and N. H. McClamroch. Hybrid feedback laws for a class ofcascade nonlinear control systems. IEEE Transactions on Automatic Control,41:1271–1282, 1996

112. I. Kolmanovsky, M. Reyhanoglu, and N. H. McClamroch. Switched mode feed-back control laws for nonholonomic systems in extended power form. Systems& Control letters, 27:29–36, 1996

113. G. Kreisselmeier and T. Birkholzer. Numerical nonlinear regulator design.IEEE Transactions on Automatic Control, 39:33–46, 1994

114. A. J. Krener. A generalization of chow’s theorem and the bang-bang theoremto nonlinear control systems. SIAM Journal on Control, 12:43–52, 1974

115. M. J. B. Krieger, J.-B. Billeter, and L. Keller. Ant-like task allocation andrecruitment in cooperative robots. Nature, 406:992–995, 2000

116. M. Krstic, I. Kanellakopoulos, and P. V. Kokotovic. Nonlinear and AdaptiveControl Design. Wiley, New York, 1995

117. M. C. Laiou and A. Astolfi. Exponential stabilization of high-order chainedsystem. Systems & Control Letters, 37:309–322, 1999

118. V. Lakshmikantham and S. Leela. Differential and Integral Inequalities: Theoryand Application. Academic Press, Vol.I, New York, 1969

References 309

119. V. Lakshmikantham, S. Leela, and M. N. Oguztoreli. Quasi-solutions, vectorlyapunov functions, and monotone method. IEEE Transactions on AutomaticControl, 26:1149–1153, 1981

120. A. N. Langville and C. D. Meyer. A survey of eigenvector methods for webinformation retrieval. SIAM Review, 47:135–161, 2005

121. A. Lasota and M. C. Mackey. Chaos, Fractals, and Noise: Stochastic Aspectsof Dynamics. John Wiley & Sons, New York, NY, 1989

122. J. Latombe. Robot Motion Planning. Kluwer Academic Publisher, 1998, Boston123. J.-P. Laumond. Robot Motion Planning and Control. Springer-Verlag, 1998,

London124. J.-P. Laumond, P. E. Jacobs, M. Taix, and R. M. Murray. A motion planner for

nonholonomic mobile robots. IEEE Transactions on Robotics and Automation,10:577–593, 1994

125. S. Lavalle and J. Kuffner. Randomized kinodynamic planning. InternationalJournal of Robotics Research, 20:378–400, 2001

126. J. L. Lazaro and A. Gardel. Adaptive workspace modeling, using regressionmethods, and path planning to the alternative guide of mobile robots in envi-ronments with obstacles. In Proceedings of the 7th IEEE International Con-ference on Emerging Technologies and Factory Automation, volume 1, pages529–534, October 1999

127. D. D. Lee and H. S. Seung. Learning the parts of objects by non-negativematrix factorization. Nature, 401 (6755):788791, 1999

128. N. E. Leonard and E. Fiorelli. Virtual leaders, artificial potentials and coordi-nated control of groups. In Proceedings of IEEE Conference on Decision andControl, pages 2968–2973, Orlando, FL, Dec. 2001

129. W. W. Leontief. The structure of the American economy 1919-1929. HarvardUniversity Press, Cambridge, MA, 1941

130. F.L. Lewis, C.T. Abdallah, and D.M. Dawson. Control of Robot Manipulators.Macmillan, New York, 1993

131. M. A. Lewis and K. H. Tan. High precision formation control of mobile robotsusing virtual structures. Autonomous Robots, 4:387–403, 1997

132. R. M. Lewis and B. D. O. Anderson. Necessary and sufficient conditions fordelay-independent stability of linear autonomous systems. IEEE Transactionson Automatic Control, 25:735–739, 1980

133. D. Liberzon. Switching in Systems and Control. Birkhauser, Boston, MA, 2003134. J. Lin, A. Morse, and B. Anderson. The multi-agent rendezvous problem,

part 2: The asynchronous case. SIAM Journal on Control and Optimization,46:2120–2147, 2007

135. Z. Lin, B. Francis, and M. Maggiore. Necessary and sufficient graphical con-ditions for formation control of unicycles. IEEE Transactions on AutomaticControl, 50:121–127, 2005

136. Z. Lin, B. Francis, and M. Maggiore. State agreement for continuous-time cou-pled nonlinear systems. SIAM Journal on Control and Optimization, 46:288–307, 2007

137. Y. Liu and K. M. Passino. Stability analysis of one-dimensional asynchronousswarms. IEEE Transactions on Automatic Control, 48:1848–1854, 2003

138. Y. Liu, K. M. Passino, and M. M. Polycarpou. Stability analysis of m-dimensional asynchronous swarms with a fixed communication topology. IEEETransactions on Automatic Control, 48:76–95, 2003

310 References

139. J. Luo and P. Tsiotras. Exponentially convergent control laws for nonholonomicsystems in power form. Systems & control letters, 35:87–95, 1998

140. J. Luo and P. Tsiotras. Control design for chained-form systems with boundedinputs. Systems & control letters, 39:123–131, 2000

141. A. M. Lyapunov. Stability of Motion. Academic Press, New York, NY, 1966142. J. L. Mancilla-Aguilar and R. A. Garcia. A converse lyapunov theorem for

nonlinear switched systems. Systems & Control Letters, 41:67–71, 2000143. N. March and M. Alamir. Discontinuous exponential stabilization of chained

form systems. Automatica, 39:343–348, 2003144. R. Marino. On the largest feedback linearizable subsystem. System & Control

Letters, 6:245–351, 1986145. J. E. Marsden and T. S. Ratiu. Introduction to Mechanics and Symmetry.

Springer, New York, 1999146. S. Martinez. Practical rendezvous through modified circumcenter algorithms.

In Proceedings of IEEE Conference on Decision and Control, New Orleans,LA, 2007

147. S. Martinez, F. Bullo, J. Cortes, and E. Frazzoli. On synchronous roboticnetworks — part ii: Time complexity of rendezvous and deployment algorithms.IEEE Transactions on Automatic Control, 52:2214–2226, 2007

148. S. Mastellone, D. M. Stipanovic, C. R. Graunke, K. A. Intlekofer, and M. W.Spong. Formation control and collision avoidance for multi-agent nonholonomicsystems: Theory and experiments. International Journal of Robotics Research,27:107–126, 2008

149. M. Mataric. Minimizing complexity in controlling a mobile robot population.In Proceedings of IEEE International Conference on Robotics and Automation,pages 830–835, Nice, France, 1992

150. M. J. Mataric. Issues and approaches in the design of collective autonomousagents. Robotics and Autonomous Systems, 16:321–331, 1995

151. M. J. Mataric. Reinforcement learning in the multi-robot domain. AutonomousRobots, 4:73–83, 1997

152. F. Mazenc and M. Malisoff. Further constructions of control lyapunov func-tions and stabilizing feedbacks for systems satisfying jurdjevic-quinn condi-tions. IEEE Transactions on Automatic Control, 51:360–365, 2006

153. R. T. M’Closkey and R. M. Murray. Exponential stabilization of driftlessnonlinear control systems using homogeneous feedback. IEEE Transactions onAutomatic Control, 42:614–628, 1997

154. R. T. MCloskey and R. M. Murray. Exponential stabilization of driftless non-linear control systems using homogeneous feedback. IEEE Transactions onAutomatic Control, 42:614–628, 1997

155. A. M. Meilakhs. Design of stable control systems subject to parameter pertur-bation. Automat. Remote Control, 39:1409–1418, 1978

156. A. N. Michel and R. K. Miller. Qualitative Analysis of Large Scale DynamicalSystems. Academic Press, New York, NY, 1977

157. H. Minc. Nonnegative Matrices. John Wiley & Sons, New York, 1988158. A. P. Molchanov and Y. S. Pyatnitskiy. Criteria of absolute stability of dif-

ferential and difference inclusions encountered in control theory. Systems &Control Letters, 13:59–64, 1989

159. S. Monaco and D. Normand-Cyrot. An introduction to motion planning undermultirate digital control. In Proceedings of IEEE International Conference onDecision and Control, pages 1780–1785, Tucson, AZ, 1992

References 311

160. P. K. Moore. An adaptive finite element method for parabolic differentialsystems: some algorithmic considerations in solving in three space dimensions.SIAM Journal on Scientific Computing, 21:1567–1586, 2000

161. L. Moreau. Stability of multiagent systems with time-dependent communica-tion links. IEEE Transactions on Automatic Control, 50:169–182, 2005

162. P. Morin and C. Samson. Control of nonlinear chained systems: from therouth-hurwitz stability criterion to time-varying exponential stabilizers. IEEETransactions on Automatic Control, 45:141–146, 2000

163. P. Morin and C. Samson. Practical stabilization of driftless systems on liegroups: the transverse function approach. IEEE Transactions on AutomaticControl, 48:1496–1508, 2003

164. N. Moshtagh and A. Jadbabaie. Distributed geodesic control laws for flockingof nonholonomic agents. IEEE Transactions on Automatic Control, 52:681–686, 2007

165. E. Moulay and W. Perruquetti. Stabilization of nonaffine systems: A con-structive method for polynomial systems. IEEE Transactions on AutomaticControl, 50:520–526, 2005

166. P. J. Moylan. A connective stability result for interconnected passive systems.IEEE Transactions on Automatic Control, 25:812–813, 1980

167. R. R. Murphy. Introduction to AI Robotics. MIT Press, Cambridge, MA, 2000168. R. M. Murray, Z. Li, and S. S. Sastry. A Mathematical Introduction to Robotic

Manipulation. CRC Press, Inc., 1994, Boca Raton169. R. M. Murray and S. S. Sastry. Nonholonomic motion planning: Steering using

sinusoids. IEEE Transactions on Automatic Control, 38:700–716, 1993170. R. M. Murray and S. S. Sastry. Nonholonomic motion planning: steering using

sinusoids. IEEE Transactions on Automatic Control, 38:700–716, 1993171. Y. Nakamura and S. Savant. Nonlinear tracking control of autonomous under-

water vehicles. In Proceedings of IEEE Conference on Robotics and Automa-tion, pages A4–A9, Nice, France, May 1992

172. K. S. Narendra and A. M. Annaswamy. Stable Adaptive Systems. Prentice-Hall,Englewood Cliffs, NJ, 1989

173. K. S. Narendra and J. Balakrishnan. A common lyapunov function for sta-ble lti systems with commuting a-matrices. IEEE Transactions on AutomaticControl, 39:2469–2471, 1994

174. R. C. Nelson. Flight Stability and Automatical Control. McGraw-Hill, NewYork, NY, 1989

175. H. Nijmeijer and A. J. van der Schaft. Nonlinear Dynamical Control Systems.Springer-Verlag, New York, 1990

176. K. Ogata. Modern Control Engineering. Prentice Hall, Englewood Cliffs, NJ,1990

177. P. Ogren, E. Fiorelli, and N. E. Leonard. Cooperative control of a mobile sensornetworks: Adaptive gradient climbing in a distributed environment. IEEETransactions on Automatic Control, 49:1292–1302, 2004

178. A. Okubo. Dynamical aspects of animal grouping: swarms, schools, flocks andherds. Advances in Biophysics, 22:1–94, 1986

179. R. Olfati-Saber and R.M. Murray. Distributed cooperative control of multiplevehicle formations using structural potential functions. In Proceedings of the15th Triennial World Congress, Barcelona, Spain, 2002

312 References

180. R. Olfati-Saber and R.M. Murray. Flocking with obstacle avoidance: cooper-ation with limited communication in mobile network. In Proceedings of IEEEConference on Decision and Control, Maui, Hawaii, USA, 2003

181. G. Oriolo and Y. Nakamura. Free-joint manipulators: motion control undersecond-order nonholonomic constraints. In Proceedings of IEEE/RSJ Inter-national Workshop on Intelligent Robots and Systems (IROS’91), pages 1248–1253, Osaka: Japan, 1991

182. A. Pant, P. Seiler, and K. Hedrick. Mesh stability of look-ahead interconnectedsystems. IEEE Transactions on Automatic Control, 47:403–407, 2002

183. L. E. Parker. Alliance: An architecture for fault-tolerant multi-robot coopera-tion. IEEE Transactions on Robotics and Automation, 14:220–240, 1998

184. L. E. Parker. Current state of the art in distributed autonomous mobilerobotics. In Distributed Autonomous Robotic Systems 4, pages 3–12, L.E.Parker, G. Bekey and J. Barhen (Eds.), New York: Springer-Verlag, 2000

185. P. Peleties and R. A. DeCarlo. Asymptotic stability of m-switching systemsusing lyapunov-like functions. In Proceedings of American Control Conference,pages 1679–1684, Boston, MA, June 1991

186. O. Perron. Zur theorie der matrizen. The Mathematische Annalen, 64:248–263,1907

187. I.P. Petrov. Variational Methods in Optimum Control. Academic Press, NewYork, NY, 1968

188. A. Piazzi, C. G. Lo Bianco, M. Bertozzi, A. Fascioli, and A. Broggi. Quinticg2-splines for the iterative steering of vision-based autonomous vehicles. IEEETransactions on Intelligent Transportation Systems, 3:27–36, 2002

189. J.-B. Pomet. Explicit design of time-varying stabilizing control laws for a classof controllable systems without drift. Systems & control letters, 18:147–158,1992

190. S. Prajna, P. A. Parrilo, and A. Rantzer. Nonlinear control synthesists byconvex optimization. IEEE Transactions on Automatic Control, 49:310–314,2004

191. C. Prieur and A. Astolfi. Robust stabilization of chained systems via hybridcontrol. IEEE Transactions on Automatic Control, 48:1768–1772, 2003

192. Z. Qu. Robust Control of Nonlinear Uncertain Systems. Wiley Interscience,New York, 1998

193. Z. Qu. A comparison theorem for cooperative control of nonlinear systems. InProceedings of American Control Conference, Seattle, Washington, June 2008

194. Z. Qu, J. Chunyu, and J. Wang. Nonlinear cooperative control for consensusof nonlinear and heterogeneous systems. In Proceedings of the 46th IEEE Con-ference on Decision and Control, pages 2301–2308, New Orleans, Louisiana,December 2007

195. Z. Qu and D. M. Dawson. Robust Tracking Control of Robot Manipulators.IEEE Press, 1996, New York

196. Z. Qu, C. M. Ihlefeld, J. Wang, and R. A. Hull. A control-design-based solutionto robotic ecology: Autonomy of achieving cooperative behavior from a high-level astronaut command. In Proceedings of Robosphere 2004: A Workshop onSelf-Sustaining Robotic Systems, NASA Ames Research Center, Nov. 2004

197. Z. Qu, J. Wang, and J. Chunyu. Lyapunov design of cooperative control and itsapplication to the consensus problem. In Proceedings of IEEE Multi-conferenceon Systems and Control, Singapore, October 2007

References 313

198. Z. Qu, J. Wang, and R. A. Hull. Cooperative control of dynamical sys-tems with application to mobile robot formation. In Proceedings of the 10thIFAC/IFORS/IMACS/IFIP Symposium on Large Scale Systems: Theory andApplications, Japan, July 2004

199. Z. Qu, J. Wang, and R. A. Hull. Leaderless cooperative formation control ofautonomous mobile robots under limited communication range constraints. InProceedings of the 5th International Conference on Cooperative Control andOptimization, Gainesville, FL, Jan 2005

200. Z. Qu, J. Wang, and R. A. Hull. Multi-objective cooperative control of dy-namical systems. In Proceedings of the 3rd International Multi-Robot SystemsWorkshop, Naval Research Laboratory, Washington DC, March 2005

201. Z. Qu, J. Wang, and R. A. Hull. Products of row stochastic matrices andtheir applications to cooperative control for autonomous mobile robots. InProceedings of American Control Conference, Portland, Oregon, June 2005

202. Z. Qu, J. Wang, and R. A. Hull. Cooperative control of dynamical systemswith application to autonomous vehicles. IEEE Transactions on AutomaticControl, 53:894–911, 2008

203. Z. Qu, J. Wang, R. A. Hull, and J. Martin. Cooperative control design andstability analysis for multi-agent systems with communication delays. In Pro-ceedings of IEEE International Conference on Robotics and Automation, pages970–975, Orlando, FL, May 2006

204. Z. Qu, J. Wang, R. A. Hull, and J. Martin. Continuous control designs forstabilizing chained systems: A global state scaling transformation and a timescaling method. Optimal Control, Applications and Methods, 30:to appear,2009

205. Z. Qu, J. Wang, and X. Li. Quadratic lyapunov functions for cooperativecontrol of networked systems. In Proceedings of IEEE International Conferenceon Control and Automation, pages 416–421, Guangzhou, China, May 2007

206. Z. Qu, J. Wang, and C. E. Plaisted. A new analytical solution to mobile robottrajectory generation in the presence of moving obstacles. In Proceedings ofFlorida Conference on Recent Advances in Robotics, FAU, FL, May 2003

207. Z. Qu, J. Wang, and C. E. Plaisted. A new analytical solution to mobile robottrajectory generation in the presence of moving obstacles. IEEE Transactionson Robotics, 20:978–993, 2004

208. Z. Qu, J. Wang, C. E. Plaisted, and R. A. Hull. A global-stabilizing near-optimal control design for real-time trajectory tracking and regulation ofnonholonomic chained systems. IEEE Transactions on Automatic Control,51:1440–1456, September 2006

209. Z. Qu, J. Wang, and E. Pollak. Cooperative control of dynamical systemsand its robustness analysis. In Proceedings of the 45th IEEE Conference onDecision and Control, pages 3614–3621, San Diego, CA, December 2006

210. A. Rantzer. A dual to lyapunov’s stability theorem. System & Control Letters,42:161–168, 2001

211. J. A. Reeds and R. A. Shepp. Optimal paths for a car that goes both forwardand backwards. Pacific Journal of Mathematics, 145:367–393, 1990

212. J. H. Reif and H. Wang. Social potential fields: a distributed behavioral controlfor autonomous robots. Robotics and Autonomous Systems, 27:171–194, 1999

213. M. W. Reinsch. A simple expression for the terms in the baker-campbell-hausdorff series. Journal of Mathematical Physics, 41:2434–2442, 2000

314 References

214. W. Ren and R. W. Beard. Consensus seeking in multiagent systems underdynamically changing interaction topologies. IEEE Transactions on AutomaticControl, 50:655–661, 2005

215. M. Reyhanoglu. Exponential stabilization of an underactuated autonomoussurface vessel. Automatica, 33:2249–2254, 1997

216. M. Reyhanoglu, A. V. Der Schaft, N. H. McClamroch, and I. Kolmanovsky.Dynamics and control of a class of underactuated mechanical systems. IEEETransactions on Automatic Control, 44:1663–1671, 1999

217. C. W. Reynolds. Flocks, herds, and schools: a distributed behavioral model.Computer Graphics (ACM SIGGRAPH 87 Conference Proceedings), 21:25–34,1987

218. E. Rimon and D. E. Koditschek. Exact robot navigation using artificial poten-tial functions. IEEE Transactions Robotics and Automation, 8:501–518, 1992

219. R. T. Rockafellar. Convex Analysis. Princeton University Press, Princeton,NJ, 1970

220. W. Rudin. Functional Analysis. McGraw-Hill, New York, NY, 1973221. R. O. Saber and R. M. Murray. Consensus problems in networks of agents

with switching topology and time-delays. IEEE Transactions on AutomaticControl, 49:1520–1533, 2004

222. C. Samson. Control of chained systems: Application to path following and time-varying point-stabilization of mobile robots. IEEE Transactions on AutomaticControl, 40:64–77, 1995

223. S. Sastry. Nonlinear Systems: Analysis, Stability and Control. Springer-Verlag,New York, 1999

224. C. P. Schenk, P. Schutz, M. Bode, and H. G. Purwins. Interaction of selforganized quasi particles in a two dimensional reactive-diffusion system: Theformation of molecules. Physical Review E., 58:6480–6486, 1998

225. E. Seneta. On the historical development of the theory of finite inhomogeneousmarkov chains. Proc. Cambridge Philos. Soc., 74:507–513, 1973

226. E. Seneta. Non-negative Matrices and Markov Chain. Springer, New York,NY, 1981

227. R. Sepulchre, A. Paley, and N. E. Leonard. Stabilization of planar collectivemotion: all-to-all communication. IEEE Transactions on Automatic Control,52:916–922, 2007

228. D. D. Siljak. Connective stability of competitive equilibrium. Automatica,11:389–400, 1975

229. D. D. Siljak. Decentralized Control of Complex Systems. Academic press, 1991230. G. M. Siouris. Missile Guidance and Control Systems. Springer, New York,

2004231. R. Skjetne, S. Moi, and T.I. Fossen. Nonlinear formation control of marine

craft. In Proceedings of IEEE Conference on Decision and Control, pages1699–1704, Las Vegas, Nevada, Dec. 2002

232. J. J. Slotine and W. Li. Applied Nonlinear Control. Prentice-Hall, EnglewoodCliffs, NJ, 1991

233. W. M. Sluis. A necessary condition for dynamic feedback linearization. System& Control Letters, 21:277283, 1993

234. E. D. Sontag. A universal construction of artstein’s theorem on nonlinearstabilization. System & Control Letters, 13:117–123, 1989

235. E. D. Sontag. Mathematical Control Theory. Springer-Verlag, New York, NY,1991

References 315

236. E. D. Sontag. Control lyapunov function. In Open Problems in MathematicalSystems and Control Theory, pages 211–216, V. D. Blondel, E. D. Sontag, M.Vidyasagar, and J. C. Willems (Eds.), London: Springer-Verlag, 1999

237. O. J. Sordalen. Conversion of the kinematics of a car with n trailers into achained form. In Proceedings of IEEE Conference on Robotics and Automation,pages 382–387, Atlanta, Georgia, May 1993

238. O. J. Sordalen. Conversion of the kinematics of a car with n trailers into achained form. In Proceedings of the IEEE International Conference on Roboticsand Automation, pages 382–387, 1993

239. O. J. Sordalen and O. Egeland. Exponential stabilization of nonholonomicchained systems. IEEE Transactions on Automatic Control, 40:35–49, 1995

240. O. Soysal and E. Sahin. Probabilistic aggregation strategies in swarm roboticsystems. In Proceedings of the IEEE Swarm Intelligence Symposium, Pasadena,CA., 2005

241. M. W. Spong. Energy based control of a class of underactuated mechanical sys-tems. In Proceedings of IFAC World Congress, pages 431–435, San Francisco,CA, 1996

242. M. W. Spong and M. Vidyasagar. Robot Dynamics and Control. John Wiley& Sons, New York, NY, 1989

243. A. Stentz. Optimal and efficient path planning for partially-known environ-ments. In Proceedings of IEEE International Conference on Robotics and Au-tomation, volume 4, pages 3310–3317, San Diego, CA, May 1994

244. A. Stentz. The focussed d* algorithm for real-time replanning. In Proceedingsof the International Joint Conference on Artificial Intelligence, Pittsburgh,Pennsylvania, August 1995

245. B. L. Stevens and F. L. Lewis. Aircraft Control and Simulation. Wiley, Hobo-ken, N.J., 2003

246. D. M. Stipanovic, P. F. Hokayem, M. W. Spong, and D. D. Siljak. Coopera-tive avoidance control for multi-agent systems. Journal of Dynamic Systems,Measurement, and Control, 129:699–707, 2007

247. S. H. Strogatz. From kuramoto to crawford: Exploring the onset of synchro-nization in populations of coupled oscillators. Physics D, 143:1–20, 2000

248. K. Sugihara and I. Suzuki. Distributed motion coordination of multiple mobilerobots. In Proceedings of the 5th IEEE International Symposium on IntelligentControl, pages 138–143, Philadelphia, PA, 1990

249. S. Sundar and Z. Shiller. Optimal obstacle avoidance based on the hamilton-jacobi-bellman equation. IEEE Transactions on Robotics and Automation,13:305–310, 1997

250. H. J. Sussmann and W. Liu. Limits of highly oscillatory controls and theapproximation of general paths by admissible trajectories. In Tech. Rep.SYSCON-91-02, Rutgers Ctr. Syst. and Contr., Piscataway, NJ, Feb. 1991

251. J. Sussmann and G. Tang. Shortest paths for the reeds-shepp car: A workedout example of the use of geometric techniques in nonlinear optimal control.In SYCON, Rutgers Ctr. Syst. Contr. Tech. Rep., New Brunswick, NJ, Oct.1991

252. I. Suzuki and M. Yamashita. Distributed anonymous mobile robots: formationof geometric patterns. SIAM Journal on Computing, 28:1347–1363, 1999

253. D. Swaroop and J. Hedrick. String stability of interconnected systems. IEEETransactions on Automatic Control, 41:349–357, 1996

316 References

254. A. Takayama. Mathematical Economics. Cambridge University Press, NewYork, NY, 1985

255. H. G. Tanner, A. Jadbabaie, and G. J. Pappas. Stable flocking of mobileagents, part i: fixed topology. In Proceedings of IEEE Conference on Decisionand Control, pages 2010–2015, Maui, Hawaii, 2003

256. H. G. Tanner, A. Jadbabaie, and G. J. Pappas. Stable flocking of mobile agents,part ii: dynamic topology. In Proceedings of IEEE Conference on Decision andControl, pages 2016–2021, Maui, Hawaii, 2003

257. H. G. Tanner, G. J. Pappas, and V. Kumar. Leader-to-formation stability.IEEE Transactions on Robotics and Automation, 20:433–455, 2004

258. A. R. Teel, R. M. Murray, and G. Walsh. Nonholonomic control systems:from steering to stabilization with sinusoids. In Proceedings of the 31st IEEEConference on Decision and Control, pages 1603–1609, Tucson, Arizona, Dec.1992

259. D. Tilbury, R. M. Murray, and S. S. Sastry. Trajectory generation for the n-trailer problem using goursat normal form. IEEE Transactions on AutomaticControl, 40:802–819, 1995

260. J. Toner and Y. Tu. Flocks, herds, and schools: A quantitative theory offlocking. Physical Review E., 58:4828–4858, 1998

261. J. Tsinias. Existence of control lyapunov functions and applications to statefeedback stabilizability of nonlinear systems. SIAM Journal on Control andOptimization, 29:457–473, 1991

262. R. S. Varga. Matrix Iterative Analysis. Prentice Hall, New Jersey, 1962263. T. Vicsek, A. Czirok, E. B. Jacob, I. Cohen, and O. Shochet. Novel type of

phase transition in a system of self-driven particles. Physical Review Letters,75:1226–1229, 1995

264. M. Vidyasagar. Nonlinear Systems Analysis. Prentice Hall, Englewood Cliffs,NJ, 1978

265. Nguyen X. Vinh. Flight Mechanics of High Performance Aircraft. CambridgeUniversity Press, Great Britain, 1993

266. G. C. Walsh and L. G. Bushnell. Stabilization of multiple input chained formcontrol systems. Systems & Control Letters, pages 227–234, 1995

267. J. Wang, Z. Qu, R. A. Hull, and J. Martin. Cascaded feedback linearizationand its application to stabilization of nonholonomic systems. System & ControlLetters, 56:285–295, 2007

268. K. Wang and A. N. Michel. Qualitative analysis of dynamical systems deter-mined by differential inequalities with applications to robust stability. IEEETransactions on Automatic Control, 41:377–386, 1994

269. P. K. C. Wang. Navigation strategies for multiple autonomous mobile robots.In Proceedings of IEEE/RSJ International Conference on Intelligent Robotsand Systems (IROS), pages 486–493, Tsukuba, Japan, 1989

270. K. Warburton and J. Lazarus. Tendency-distance models of social cohesion inanimal groups. Journal of Theoretical Biology, 150:473–488, 1991

271. K. Y. Wichlund, O. J. Sordalen, and O. Egeland. Control of vehicles withsecond-order nonholonomic constraints: underactuated vehicles. In Proceedingsof European Control Conference, pages 3086–3091, Rome, Italy, 1995

272. B. Wie. Space Vehicle Dynamics and Control. AIAA, Reston, Virginia, 1998273. H. Wielandt. Unzerlegbare, nicht negative matrizen. Math. Z., 52:642–648,

1950

References 317

274. J. Wolfowitz. Products of indecomposable, aperiodic, stochastic matrices. Proc.Amer. Mathematical Soc., 14:733–737, 1963

275. S. Wolfram. Cellular Automata and Complexity. Addison-Wesley, Reading,MA, 1994

276. C. W. Wu. Synchronization in arrays of coupled nonlinear systems with delayand nonreciprocal time-varying coupling. IEEE Transactions on Circuits andSystems II: Express Brief, 52:282–286, 2005

277. C. W. Wu and L. O. Chua. Synchronization in an array of linearly coupled dy-namical systems. IEEE Transactions on Circuits and Systems I: Fundamentaltheory and applications, 42:430–447, 1995

278. H. Yamaguchi. A distributed motion coordination strategy for multiple non-holonomic mobile robots in cooperative hunting operations. Robotics and Au-tonomous Systems, 43:257–282, 2003

279. J. Yang, A. Daoui, Z. Qu, J. Wang, and R. A. Hull. An optimal and real-timesolution to parameterized mobile robot trajectories in the presence of movingobstacles. In Proceedings of IEEE International Conference on Robotics andAutomation, pages 4423–4428, Barcelona,Spain, April 18-22 2005

280. J. Yang, Z. Qu, and R. Hull. Trajectory planning for ugvs in an environmentwith ‘hard’ and ‘soft’ obstacles. In Proceedings of AIAA GNC Conference,Keystone, CO., August 2006

Index

aggregation, 6algebraical multiplicity, see multiplicityalgorithm

biased maximum, 263, 265biased minimum, 266circumcenter, 261gradient descent, 10maximum, 261, 263minimum, 261, 266modified circumcenter, 260

artificial behavior, see behavior

Banach space, 43behavior

alignment, 5, 7artificial, 8cohesion, 5consensus, 200cooperative, 4, 5, 269motor schemas, 9Rendezvous, 201rule-based, 9separation, 5vector consensus, 201virtual leader, 202

boids, 5boundedness

uniform, 49, 52uniform ultimate, 49, 52

Cauchy sequence, 43center of mass, 187

preserving property, see propertychained form, 82

extended, 97power, 96

extended, 97skew-symmetric, 96

circumcenter, 211circumcenter algorithm, see algorithmcollision avoidance, 11, 98, 103, 109,

126, 132, 137, 272, 274criteria, 110

geometric criterion, 110time criterion, 110

consensus, 200, 261vector, 201, 266

constraintdriftless, 12

dynamic, 10holonomic, 11integrable, 11

kinematic, 10motion, 11

network, 10nonholonomic, 11

Pfaffian, 12, 18pliable to, 2, 10, 269

rheonomic, 12scleronomic, 12

with drift, 12, 18contraction mapping, 171, 174

controladaptive, 2

adaptive cooperative, 203backstepping design, 74continuous and dynamic, 92

320 Index

continuous and static, 90, 92continuous and time independent, 92continuous and time varying, 92, 115cooperative, 185, 237, 272cooperative and reactive, 273cooperative formation, 204coverage, 9discontinuous, 92, 115, 123dynamic, 120formation, 126, 204hierarchy, 11hybrid, 125integral cooperative, 203inversely optimal, 122kinematic, 12linear cooperative, 191multi-layer cooperative, 233multi-objective, 136networked, 4optimal, 73reactive, 136, 140, 272recursive design, 73regulation, 115robust, 2smooth, 81Sontag formula, 58stabilization, 115standard, 2time varying, 118, 120, 121, 124tracking, 114, 115vehicle level, 4

controllability, 2, 17, 69, 81, 85canonical form, 72cooperative, 2Gramian, 72Lie algebra, 71, 72linear, 72rank condition, 72small time, 71, 91uniform complete, 73, 81, 92, 95, 117

convergencematrix multiplicative sequence, 169,

178, 179measure, 171

matrix power sequence, 169robust cooperative, 228

convex conepointed, 157solid, 157

convex hull, 171Couzin model, see modelcoverage control, see control

designbackstepping, 16cooperative control

linear, 200nonlinear, 250

cooperative formation control, 204cooperative reactive control, 272optimal control, 77robust cooperative control, 224, 269

digraph, 157distribution, 70

involutive, 70involutive closure, 70, 71regular, 70

dwell time, 61, 68, 69dynamics

internal, 18, 75zero, 75

eigenvalue, see matrixeigenvector, see matrixenergy

kinetic, 13potential, 13

environment, 2robust to changes, 2, 10, 269, 272

equationcostate, 78Hamilton, 15Hamilton-Jacobi-Bellman, 78HJB, 78Lagrange-d’Alembert, 14Lagrange-Euler, 14, 15, 28Lyapunov, 66, 67, 73Newton-Euler, 29Riccati, 73, 116, 119, 122

equilibrium, 1, 48, 186Euler angles, 27

feedback linearization, 18, 75, 81, 87cascaded, 90dynamic, 88full state, 76partial, 87

flocking, 6, 7

Index 321

forcecentrifugal, 16Coriolis, 16

functionCCLF, 216CLF, 58, 216common cooperative control

Lyapunov, 224common Lyapunov, 59, 68, 224composite potential field, 132contraction mapping, 43control Lyapunov, 58, 216cooperative control Lyapunov, 216,

224, 240decrescent, 51energy, 15, 54homogeneous Lyapunov, 125Lyapunov, 51, 78, 240matrix exponential, 62n.d., 51n.s.d., 51negative definite, 51negative semi-definite, 51p.d., 51p.s.d., 51positive definite, 51positive semi-definite, 51potential field, 7, 132quadratic, 66quadratic Lyapunov, 165radially unbounded, 51strictly decreasing over a time

interval, 241strictly increasing over a time

interval, 241strictly monotone decreasing, 51strictly monotone increasing, 51, 226,

241uniformly continuous, 44uniformly nonvanishing, 93, 115, 119uniformly right continuous, 93vanishing, 93, 115

geometrical multiplicity, see multiplicityGerschgorin theorem, see theoremgraph

a global reachable node, 158a spanning tree, 158adjacency matrix, 157

balanced, 157directed, 157loop, 158path, 158strongly connected, 158undirected, 157

groupfollower, 156leader, 156one, 156several, 156

Hamiltonian, 15, 77

inertiamatrix, 15

instability, 49, 52internal dynamics, see dynamics

Jacobi identity, 70

Kronecker product, 40Kuramoto model, see model

Lagrange multiplier, 14, 20, 22, 77Lagrangian, 13, 15, 22

regular, 15latency, 270lemma

Barbalat, 44, 56, 240Lie algebra, 70Lie bracket, 64, 70, 213Lie derivative, 57linear combination, 40linear dependence, 40linear independence, 40linear vector space, 40Lipschitz condition, 43Lyapunov

diagonal matrix, see matrix - diagonalLyapunov matrix

matrix, see matrix - LyapunovLyapunov function, see function -

LyapunovLyapunov function components, 241

Markov chain, 153, 169ergodicity, 153

matrixB(·), 212

322 Index

aperiodic, 170binary, 145characteristic, 174controllability, 72cyclic, 65, 150, 151, 156, 158defective, 40diagonal dominance, 159diagonal Lyapunov matrix, 161,

164–167diagonalizable, 40diagonally positive, 145, 175–177,

180, 186, 259, 260eigenvalue, 39, 62, 148eigenvector, 39, 148geometrical meaning, 156graphical meaning, 157Hessian, 78Hurwitz, 63, 159indecomposable, 170irreducible, 146, 153–156, 158, 186,

195, 261, 265irreducible M-matrix, 163Jordan block, 41Jordan form, 149, 163lower triangular, 146lower triangularly complete, 147, 156,

158, 170, 175, 177, 178, 187, 190,195, 247, 261

lower triangularly positive, 147,175–177, 190

Lyapunov, 161, 162Lyapunov function, 66Lyapunov stable, 162M-matrix, 159Metzler, 152–154, 159multiplicative sequence, 169nonnegative, 145, 154, 186, 237nonsingular M-matrix, 159, 160, 164,

166permutation, 40, 65, 146Perron-Frobenius theorem, 147–149physical meaning, 155piecewise constant, 186, 237positive, 145positive definite, 159positive semi-definite, 159power sequence, 169primitive, 150–153, 156, 170pseudo-diagonal dominance, 162, 164

rank, 164reducible, 146, 154–156regular, 170rotation, 27row stochastic, 186, 237Schur, 65scrambling, 174, 175sensing and communication, 192sequence convergence, see conver-

gencesequentially complete, 179, 238, 243,

247, 249, 254, 257, 260, 261sequentially lower triangular, 177sequentially lower triangularly

complete, 177, 178sequentially scrambling, 174–176SIA, 170, 175, 177, 178singular M-matrix, 159, 161, 166, 239state transition, 62, 63stochastic, 146, 153–155stochastic-indecomposable-aperiodic,

170strict diagonal dominance, 159strict pseudo-diagonal dominance,

161, 165unformly sequentially complete, 180unformly sequentially lower triangu-

larly complete, 180uniform sequence convergence, 179uniformly sequentially lower

triangularly complete, 178uniformly sequentially scrambling,

174, 177, 178unitary, 40

maximum, 187maximum algorithm, see algorithmminimum, 187minimum algorithm, see algorithmmodel

aggregation, 249constraints, 11Couzin, 5dynamic, 13fixed-wing aircraft, 31flocking, 260helicopter, 33kinematic, 12Kuramoto, 8, 250missile, 33

Index 323

munitions, 33reduced order, 15synchronization, 249vehicle, 18Vicsek, 7, 250VTOL aircraft, 33

momentum, 15multiplicity

algebraical, 41geometrical, 41

neighboring rule, 190network, 2

ad hoc sensor, 3connectivity, 209delay, 2, 270feedback, 3intermittent, 2, 269noise, 2, 224, 251sensing/communication, 2, 209sensor, 9wireless communication, 3

norm, 41, 171, 187function, 42matrix, 42series, 42vector

p-norm, 41Euclidean norm, 41

null space, 41

objectivesmultiple, 11

obstacle, see collision avoidanceoptimality, 77

inverse, 79optimization, 10

positive definite, see function - positivedefinite

principled’Alembert, 14Hamilton, 13Lagrange-d’Alembert, 13Lagrange-Euler, 14Newton, 26, 28, 29optimality, 77, 78Pontryagin, 77variational, 13

property∞-norm preserving, 187amplitude dominant on the diagonal,

241, 247, 260center-of-mass preserving, 187maximum preserving, 187, 248, 249,

260minimum preserving, 187, 248, 249,

260mixed quasi-monotone, 46quasi-monotone, 46relative amplitude dominant on the

diagonal, 242, 247, 248relative-amplitude strictly dominant

on the diagonal, 247–249, 260small control, 58

quasi-monotone property, see propertyquaternions, 28

rank, 40, 71, 72rendezvous, 263

output, 263roll-pitch-yaw convention, 27

setL1 space, 42L∞ space, 42Ωe, 186ℜ, 39ℜn, 39ℜn×m, 39ℜ

r1×r2+ , 145

ℜ+, 39ℜn

+, 39ℜ+/0, 39ℵ, 7C, 39σ(A), 39lp space, 42l∞ space, 42M1(c0), 153, 189M2(c0), 153cone, 156convex, 156face, 157invariant, 57proper cone, 157

similarity transformation, 40

324 Index

simplex, 171spectral radius, 39spectrum, 39stability

asymptotic, 49, 63, 155asymptotic cooperative, 169, 188,

200, 241, 243, 247, 249, 254, 257,260, 261

asymptotic output cooperative, 196cooperative, 1, 5, 188exponential, 49, 52, 63global, 49input-to-state, 18, 49, 63ISS, 49, 52local, 49Lyapunov, 49, 52, 63, 155, 241, 243,

247, 249, 254, 257, 260region, 49uniform, 49, 52uniform asymptotic, 49, 52uniform asymptotic cooperative, 188,

238steering control, 98, 115

piecewise-constant, 100polynomial, 102sinusoidal, 99

swarm, 6symbol

=, 145>, 145J, 145≥, 1451, 145⊗, 40ρ(A), 39B(·), 174, 195, 212det(·), 164, 165h.o.t., 212

synchronization, 7, 8, 207system

affine, 48, 256autonomous, 48average, 64, 212, 214, 224cascaded, 74chained form, 81, 82complex, 4constrained, 2, 11continuous-time, 166controllable, 69, 71

cooperative, 1, 224discrete-time, 167, 259feedback form, 73, 253linear, 62linear cooperative, 185minimum phase, 75networked control, 3nonaffine, 257nonautonomous, 48nonholonomic, 81, 262nonlinear cooperative, 237piecewise-constant, 64pliable, 1positive, 154, 166, 167, 258relative degree, 250robust cooperative, 224, 226robust cooperatively convergent, 228standard feedback, 2switching, 59, 64time invariant, 62time varying, 63underactuated, 17, 18uniformly bounded, 228

theoremcomparison, 45, 55, 240cooperative behavior, 190, 195, 199,

200, 214, 223, 238, 243, 247, 249,257, 260, 261

fixed-point, 43Gerschgorin, 159invariant set, 57LaSalle, 57Lyapunov, 52Lyapunov converse, 54, 60robust cooperative behavior, 226,

228, 230, 232time constant, 68, 69topology

balanced, 157, 238bidirectional, 238directed, 157undirected, 157

trajectory planning, 98, 140feasible, 107, 115optimized, 106, 111piecewise-continuous, 100pliable, 103polynomial, 102, 107

Index 325

real-time, 106sinusoidal, 99

transformationcontrol, 75Legendre, 15state, 75time unfolding, 95time-varying and state, 95

variation, 13vector fields

involutive, 70vehicle

n-trailer, 24aerial, 18, 31car-like, 20, 84, 89differential-drive, 18, 83, 89

fire truck, 23, 84front-steering back-driving, 21front-steering front-driving, 22ground, 18mobile robotic, 4space, 18, 25surface vessel, 30tractor-trailer, 23, 84underwater, 30virtual, see virtual leader

Vicsek model, see modelvirtual displacement, 13virtual leader, 4, 202, 209, 230, 233,

266, 272Voronoi diagram, 10

zero dynamics, see dynamics

Documents

Cooperative Control of Dynamical Systems Applications to Autonomous Vehicles