Moncrief-O’Donnell Chair, UTA Research Institute (UTARI)The University of Texas at Arlington, USA

and

F.L. Lewis, NAI

Talk available online at http://www.UTA.edu/UTARI/acs

Cooperative Control of Multi‐Agent Systemson Communication Graphs

Supported by :NSFONRARO/TARDEC

Qian Ren Consulting Professor, State Key Laboratory of Synthetical Automation for Process Industries,

Northeastern University, Shenyang, China

China Qian Ren Program, NEUChina Education Ministry Project 111 (No.B08015)

It is man’s obligation to explore the most difficult questions inthe clearest possible way and use reason and intellect to arriveat the best answer.

Man’s task is to understand patterns in nature and society.

The first task is to understand the individual problem, then toanalyze symptoms and causes, and only then to designtreatment and controls.

Ibn Sina 1002-1042(Avicenna)

Patterns in Nature and Society

Many of the beautiful pictures are from a lecture by Ron Chen, City U. Hong KongPinning Control of Graphs

1. Natural and biological structures

J.J. Finnigan, Complex science for a complex world

The internet

ecosystem ProfessionalCollaboration network

Barcelona rail network

Airline Route Systems

2. Motions of biological groups

Fishschool

Birdsflock

Locustsswarm

Firefliessynchronize

Herd and Panic Behavior During Emergency Building Egress

Helbring, Farkas, Vicsek, Nature 2000

1. Random Graphs – Erdos and Renyi

N nodesTwo nodes are connected with probability p independent of other edges

m= number of edges

There is a critical threshold m0(n) = N/2 above which a large connected component appears – giant clusters

Phase Transition

J.J. Finnigan, Complex science for a complex world

Poisson degree distributionmost nodes have about the same degreeave(k) depends on number of nodes

Connectivity- degree distribution is PoissonHomogeneity- all nodes have about the same degree

kave k

P(k)

Watts & Strogatz, Nature 1998

2. Small World Networks- Watts and Strogatz

Start with a regular latticeWith probability p, rewire an edge to a random node.

Connectivity- degree distribution is PoissonHomogeneity – all nodes have about the same degree

Small diameter (longest path length)Large clustering coeff.- i.e. neighbors are connected

Phase TransitionDiameter and Clustering Coeficient

Clustering coefficient

Nr of neighbors of i = 4Max nr of nbr interconnections= 4x3/2= 6Actual nr of nbr interconnects= 2Clustering coeff= 2/6= 1/3

i

12

34

3. Scale-Free Networks– Barabasi and Albert

Nonhomogeneous- some nodes have large degree, most have small degreeScale-Free- degree has power law degree distribution

Start with m0 nodesAdd one node at a time:

connect to m other nodes

with probability

i.e. with highest probability to biggest nodes(rich get richer)

1( )( 1)

i

jj

dP id

2

3

2( ) mP kk

0 50 100 150 200 250 300 350 400 4500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

4. Proximity Graphs

Randomly select N points in the planeDraw an edge (i,j) if distance between nodes i and j is within d

2d

x

y

When is the graph connected?for what values of (N,d)

What is the degree distribution?

Mobile Sensor Networks Project with Singapore A-Star

The Power of Synchronization Coupled OscillatorsDiurnal Rhythm

1

2

3

4

56

Diameter= length of longest path between two nodes

Volume = sum of in-degrees1

N

ii

Vol d

Spanning treeRoot node

Strongly connected if for all nodes i and j there is a path from i to j.

Tree- every node has in-degree=1Leader or root node

Followers

Communication Graph

Communication Graph1

2

3

4

56

N nodes

[ ]ijA a

0 ( , )ij j i

i

a if v v E

if j N

oN1

Noi ji

jd a

Out-neighbors of node iCol sum= out-degree

42a

Adjacency matrix

0 0 1 0 0 01 0 0 0 0 11 1 0 0 0 00 1 0 0 0 00 0 1 0 0 00 0 0 1 1 0

A

iN1

N

i ijj

d a

In-neighbors of node iRow sum= in-degreei

(V,E)

i

Dynamic Graph- the Graphical Structure of ControlEach node has an associated state i ix u

Standard local voting protocol ( )i

i ij j ij N

u a x x

1

1i i

i i ij ij j i i i iNj N j N

N

xu x a a x d x a a

x

( )u Dx Ax D A x Lx L=D-A = graph Laplacian matrix

x Lx

If x is an n-vector then ( )nx L I x

x

1

N

uu

u

1

N

dD

d

Closed-loop dynamics

i

j

[ ]ijA a

1

2

3

4

56

Theorem. Graph contains a spanning tree iff e-val at is simple.

Graph strongly connected implies exists a spanning tree

Then 2 0

Then -L has one e-val at zero and all the rest stable

1 0

1 1

( ) (0) (0) (0)i i

N Nt tLt T T

i i i ij j

x t e x v e w x w x e v

Consensus Value and Convergence Rate

x Lx Closed-loop system with local voting protocol

Modal decomposition

Let be simple. Then for large t1 0

2 1 22 2 1 1 2 2

1( ) (0) (0) (0) 1 (0)

Nt t tT T T

j jj

x t v e w x v e w x v e w x x

2 determines the rate of convergence and is called the FIEDLER e-value

1 0

and the Fiedler e-val 2There is a big push to find expressions for the left e-vector for

Let graph have a spanning tree. Then all nodes reach consensus.

Consensus: Final Consensus Value

x Lx Closed-loop system with local voting protocol

Modal decomposition

1

1

1 1 11

1 (0) 1( ) (0) (0)

1 (0) 1

Nt T

N j jj

N

xx t v e w x x

x

Let be simple. Then at steady-state1 0

with 1 1T

Nw the normalized left e-vector of 1 0

Therefore 1

( ) (0)N

i j jj

x t x

for all nodes CONSENSUS

1 1

( ) (0) (0) (0)i i

N Nt tLt T T

i i i ij j

x t e x v e w x w x e v

Importance of left e-vector of 1 0

Consensus value depends on communication graph structure

1( ) (0)

N

i j jj

x t x

= weighted average of initial conditions of nodes

A graph is balanced if in-degree=out-degree

Balanced means that row sum= column sum

[ ]ijA a

1

N

i ijj

d a

Row sum= in-degree

1

No

i iji

d a

Column sum= out-degree

Then L has row sum=0 and column sum=0

1 0Tw L means that left e-vector is also 1

11

1w

Consensus value is

1

1( ) (0)N

i jj

x t xN

= average of initial conditions of nodes

Then Consensus value is

i

i

Consensus value depends on communication graph structure

Independent of graph structure

Undirected Graphseij is an edge if eji is an edge, and eij = eji

1

2

3

4

56

0 1 1 0 0 01 0 1 1 0 11 1 0 0 1 00 1 0 0 0 10 0 1 0 0 10 1 0 1 1 0

A

A is symmetric

L=D-A is symmetric and positive semidefinite

A symmetric graph is balanced

Connected undirected graph has average consensus value

Row sum= column sum so that in-degree= out-degree

1

2 3

4 5 6

12

3

4 5

6

Graph Eigenvalues for Different Communication Topologies

Directed Tree-Chain of command

Directed Ring-Gossip networkOSCILLATIONS

Graph Eigenvalues for Different Communication Topologies

Directed graph-Better conditioned

Undirected graph-More ill-conditioned

65

34

2

1

4

5

6

2

3

1

Synchronization on Good Graphs

Chris Elliott fast video

65

34

2

1

1

2 3

4 5 6

Synchronization on Gossip Rings

Chris Elliott weird video

12

3

4 5

6

Dynamic Graph- the Graphical Structure of ControlEach node has an associated state i ix u

Standard local voting protocol ( )i

i ij j ij N

u a x x

1

1i i

i i ij ij j i i i iNj N j N

N

xu x a a x d x a a

x

( )u Dx Ax D A x Lx L=D-A = graph Laplacian matrix

x Lx

If x is an n-vector then ( )nx L I x

x

1

N

uu

u

1

N

dD

d

Closed-loop dynamics

i

j

[ ]ijA a

FlockingReynolds, Computer Graphics 1987

Reynolds’ Rules:Alignment : align headings

Cohesion : steer towards average position of neighbors- towards c.g.Separation : steer to maintain separation from neighbors

( )i

i ij j ij N

a

Distributed Adaptive Control for Multi‐Agent Systems

Sun Tz bin fa孙子兵法

500 BC

Nodes converge to consensus heading

( )ci

i ij j ij N

a

Consensus Control for Swarm Motions

heading angle

Convergence of headings

1

2

3

4

56

0 5 10 15 20 25 30 35 400

1

2

3

4

5

6

-350 -300 -250 -200 -150 -100 -50 0 50-300

-250

-200

-150

-100

-50

0

50

cossin

i i

i i

x Vy V

time x

y

Chris McMurrough Simulation Series for Swarm Motion

Causes Unstable Formation

( )ci

i ij ij j ij N

a

Trust-Based Control: Swarms/FormationsMalicious Node

Divergence of trust Divergence of headings

1

2

3

4

56

Node 5 injects negative trust values

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5-20

-15

-10

-5

0

5

10

15

0 0.5 1 1.5 2 2.5 30

5

10

15

20

25

30

35

40

-30 -25 -20 -15 -10 -5 0 5 10 15 20-25

-20

-15

-10

-5

0

5

10

15

20

25

Internal attackMalicious node puts out bad trust values

i.e. false informationc.f. virus propagation

Trust-Based Control: Swarms/FormationsCUT OUT Malicious Node

heading angle

1

2

3

4

56

Node 5 injects negative trust values

If node 3 distrusts node 5,Cut out node 5

Convergence of trust0 5 10 15 20 25 30 35 40

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Other nodes agree that node 5 has negative trust

Restabilizes FormationConvergence of headings

0 5 10 15 20 25 30 35 400

1

2

3

4

5

6

Node 5

-400 -350 -300 -250 -200 -150 -100 -50 0 50 100-400

-350

-300

-250

-200

-150

-100

-50

0

50

Node 5

( )ci

i ij ij j ij N

a

Work by Sajal Das

Leader

Followers

0 5 10 15 20 25 30-120

-100

-80

-60

-40

-20

0

20

40

60

Time

Hea

ding

Heading Consensus using Equations (21) and (22)

Nodes converge to heading of leader

Consensus Control for Formations

heading angle

Formation- a Tree network

Convergence of headings

10 20 30 40 50 60 70 80 90 100-200

-150

-100

-50

0

50

100

150

200

Heading Update using Spanning Tree Trust Update

x

y

Leader

y

x

( )ci

i ij j ij N

a

cossin

i i

i i

x Vy V

time

leader

Second Order Consensus – Kevin Moore and Wei Ren

Let each node have an associated state i ix u

Second-order local voting protocol

0 1( ) ( )i i

i ij j i ij j ij N j N

u a x x a x x

x Lx Consensus works because the closed-loop system is Type I.

has an integrator- simple e-val at 0.

Closed-loop system

0 1

0 Ix x

L L

Reaches consensus in both iff graph has a spanning tree and gains are chosen for stability

Has 2 integrators- Can follow a ramp consensus input

,i ix x

Second Order Controlled Consensus for Position Offset Control – Kevin Moore and Wei Ren

node state i ix u

Second-order controlled protocol

0 1 0 0 0( ) ( ) ( ) ( )i i

i ij j i ij ij j i i i i ij N j N

u a x x a x x b x x x x

where node 0 is a leader node.

ij is a desired separation vector

Good for formation offset position control

Leader

Followers

ij

x0

v

Controlled Consensus: Cooperative Tracker

Node state i ix uLocal voting protocol with control node v

( ) ( )i

i ij j i i ij N

u a x x b v x

( ) 1x L B x B v

i i

i i ij i ij j ij N j N

u b a x a x b v

0ib If control v is in the neighborhood of node i

Control node is in some neighborhoods iN

{ }iB diag b

Theorem. Let at least one . Then L+B is nonsingular with all e-vals positiveand -(L+B) is asymptotically stable

0ib

So initial conditions of nodes in graph A go away.Consensus value depends only on vIn fact, v is now the only spanning node

Get rid of dependence on initial conditions control node vRon Chen

Strongly connected graph L

1

2

3

4

A = [0 0 0 0;1 0 1 0;0 0 0 1;1 0 0 0];

Lamda’s = 0 1 1 2Consensus time approx 7.5 sec

0 2 4 6 8 10 12 14 16 18 200

1

2

3

4

5

6

7

8

9

Time

Sta

tes

with

diff

. Ini

. con

d.

Controlled Consensus

Average of ICS

Original network

Controlled network L

Lamda’s = 0 1 1 1 2

0 2 4 6 8 10 12 14 16 18 200

1

2

3

4

5

6

7

8

9

Time

Sta

tes

with

diff

. Ini

. con

d.

Consensus time approx 8 sec

Leader’s IC

1

2

3

4

74

Balancing HVAC Ventilation Systems

SIMTech 5th floor temperature distribution

Work with SIMTech – Singapore Inst. Manufacturing Technology

Automated VAV control system

AHUFan

C 1 C 2

CWRCWS

Air Flow

Diffuser outlets

VSD

Control Panel

Control stationVAV box

Room thermostat

Air diffuser

LEGENDS

Extra WSN temp. sensors

SIMTech

( 1) ( ) ( ) ( )i i i ix k x k f x u k

1( ) ( ) ( ( ) ( ))1

i

i i ij j ij Ni

u k k a x k x kn

1 12 4( ) 1, , ,...i k

Control damper position based on local voting protocol

Temperature dynamics

Unknown fi(x)

Under certain conditions this converges to steady-state desired temp. distribution

Adjust Dampers for desired Temperature distributionSIMTech

Open Research Topic - HVAC Flow and Pressure control

Herd and Panic Behavior During Emergency Building EgressWork with Bari Polytechnic University, Italy (David Naso) – comes April 24

Helbring, Farkas, Vicsek, Nature 2000

Crowd Panic Behavior

Modeling Crowd Behavior in Stress SituationsHelbring, Farkas, Vicsek, Nature 2000

Radial compressionterm

Tangential frictionterm

Consensus term Interaction pot. fieldWall pot. field

Repulsive force

Simulated Crowd Panic BehaviorChris McMurrough

Synchronization : Ron Chen – Pinning Control

Leader node dynamics

Connected undirected graphs

ii ia d

i ij jij i j i

d a a

In-degree = out-degree

Diffusivity condition

0( ) ( ) ( )i i ij j i i ij

x f x c a C x x cbC x x

Pinning Control – inputs to some nodes

1ib if node i is pinned

Results –Node motions synchronize if is stable

Pin to the biggest node = highest degree node= highest social standing- c.f. Baras

0( ) ( ) 1x f x c L B Cx cB C x

( )i

f x c Cx

L B D B A has e-vals i

Must have control gain c big enough

x0(t)

( )iB diag b

0 0( )x f x

( ) ,i i i i ix f x u y Cx

Synchronization of Chaotic node dynamics – Ron Chen

Chen’s attractornode dynamics

Pinning control of largest node(for increasing coupling strengths)

c=0 c=10

c=15

c=20

The cloud-capped towers, the gorgeous palaces,The solemn temples, the great globe itself,Yea, all which it inherit, shall dissolve,And, like this insubstantial pageant faded,Leave not a rack behind.

We are such stuff as dreams are made on, and our little life is rounded with a sleep.

Our revels now are ended. These our actors, As I foretold you, were all spirits, and Are melted into air, into thin air.

Prospero, in The Tempest, act 4, sc. 1, l. 152-6, Shakespeare

Swarm Stability AnalysisGazi & Passino 2003, IEEE TAC

( )i i jj i

x g x x

2

( ) ( expy

g y y a bc

c.g. motion is invariantAll agents converge to c.g.form a hyperball of

constant radius and increasing density

LocustSwarm

2

( ) ( expy

g y y a bc

Results of Gazi and Passino

1. Center of gravity of swarm is stationary

1

1 ( ) 0N

ii

d dx x tdt dt N

2. All states converge in finite time to the region

3. Let nodes have finite body size of sphere with radius

Then all states converge to the region

and the final density is

( 1)i

b N bx xaN a

and the final density is 3

3

34

a Nb

1/3ix x N

3

34

Synchronization : Ron Chen – Pinning Control

Leader node dynamics

Connected undirected graphs

ii ia d

i ij jij i j i

d a a

In-degree = out-degree

Diffusivity condition

0( ) ( ) ( )i i ij j i i ij

x f x c a C x x cbC x x

Pinning Control – inputs to some nodes

1ib if node i is pinned

Results –Node motions synchronize if is stable

Pin to the biggest node = highest degree node= highest social standing- c.f. Baras

0( ) ( ) 1x f x c L B Cx cB C x

( )i

f x c Cx

L B D B A has e-vals i

Must have control gain c big enough

x0(t)

( )iB diag b

0 0( )x f x

( ) ,i i i i ix f x u y Cx

Synchronization of Chaotic node dynamics – Ron Chen

Chen’s attractornode dynamics

Pinning control of largest node(for increasing coupling strengths)

c=0 c=10

c=15

c=20

Synchronization on Gossip Rings with Leader

Chris Elliott video #3

The cloud-capped towers, the gorgeous palaces,The solemn temples, the great globe itself,Yea, all which it inherit, shall dissolve,And, like this insubstantial pageant faded,Leave not a rack behind.

We are such stuff as dreams are made on, and our little life is rounded with a sleep.

Our revels now are ended. These our actors, As I foretold you, were all spirits, and Are melted into air, into thin air.

Prospero, in The Tempest, act 4, sc. 1, l. 152-6, Shakespeare

0A with row sum positive= di

L D A

Zhihua Qu 2009 book

is an M matrix

00

M

Off-diagonal entries 0Principal minors nonnegative – singular M matrix

Nonsingular M matrix if all principal minors positive

L also has all row sums = 0

Do not confuse with stochastic matrix

0E is row stochastic if all row sums =1

LtE e is row stochastic with positive diagonal elements

0E be row stochastic. Then A=I-E is an M matrix with row sums zero.Let

Discrete-time voting protocol gives stochastic c.l. matrix 1k kx Ex 1 1( ) ( ) ( )E I D I A I I D L

F is said to be lower triangularly complete if in every row i there exists at least one

is a scalar, it is equal to 0.) is square and irreducible. (note- if

Matrix E is reducible if it can be brought by row/column permutations to the form* 0* *

Two matrices that are similar using permutation matrices are said to be cogredient.

Irreducibility

A graph G(A) is strongly connected iff its adjacency matrix A is irreducible.

11

21 22

1 2

0 00T

p p pp

FF F

F TET

F F F

iiF iiF

A reducible matrix E can be brought by a permutation matrix T to the lower block triangular (LBT) Frobenius canonical form

where

j i0ijF

0,ijF j i

such that (i.e. it has least one nonzero entry).

.

F is said to be lower triangularly positive if

F is lower triang. Complete iff the associated graph has a spanning tree

Then is an eigenvalue of E iff all row sums are equal to 0.Moreover, let E be irreducible with all row sums equal to zero. Then

It is strictly diagonally dominant if these inequalities are all strict. E is strongly diagonally dominant if at least one of the inequalities is strict [Serre 2000]E is irreducibly diagonally dominant if it is irreducible and at least one of the inequalities is strict.

Let E be a diagonally dominant M matrix (i.e. nonpositive elements off the diagonal, nonnegative elements on the diagonal).

[ ] n nijE e R ii ij

j ie e

0

Matrix is diagonally dominant if, for all i,

has multiplicity of 1.

Diagonal Dominance

0

Thm. Let E be strictly diagonally dominant or irreducibly diagonally dominant. Then E is nonsingular. If in addition, the diagonal elements of E are all positive real numbers, then

Re ( ) 0, 1i E i n

SO. Let graph be irreducible. Then the Laplacian L has simple e-value at 0.Add a positive number to any diagonal entry of L to get .

Then is nonsingular and is stable.L

L L

Thm. Properties of Irreducible M matrices.

Let E=sI-A be an irreducible M matrix, that is,

5. exists and is nonnegative for all nonnegative diagonal matrices D with at least one positive element.

6. Matrix E has Property c.7. There exists a positive diagonal matrix P such that

Then,1. E has rank n-1.2. there exists a vector v>0 such that Ev=0.3. implies Ex=0.4. Each principal submatrix of order less that n is a nonsingular M matrix

0A

0Ex

1( )D E

TPE E PThat is, matrix E is pseudo-diagonally dominant, That is, matrix -E is Lyapunov stable.

and is irreducible.

is positive semidefinite, Used for Lyapunov Proofs

Interpret E as the Laplacian matrix L

Zhihua Qu Book 2009

Original network to be controlledHow to pick

Injection nodes?

u1

u2 u3 External control nodes added

Lei Guo – Soft controlDo not change the local protocols of the nodescan only add additional neighbors to influence existing nodes

Extension of virtual leader approach

11 12 1 11 12 1

21 21 2 ( )

1 2 1

[ ]

N m

m N N mij

N N NN N Nm

a a a b b ba b b

A A B a R

a a a b b

1 2 1{ : 1, } { , , , , , , }i N mx i N m x x x u u

Add m additional control nodes[ ] 0ijB b 0ijb is the weight from control node uj to existing network node vi. where

Augmented state

New connectivity matrix

Row sum = id

11 12 11 11 12 1

21 221

11 2

mN

m

N NmN N N NN

b b bd a a ab ba

D A B

b ba a d a

Laplacian

L D A

Modified local voting protocol

( )i

i ij j ij N

a x x

{ }iD diag d

1,i ij

j N md a

A

modified diagonal matrix of in-degrees

with the i-th row sum of , which includes the new control nodes.

SOFT CONTROL, includes new control nodes in some nbhds

N NL D A R

x Lx Bu

Modified Laplacian

New closed-loop system

Row sum of [ ]L B D A B is zero

But i-th row sum of L D A

i i id d where i = i-th row sum of control matrix B.

has been increased by i

i ix Lei Guo

1 11 12 1

21 2 22

1 2

[ ]

N

N Nij

N N N NN

L L LL L

L D A L L R

L L L

i = i-th row sum of control matrix B.

Lemma Let L have row sum zero and be irreducible.At least one 0i

Then 1 11 12 1

21 2 22

1 2

[ ]

N

N Nij

N N N NN

L L LL L

L D A L L R

L L L

is irreducibly diagonally dominant and hence nonsingular.

Lemma. Then is asymp. stable.L

RTP 1. Let { } , 0i iL diag L

Then relate eigenvalues of to those of L . They are shifted right by some amount.

and L be irreducible

L

Special case. ,i c i

Then all e-vals are shifted right by c

Conjecture. Let L be irreducible. Then L

{ }idiag Define

1 11 12 1

21 2 22

1 2

[ ]

N

N Nij

N N N NN

L L LL L

L D A L L R

L L L

i = i-th row sum of control matrix B.

1 11 12 1

21 2 22

1 2

[ ]

N

N Nij

N N N NN

L L LL L

L D A L L R

L L L

My Theorem

Let

{ }idiag

N NL R

1 2

1 2 1 21 21 1

j j js

j j j j j js ss

ki i i

i i i i i is j j j k

L L L L

Then the determinant of is given by

Define

Minor with rows and columns struck out

11 1 1 11 1 1 11 1

1 1 1

1 1 1

0

0

i N i N N

i i ii i ii i i

N Ni NN N Ni NN N NN

ii i

L L L L L L L LL L L L L

L L

L L L L L L L L

L L

Example- one diagonal entry positivei

To increase the determinant as much as possible-Add to the node with the largest OUT-degree, i.e. largest column sumi

We call the out-degree of node i its influence or social standing. - John Baras

Original network to be controlledwith fixed structure

Control graphwith desired structure

0 0x xL Bu u

Overall Dynamics

Does not reach consensus unless matrix is irreducible.

00 00 0 0 0

aug A BL B DL

augL irreducible iff m=1

Add control graph 000 0

augG G

A BL B DL

GL D

0 G

x xL Bu uL

Induced Strogatz Small World StructureReduced diameter= longest path length, larger Fiedler e-val, so faster

has m e-vals at 0

Original network to be controlled

u1u2 External control nodes addedy1y2

Control outputs

Results. To make the controlled network as fast as possible,

Tap into the node with the LARGEST out-degree (highest social standing)

And take measured outputs from the nodes with the SMALLEST out-degree

- Zhihua Qu