Rotkowitz-when is a Linear Controller Optimal

7/24/2019 Rotkowitz-when is a Linear Controller Optimal

1/57

When is a Linear Controller Optimal?

Michael Rotkowitz

Department of Electrical and Electronic Engineering

Department of Mathematics and StatisticsThe University of Melbourne

DISC Summerschool on Distibuted Control and EstimationNoordwijkerhout, The Netherlands5 June 2009


2/57

2 M. Rotkowitz, Melbourne Uni

General Formulation

P11 P12

P21

G

K

w

uy

z

minimize J(fl(P, K))


3/57


General Formulation

P11 P12

P21

G

K

w

uy

z

minimize J(fl(P, K))

subject to KS


4/57


Uniform Optimal (Centralized) Control

minimize fl(P, K)

where

G = supw=0

Gw2w2

When plantP is linear, optimal controllerK is indeed linear

Feintuch and Francis, 1985 Khargonekar and Poolla, 1986


5/57


LQGminimize E (h (z)) wherez = fl(P, K) w

strictly classical information pattern separation

= F

Gaussian noise linear estimation

F linear

quadratic cost linear control

linear

Witsenhausen, 1971


6/57


Witsenhausen Counterexample (1968)

1

x0 x1

2

x2

y2 u2u1y1

w1

v = w2

+

+

+

+

+

Objective was to seek1, 2 to minimize

E k2u21+x22where

w1 N(0, 1) w2 N(0, 1)


7/57

0 y1

y1 0


W.C. - Full Information

1

x0 x1

2

x2

y2 u2u1y1

w1

v

=w

2

+

+

+

+

+

1 2 E(u21) x2 E(x

22) J

(0) full info 0 y1 0 0 0 0


8/57

0

y1

y2

v


W.C. - No Input Cost

1

x0 x1

2

x2

y2 u2u1y1

w1

v = w2

+

+

+

+

+

1 2 E(u21) x2 E(x

22) J

(0) full info 0 y1 0 0 0 0(1) no input cost 0 y2 0 v 1 1


9/57

y1 0

0 0


W.C. - No Output Cost

1

x0 x1

2

x2

y2 u2u1y1

w1

v = w2

+

+

+

+

+

Letk= 0.1and = 10.

1 2 E(u21) x2 E(x

22) J

(0) full info 0 y1 0 0 0 0

(1) no input cost 0 y2 0 v 1 1(2) no output cost y1 0

2 = 100 0 0 k22 = 1


10/57

ay1 by2


W.C. - Best Linear

1

x0 x1

2

x2

y2 u2u1y1

w1

v = w2

+

+

+

+

+

1 2 E(u21) x2 E(x22) J(0) full info 0 y1 0 0 0 0(1) no input cost 0 y2 0 v 1 1

(2) no output cost y1 0 100 0 0 1(3) best linear 0.01y1 0.99y2 0.01 - 0.9899 0.99


11/57

y1+H sgn(y1) H sgn(y2)

+H, ify10H, ify1


12/57


13/57



1

x0 x1

2

x2

y2 u2u1y1

w1

v = w2

+

+

+

+

+

Objective was to seek1, 2 to minimize

E k2u21+x22where

w1 N(0, 1) w2 N(0, 1)


14/57

P11 P12P21 G

K

w

uy

z



1

x0 x1

2

x2

y2 u2u1y1

w1

v = w2

+

+

+

+

+

Letting

w=

w1w2

z=

k u1

x2

we then seek to minimize

J(1, 2) = Ez22

wherew N(0, I)


15/57

P11 P12P21 G

K

w

uy

z


Witsenhausen Counterexample - Induced

Norm

1

x0 x1

2

x2

y2 u2u1y1

w1

v = w2

+

+

+

+

+

Instead consider the induced norm, and seek1, 2 to minimize

J(1, 2) = sup

w=0

z2

w2


16/57


Induced Norms

supw=0

z2w2


17/57


Induced Norms

supw=0

z2w2

supw

z2w2

18 M R k i M lb U i


18/57


Induced Norms

supw=0

z2w2

supw

z2w2

a = d1

dy1

y1=0

b = d2

dy2

y2=0

1 ay1 2 by2

19 M Rotkowitz Melbourne Uni


19/57


Induced Norms

supw=0

z2w2

supw

z2w2

a = d1

dy1

y1=0

b = d2

dy2

y2=0

1 ay1 2 by2

J(1, 2) = supw=0

z2w2 Jl(a, b)



20/57


Induced Norms

supw=0

z2w2

supw

z2w2

a = d1

dy1

y1=0

b = d2

dy2

y2=0

1 ay1 2 by2

J(1, 2) = supw=0

z2w2 Jl(a, b)

May as well fix

1(y1) = ay1 2(y2) = by2Cant do any better!



21/57


Induced Norms

1 ay1 2 by2

J(1, 2) = supw=0

z2w2

Jl(a, b)

May as well fix1(y1) = ay1 2(y2) = by2

Cant do any better!

Given any nonlinear controller thats differentiable in the origin, wehave a linear controller thats at least as good.

End of story? Not so fast.



22/57

,

Another Induced Norm

L1 / 1 ( )Norm

sup

w=0

zw

(M. Dahleh and J. Shamma, 1992) Nonlinear controllers which aredifferentiable in the origin cannot do better than linear controllers.

But.............



23/57

A. Stoorvogel, 1995

z =

0 3 01.5 1.5 33 0 00 0 3

w+

1111

u

y = w

u = Ky

FindKto minimize

J(K) = sup

w=0

z

w



24/57

A.S. - Unit Cube

For linear controllers, need only con-sider the eight corners of the unitcube.



25/57

A.S. - Outputs


zi = 3 + |ui| fori= 1, 2, 3, 5, 6, 7

z4 = max{|u4 3|, |u4 6|}

z8 = max{|u8+ 3|, |u8+ 6|}



26/57

A.S. - Best Linear


zi = 3 + |ui| fori= 1, 2, 3, 5, 6, 7

z4 = max{|u4 3|, |u4 6|}

z8 = max{|u8+ 3|, |u8+ 6|}

Best linear controller achieves closed-loop norm of 3.75; achieved by

u =

0.75 0.75 0.75

w



27/57

A.S. - Linear Controller



28/57

A.S. - Nonlinear Control

For nonlinear controllers, also con-sider the eight corners of the unitcube.

zi = 3 + |ui| fori= 1, 2, 3, 5, 6, 7

z4 = max{|u4 3|, |u4 6|}

z8 = max{|u8+ 3|, |u8+ 6|}



29/57



zi = 3 + |ui| fori= 1, 2, 3, 5, 6, 7

z4 = max{|u4 3|, |u4 6|}

z8 = max{|u8+ 3|, |u8+ 6|}



30/57




31/57



zi = 3 + |ui| fori= 1, 2, 3, 5, 6, 7

z4 = max{|u4 3|, |u4 6|}

z8 = max{|u8+ 3|, |u8+ 6|}

Best nonlinear controller achieves closed-loop norm of 3.00.



32/57



zi = 3 + |ui| fori= 1, 2, 3, 5, 6, 7

z4 = max{|u4 3|, |u4 6|}

z8 = max{|u8+ 3|, |u8+ 6|}

Best nonlinear controller achieves closed-loop norm of 3.00.

Nonlinear outperforms linear - for a centralized problem!



33/57

Back to Witsenhausen

Consider controllers of the form

u1 =

ay1 ify10

a+y1 ify10u2 =

by2 ify20

b+y2 ify20



34/57

Notation

Q()

ratio z2

w2for a given controller and given noise w1, w2

J()

worst case ratio supw=0 z2w2 for a given controller, i.e. the costassociated with that controller

J

best achievable cost for a given class of controllers



35/57

Back to Witsenhausen

Find best piecewise affine controller

J

pa = inf a,a+ infb,b+ Jpa(a, a+, b, b+)


Pi i Affi C


36/57

Piecewise Affine Cost

Find best piecewise affine controller

J

pa = inf a,a+ infb,b+ Jpa(a, a+, b, b+)

As with linear, only the direction of the noise matters

Jpa(a, a+, b, b+) = sup

Qpa(a, a+, b, b+, )


R i f Diff G i


37/57

1

x0 x1

2

x2

y2 u2u1y1

w1

v = w2

+

+

+

+

+

Regions of Different Gains


R i f Diff G i


38/57

1

x0 x1

2

x2

y2 u2u1y1

w1

v = w2

+

+

+

+

+


Jpa(a, a+, b, b+) = sup Qpa(a, a+, b, b+, )


R i f Diff t G i


39/57


At, we have y2= 0. Thus

sup

Qpa(a, b, b+, ) Qpa(a, b, b+, ) for allb, b+

and so

Jpa(a) = inf b,b+

sup

Qpa(a, b, b+, ) Qpa(a, b, b+, )


S t S d C t ll


40/57

Set Second Controller

Wed like

Qpa(a, b, b+, )

=

= 0

and also2Q2pa(a, b, b+, )

2

=


41/57

Gain vs. Noise for Optimal Controllers

1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 10.5

0

0.5

1

a= a*, a

+= a*, b

= b*(a*), b

+= b*(a*), k =0.1, =10

/

y1,y2,

Qpa

Qpa

(a,a

+,b

,b

+,)

y1/ (2r)

y2/ (2r)

y1= 0

y2= 0


Cost vs First Controller


42/57

Cost vs. First Controller

5 0 50.5

1

1.5

2

2.5

3

3.5

a

Qlb

Qlb(a), with k = 0.1, = 10

Qlb

a*= a

1

a2

(k+1)

1/2

a = 2+ 2 2 /2

where=k22 +k2 + 1 and= 2k.


General Nonlinear Controller


43/57


Given any1, 2: R R, let

a=1(1)

and then let

0 = (a)

and further choose r0 such that

y1(r0, 0) = 1

and

y2(r0, 0) = 0




44/57





45/57


Obvious that

J Jl




46/57


Obvious that

J Jl

Now

Q(1, 2, r0, 0)




47/57


Obvious that

J Jl

Now

Q(1, 2, r0, 0)

= Ql(a, b(a), (a))




48/57


Obvious that

J Jl

Now

Q(1, 2, r0, 0)

= Ql(a, b(a), (a))

= Jl(a, b(a))




49/57


Obvious that

J Jl

Now

Q(1, 2, r0, 0)

= Ql(a, b(a), (a))

= Jl(a, b(a))

which yields

J(1, 2) Jl(a, b(a))




50/57


Obvious that

J Jl

and weve now shown that

J Jl

thusJ = Jl


Redundancy


51/57

Redundancy

Argued that piecewise affine dominates general nonlinear.

Showed that best piecewise affine controller is linear.

Showed that linear dominates general nonlinear.


Redundancy


52/57

Redundancy




Why?

Generalization


Redundancy


53/57

Redundancy




Why?

Generalization

Surprising


Related Work


54/57

t

Separation / Linear Optimality over Lossy Networks

X. Liu, A. Goldsmith

L. Schenato, B. Sinopoli, M. Franceschetti, K. Poola, S. Sastry

(Decentralized) Robust Control

Ian Petersen, Valery Ugrinovskii

1

x0 x1

2

x2

y2 u2u1y1

w1

v = w2

+

+

+

+

+

Linear Suboptimality with Communication Link

Nuno Martins


LTI vs LTV


55/57

unstable fixed modes stabilzable with LTV controllers

Kobayashi, Hanafusa, and Yoshikawa

Anderson and Moore Wang

Surprise coupling and nonlinearity - Allerton 2009


Conclusion(s)


56/57

( )

Very simple question gives rise to extremely rich structure.

Linear controllers are uniformly optimal for the Witsenhausen Coun-

terexample. Many new surprises await!

System Norm Nonlinear Linear Optimal Linear Optimal

Generalization for Centralized? for Decentralized?1

No ??

2 LQG Yes Sometimes (PN/QI)

22 Yes ??


The Witsenhausen Counterexample:


57/57

p

40 Years LaterSpecial Invited Session,IEEE Conference on Decision and Control,

Cancun, Mexico, 9 December 2008

Authors include:

Yu-Chi Larry Ho

Tamer Basar

Demosthenis Teneketzis

Anant Sahai

Nuno Martins

Michael Rotkowitz

Documents

Rotkowitz-when is a Linear Controller Optimal