MODERN CONTROL THEORYLecturer：Qilian Bao鲍其莲

1

Chapter 8 Fundamentals of Optimal

Control

Objectives:

• Optimal control from Hamilton-Pontryagin Equation

• Optimal Control Law for Linear System with Quadratic

Performance Index

• Design procedure and examples

2Chapter 8

8.1 Optimal Control based on Hamilton-Pontryagin

Equation1.The optimal control problem for feedback control systems.

To minimize T

dtUXLJ0

),(

subject to ),()( UXfX t

state vectornt RX )(rt RU )(

),( UXf

),( UXL

the control vector

the function vector(Rn+r→Rn)

Rn+r→Rn, J is a function

2.Target:

Euler-Lagrange Equation Hamilton-Pontryagin Equation

Chapter 8

3.The design approach

Step 1. Rewrite the state equation

0))(),(()( ttt UXfX

Step 2. Set up the augment functional

T TT

T T

dtL

dtLJ

0

0

)]),([

)]),((),([

XΛfΛUX

XUXfΛUX

Define：XΛUXfΛUX TTLF ),(),(

Step 3. Define a scalar function —Hamiltonian

),(),(),,( UXfΛUXΛUXTLH

then

XΛΛUX THF ),,(

Chapter 8

Step 4. Applying Euler-Lagrange Equation to this problem

0

0

0

ΛΛ

UU

XX

F

dt

dF

F

dt

dF

F

dt

dF

Step 5. By substituting into the equation above

0),,,(

0),,(

0),,(

Λ

XUΛXU

UΛX

ΛX

UΛX

F

Hdt

dH Furthermore

f(X,U)XU

HX

H(t)Λ

0

Hamilton-Pontryagin equation------The necessary condition

for optimizing control systems.

Chapter 8

Example Consider the optimal control problem

Min T

dttutxJ0

22 ))()((2

1

s.t )()()( tutxtx

boundary condition : x(0)=1, x(T)=0

Our task: To find the optimal control law u*(t).

Step 1. Rewrite the equation as

0)()()( tutxtx

Step 2. Set up the augment functional

T

dtxuxuxJ0

22 )](2

1

2

1[

Step 3. Write Hamiltonian

uxuxH 22

2

1

2

1

Chapter 8

Step 4. From H-P equation, we have

)()()(

0)()(

)()()(

tutxtx

ttu

ttxt

Above equation set can be rewritten as

)(

)(

11

11

)(

)(

t

tx

t

tx

Step 5. Solve the equation

011

11

IA

The characteristic is 22,1

Chapter 8

Then the solution is

)ee)((ee)t(x tttt 2222 01212 22

1

tttt eeeet 2222 1212)0(22

1)(

The optimal control u*(t) is

tttt eeeettu 2222 1212)0(22

1)(*)(*

where

TT

TT

ee

ee

22

22 1212)0(

Chapter 8

8.2 Optimal Control Law for Linear System with

Quadratic Performance Index

1.Description of the problem

dt(t)RU(t))U(t)QX(t)(XJTT

02

1

s.t B(t)U(t)A(t)X(t)(t)X

where J is The cost functional (or say performance functional).

Q is a symmetric constant positive semi-definite weighting matrix

R is a symmetric constant positive definite weighting matrix.

Find optimal control law to minimize the cost functional J.

minU

Chapter 8

2 Design approach------Riccati Differential Equation

Construct an augment functional

(t)]dtX(t)ΛB(t)U(t))t)(t)(A(t)X(ΛRU)UQX[(XJTTTT

0 2

1

The Hamiltonian 1 1

2 2

T T T TH(X,Λ,U) X QX U RU Λ AX Λ BU

According to Hamilton-Pontryagin Equation, get the optimizing

condition

)()(Λ t(t)ΛAQX(t)H(X,Λ(X,X

tT

0)()()()(

)(

ttt

t

H TΛBRU

U

U,X,

)()()()()( ttttt UBXAX Since

so )()()()()()( 1 tttttt TΛBRBXAX

)(t(t)ΛBRU(t)T1

Chapter 8

Suppose )()()( ttt XPΛ ),()( nnt P

From H-P equation, we have

)()()()()()()()( tttttttt TXPAQXXPXP

)()()()( 1 tttt TXPBRU

)()()()()()()( ttttttt TXPBRBXAX

1

Through substituting

)())()(()()()()()()[()( tttttttttt TTXPAQXPBRBAPP

1

QtPtBRtBtPtPtAtAtPtP TT )()()()()()()()()( 1

The Riccati Differential Equation

Chapter 8

The optimal control law

)()(*)()(* tttt TXPBRU

1

where )(tP is the solution of Riccati Equation.

Let )()(*)( tttTΚPBR

1

The optimal control law can be written as

)()()( ttt XKU

)()()( ttt T PBRK1

Chapter 8

3. Riccati Matrix Equation (R.E. Kalman, 1960)

For LTI system i.e., matrices A and B are constant,

0)( tP

Riccati Differential Equation deduces to Riccati Matrix Equation

0 QPBPBRPAPA

1 TT

A very famous equation----Riccati Equation

The optimal control

)(

)(**

t

tT

KX

XPBRU1

4.The symmetric solution of Riccati Equation

By transposing Riccati equation:

0 QPBBRPAPPA

1 TTTTTT

Through comparing TPP

Chapter 8

The Flow-chart for Design

Set up the mathematical model for the linear or

linearized system under consideration

)()()( ttt BUAXX

Input the data of the system to form matrices A and B

Choose the quadratic performance index

and the weighting matrices Q and R.

0)(

2

1dtJ TT

RUUQXX

Chapter 8

Check if the system controllable:

Matrix D=[B AB A2B …An-1B] possesses full rank .

If yes

Solve the Riccati equation

P*0QPBPBRPAPA1 TT

Calculate the feedback gain matrix

K*=R-1BTP*

Design the optimal controller according to

U*=-K*X(t)

Test the stability & dynamic performance of the

controlled system

Chapter 8

Example1

uxx

dtuxJu

)(2

1min 2

0

2

How to solve the optimal control law?

Solution:A=(1) B=(1) Q=(1) R=(1)

Riccati equation:01 QPBPBRPAPA TT

can be written as:012 2 PP

rankD=rank[B|AB|…|An-1B]=rank[1]=1

(A,B) is controllable, so Riccati Equation has a positive-definite solution

In fact :

12

12

*1*

*

PBRK

P

T

The optimal control law:

xxKU )12(**

Chapter 8

Example2

Controller

dt dtu 2x

1x

The model

ux

xx

2

21

The matrix formBUAXX

uUBA

1

0

00

10

The optimal control problem

BUAXXts

dtUUQXXJ TT

u

.

)(2

1min

0

Where 1)0(

0

01

RQ

Chapter 8

Riccati Equation

01 QPBPBRPAPA TT

Judgment

201

10

rank

ABBrankrankD

So the Riccati equation has a positive definite solution

21

12

02001

00

01]10][1[

1

0

00

10

01

00

2212

1211

2

2212221211

2

12

2212

1211

2212

1211

2212

1211

2212

1211

pp

ppP

pppppp

pp

pp

pp

pp

pp

pp

pp

pp

The optimal control law

21

1* 2xxPXBRKXU T

Chapter 8

• Exercise: Find the optimal control u*(t) to minimize J:

19

u-

1

0

10

10xx

0

1)0(x

tuxxJ T d0

01

0

2

≥0

• Exercise: Find the optimal control u*(t) to minimize J:

• Solution:

• Let:

•

20

u-

1

0

10

10xx

0

1)0(x

tuxxJ T d0

01

0

2

≥0

2]rank[ ABBQc

2221

1211

PP

PPP

131

13

P

0 QPBBRPPAAP

1 TT

3

2

1

0

1

131

1301

2

1)0()0(

2

1xPx

T*J

02

3* J 1* J1

现代控制理论 21

3

2

1

0

1

131

1301

2

1)0()0(

2

1xPx

T*J

1当 时， ；当 时， 。02

3* J 1* J

Design process

Establish control

goals

Identify the

variable

to control and the

manipulating

variables

Construct the

system model

Determine the

performance

specifications

Select sensors

and actuators

Simulation study

and validation

Select a

controller and

adjust the

controller

parameters

Chapter 8

MATLA B

Homework:

• Learn and practice functions related to :

1. LQR controller

2. Solution to Ricatti equation

23Chapter 8

Summary

• Principle of optimal control

• Hamilton-Pontryagin Equation

• Optimal Control Law for Linear System with Quadratic

Performance Index

• Design procedure and examples

24Chapter 8

),( tux,fx

)()( 00

tttt

xx

0

L(x,u, )dft

t

J t t MIN

)(

)(

)(

)(2

1

t

t

t

t

n

λ

0

{L(x,u, ) λ ( )[f (x,u, ) x]}dft

T

t

J t t t t

),,()(),,(),,,( ttttH TuxfλuxLλux

0

[ (x,u,λ, ) λ ( )x]dft

T

t

J H t t t

0 0

(x,u,λ, )d λ ( )x df ft t

T

t t

H t t t t

00 0

(x,u,λ, )d λ ( )x λ ( )x df ff

t ttT T

tt t

J H t t t t t

0δ Juuu δ)()( tt

xxx δ)()( tt

)(δ)()( fff ttt xxx

0

δ δx δu λ δx d 0x u

f

T Tt

T

t

H HJ t

0

δ λ δx δu d 0x u

f

T Tt

t

H HJ t

xλ

H λx

f

x

L

xλ

H

0

u

H 0

λ

u

f

u

L

Chapter 8