Lecture 4 Maximum Principle

8/8/2019 Lecture 4 Maximum Principle

1/21

LECTURE 4

Maximum Principle


2/21

Pontryagin's maximum principle is used inoptimal control theory to find the best possible

control for taking a dynamic system from one

state to another, especially in the presence of

constraints for the state or input controls. One

of these methods is the max principle which

was initially proposed by Pontryagin. It was

formulated by the Russian mathematician LevSemenovich Pontryagin and his students. It has

a general case as the Euler-Lagrangian

equation of the calculus of variations.


3/21

Example: Planners problem Maximization of consumption over time

Notation:

K(t) = capital (only factor of production)

Q(K) =well-behaved output function i.e. Q K)>0, and Q(K)0

C(t) = consumption

I(t) = investments = Q(K) C(t)

H = constant rate ofdepreciation of capital.r = constant rate ofinterest of capital

We are not looking for a single optimal value C*, but for values C(t) that produce an

optimal value for the integral (or aggregate discounted consumption over time).

T

rt dttCe

0

)(

0x

x

K

Q0

2

2

x

x

K

Q

Max

subject to Q = Q(K) with

change in the capital stock

and

HKCQK !y


4/21

The principle states informally that the Hamiltonian

(1-23)

where(t) is a vector of co-state variables of the same dimension as the state

variablesx(t).

must be minimized over the set of all permissiblecontrols U, as seen in the second Euler-Lagrange

equation.

Which is the necessary condition for optimum value.

),,(),,(),,,( tuxftuxLtuxH TPP !

.

),,,(

x

tuxH

x

x

!

y PP

u

tuxH

x

x!

),,,(0

P


5/21

First Pontryagin proved that a necessary conditionfor solving the optimal control problem is that the

control should be chosen so as to optimize theHamiltonian; that is:

(1-24)

In case of closed and bounded control region, the optimalcontrol is found by optimizing w.r.t uin the given control region U, while treating other variables asif they are constants.

Therefore, is the admissible control vector for which

is maximum or (minimum).

]t,[tt,),,,(),,,( f0e UutuxHtuxH PP

),,( txu P

),,,( tux P

),,( txu P

),,,( tux P


6/21

Then he generalized Pontryagin's maximumprinciple which states that the optimal statetrajectoryx*, optimal control u *, andcorresponding Lagrange multiplier vector *must optimize the Hamiltonian Hso that:

(1-25)

Then after he introduced the general theorem: ifu*(.) is an optimal control, then there exist afunction H*(.), is called the co-state, that

satisfies a certain maximization principle.

),),,,(,(),,( ttxuxHtxH PPP !

),,,(min tuxUu

P

!


7/21

Using Pontryagin Principle in solving Optimal ControlProblems can be summarized in the following steps:

Given the plant equation as in equation (1-15)

Given the performance index as in equation (1-14)

Given the control variable constraints

Step 1: Form the Pontryagin function as in equation (1-17)

Step 2: Minimize H w.r.t all admissible control vectors tofind u*

Step 3: Find the Pontryagin function H* by substitute u*in step 1.

Step 4: Solve the state and co-state equations with theboundary conditions given.

Step 5: Substitution of results 4 into step 2 results in the

Optimum Control u*.


8/21

Example: Given the plant equation as

The performance index to be minimized

The control inequality constraints are given by.

Solution:

Step 1. pontryagin function

dtuxJ

tf

t

)(

2

1 2

0

2

!

)()()()()( 2221 tutxtxtxtx !!yy

uxxuxtH 2222122

2

1

2

1),( PPP !u,x,

],[1)(0 ftttfortu e


9/21

Step 2: minimize H subject to inequality constraint which results in

the following equation (whenu is unsaturated)

Thus for the control u is given by

and when we find that the control minimizes H is:

The optimal control strategy is given in the figure below:

If the boundary conditions are given then optimal control u(t) can be found

solving the state and co-state equations.

)()( 2* ttu P!

)()( 2* ttu P!

1)(2 "tP

1)(2 etP

1)(1

1)(1)(

2

2*

"!

tfor

tfortu

P

P

1

1

-1

-1

)(*2 tP

)(* tu


10/21

The Maximum Principle in Nonlinear

Programming Problems

We begin by starting a general form of a

nonlinear programming problem.

y: be an n-component column vector,

a: be an r-component column vector,

b: be an s-component column vector.h, g,w : be given functions.


11/21

We assume functions gand w to be column

vectors with components rand s ,respectively.

We consider the nonlinear programmingproblem:

subject to


12/21

With equality constraints, we can use the

Lagrangian as:

( 4)

where P is an r-component row vector.

The necessary condition fory* to be a

(maximum) solution to be (1) and (2) is that

there exists an r- component row vectorP such

that:

and

Finding out and

0),( !x

xyyL P 0

),(!

xx

P

P

yL

*P

*y


13/21

For Inequality Constraints

(5)

(6)

(7)

(8)

However, the conditions in (8) are new and are

particular to the inequality-constrained

problem.

0!x

x

x

x!

x

x

yy

H

y

L [Q


14/21

Example

Solution. We form the Lagrangian

The necessary conditions (6)-(8) become

(9)

(10)

(11)

028!

!

x

x

Qxx

L


15/21

Case 1:

From (9) we getx= 4, which also satisfies (10).

Hence, this solution, which makes h(4)=16, is a

possible candidate for the maximum solution.

Case 2:

Here from (9) we get Q = - 4, which does not

satisfy the inequality Q u 0 in (11).

From these two cases we conclude that the

optimum solution isx*= 4 and


16/21

Example : solve the problem:

Solution. The Lagrangian is

The necessary conditions are

(12)

(13)

(14)

028 !

!x

x

xx

L


17/21

Case 1: Q = 0

From (12) we obtainx= 4 , which does not

satisfy (13), thus, infeasible solution.

Case 2:x=6

(13) holds. From (12) we get Q = 4, so that (14)

holds. The optimal solution is then

since it is the only solution satisfying thenecessary conditions.


18/21

Example: Find the shortest distance between

the point (2.2) and the point solving the

following :


19/21

The Lagrangian function for this problem is

(15)

The necessary conditions are(16)

(17)

(18)

(19)

(20)

(21)

From (20) we see that eitherQ =0 orx2+y2=1,i.e., we

are on the boundary of the semicircle. IfQ =0, we see

from (16) thatx=2. Butx=2 does not satisfy (18) for

any y , and hence we conclude Q>0andx2

+y2

=1.


20/21

From (21) we conclude that either ory=0. If

, then from (16), (17) and Q >0, we get

x = y. Solving the latter withx2

+y2

=1 , gives; and

;

Ify=0, then (17) is not satisfied the two points areshown in figure. Of the two points found that satisfythe necessary conditions, clearly the point

is the nearest point and solves the closest-

point problem. The other point is in factthe farthest point.

)2

1,

2

1(),( !yx )

2

1,

2

1(),( !yx

249 !h 249

)2

1,

2

1(),( !yx

)2

1,

2

1(),( !yx


21/21

Documents

Lecture 4 Maximum Principle