Vasile Staicu - dottorati.unica.it · Vasile Staicu (University of Aveiro) An introduction to Mathematical Theory of Control UNICA, May 2018 15 / 120. Example 2: cart on a rail Consider

An introduction to Mathematical Theory of Control

Vasile Staicu

University of Aveiro

UNICA, May 2018

Vasile Staicu (University of Aveiro) An introduction to Mathematical Theory of Control UNICA, May 2018 1 / 120

Introduction

Since the beginnings of Calculus, differential equations have providedan effective mathematical model for a wide variety of physicalphenomena.

Consider a system whose state can be described by a finite number ofreal valued parameters, say x = (x1, x2, ..., xn) .

If the rate of change x = dxdt is entirely determined by the state x

itself, then the evolution of the system can be modelled by theordinary differential equation

x = g (x) . (1)

If the state of the system is known at some initial time t0, the futurebehavior for t > t0 can then be determined by solving a problem ofCauchy consisting of (1) together with the initial condition

x (t0) = x0. (2)


Introduction

We are here taking a spectator’s point of view: the mathematicalmodel allows us to understand a portion of the physical word andpredict its future evolution, but we have no means of altering itsbehavior in any way.

Celestial mechanics provides a typical example of this situation.

We can accurately calculate the orbits of moons and planets andexactly predict time and location of eclipses, but we cannot changethem in the slightest amount.

This classical point of view assumes that physical systems are isolatedand all the information contained in them is completely accessible tothe external observer.

The mathematical model describes the evolution of the system takinginto account its internal strengths and observations on the systemsmade from abroad but there is no way to interfere in this evolution.


Introduction

The failure of this framework turns out to be clear in front of thedevelopment of technology and engineering. The invention ofmachines and tools and their dissemination put the researcher in frontof the reality of systems that can be easily guided from abroad inorder to regulate its operation based on a pre-established project andwith the specific objective to be realized.

Mathematical models of these systems should take into account alsothe forces or impulses coming from abroad.

Thus, comes the need to accept the modern point of view, where anyphysical system (machine, or chemical, biological or social process) isdefined in a certain region of space and has its own dynamics in basewhich develops, exchanging continuously information with theexternal environment.


Introduction

Such exchange of information occurs through certain channels thatare named generically inputs (if the information coming from theexterior to the system) and outputs (the information will be to theoutside of the system).

Graphically we can represent a system like

u (t) 7−→ 7−→ x (t)

The mathematical control theory is the theoretical basis for the studyof physical systems equipped with inputs and outputs and provides adifferent paradigm.

We now assume the presence of an external agent, namely a"controller" who can actively influence the evolution of the system.


Introduction

This new situation is modelled by a control system,

x = f (x , u) , u (.) ∈ U (3)

where U is a family of admissible control functions.In this case, the rate of change x (t) depends not only of the state xitself, but also on some external parameters, say u = (u1, u2, ..., um) ,which can also vary in time.

The control function u (.) subject to some constraints, will be chosenby a controller in order to modify the evolution of the system andachieve certain preassigned goals:

steer the system from a state to another,maximize the terminal value of one of the parameters,minimize a certain cost functional, etc.


Introduction

In a standard setting, we are given a set of control values U ⊂ Rm .

The family of admissible control functions is defined as

U := u : R→ Rm : u measurable, u (t) ∈ U for a. e. t

The control system (3) can be written as a differential inclusion,namely

x ′ ∈ F (x) (4)

where, for each x , the set of possible velocities is given by

F (x) := y : y = f (x , u) for some u ∈ U . (5)


Introduction

An absolutely continuous function x : [0,T ]→ Rn is said to be anadmissible solution (or trajectory) of the control system (3) if thereexists a measurable control u : [0,T ]→ U such that

x ′ (t) = f (x (t) , u (t)) for almost every t ∈ [0,T ] . (6)

By solution (or trajectory) of the differential inclusion (4) we meanany absolutely continuous function x : [0,T ]→ Rn such that

x ′ (t) ∈ F (x (t)) for almost every t ∈ [0,T ] . (7)

Clearly, every admissible trajectory of the control system (3) is also asolution of the differential inclusion (4).


Introduction

Under some regularity assumptions on f , given any solution of (4) ,one can select a measurable control u : [0,T ]→ U such that

x ′ (t) = f (x (t) , u (t)) for almost every t ∈ [0,T ] ,

that is, x is a trajectory of the control system (3) .

Differential inclusions often provide a convenient approach for theanalysis of control systems, as we will see in what follows.


Introduction

The control law can be assigned in two basically different ways:

in "open loop" form, as a function of time: t → u (t), andin "closed loop" or "feedback", as a function of the state: x → u (x).

Implementing an open loop control u = u (t) is in a sense easier, sincethe only information needed is provided by a clock, measuring time.

On the other hand, to implement a closed loop control u = u (x) oneconstantly needs to measure the state x of the system.


Introduction

Designing a feedback control, however yields to some distinctadvantages. The feedback controls are more robust in the presence ofrandom perturbations.

For example, assume that we seek a control u (.) which steers thesystem described by (3) from an initial state P to the origin. This canbe achieved say by an open loop control t → u (t) .

In many practical situations, however, the evolution is influenced byadditional disturbance which cannot be predicted in advance. Theactual behavior of the system will be thus be governed by

x = f (x , u) + η (t) , (8)

where t → η (t) is a perturbation term. In this case, if the open loopcontrol u = u (t) steers the system (3) from an initial state P to theorigin, this same control function may not accomplish the same taskwith (8) , when a perturbation is present.


Introduction

Alternatively, one can solve the problem of steering the systemdescribed by (3) from an initial state P to the origin by mean of aclosed loop control. In this case, we would seek a control functionu = u (x) such that all the trajectories of the ordinary differentialequation

x = g (x) := f (x , u (x)) (9)

approach the origin as t → ∞.This approach is less sensitive to the presence of externaldisturbances.

Two examples will be presented below,


Example 1: boat on the river

Consider a river with a straight course, and using a set of planarcoordinates, assume that it occupies the horizontal strip

S :=(x1, x2) ∈ R2 : −∞ < x1 < +∞, − 1 < x2 < 1

.

Moreover, assume that speed of the water is given by the velocityvector

v (x1, x2) =(1− x22 , 0

).

If a boat on the river is merely dragged by the current, its positionwill be determined by the differential equation

(x1, x2) =(1− x22 , 0

).

On the other hand, if the boat is powered by an engine, then itsmotion can be modelled by the control system

(x1, x2) =(1− x22 + u1, u2

), (10)

where u = (u1, u2) describe the velocity of the boat relative to thewater.


Example 1

The set U of admissible controls consists of all measurable functionsu : R→ R2 taking values inside the closed disc

U =(ω1,ω2) ∈ R2 :

√ω21 +ω2

2 ≤ M, (11)

where M accounts for the maximum speed (in any direction) that canbe produced by the engine.

Given an initial condition (x1, x2) (0) = (x1, x2) solving (10) one finds

x1 (t) = x1 + t +

t∫0

u1 (s) ds −t∫0

x2 + s∫0

u2 (τ) dτ

2

ds,

x2 (t) = x2 +

t∫0

u2 (s) ds (−1 ≤ x2 ≤ 1) .


Example 1

In particular, the constant control u = (u1, u2) =(− 23 , 1

)takes the

boat from a point (x1,−1) on one side of the river to the point(x1, 1) on the opposite side, in two units of time.

Observe that if M > 0 then the boat can be steered from any point Pof the river to any other point of the river (exercise: justify !) andthat the admissible trajectories of the control system (10)− (11)coincide with the solutions of the differential inclusion

(x1, x2) ∈ F (x1, x2) =(y1, y2) :

√(y1 − 1+ x22 )

2+ y22 ≤ M

.


Example 2: cart on a rail

Consider a cart which can move without friction along a straight rail.For simplicity assume that it has unit mass.

Let y (0) = y be its initial position and v (0) = v be its initialvelocity.

If no forces are present, its future position is simply given by

y (t) = y + tv .

Next, assume that a controller is able to push the cart, with anexternal force u = u (t) .


Example 2

The evolution of the system is then determined by the second orderequation (determined by the Newton law)

y = u (t) . (12)

Calling now x1 (t) = y (t) and x2 (t) = v (t) , respectively theposition and the velocity of the chart at time t, the equation (12) canbe written as a first order control system

x1 = x2x2 = u

(13)

Given the initial condition x1 (0) = y , x2 (0) = v , solving (13) onefinds

x1 (t) = y + tv +

t∫0

(t − s) u (s) ds, x2 (t) = v +t∫0

u (s) ds.


Example 2

Assuming that the force satisfies the constraint

|u (t)| ≤ 1,

the control system (13) is equivalent to the differential inclusion

(x1, x2) ∈ F (x1, x2) = (x2,ω) : −1 ≤ ω ≤ 1 .


Example 2

We now consider the problem of steering the system at the origin(0, 0) ∈ R2.

In other words, we want the chart to be eventually at the origin withzero speed.

For example, if the initial condition (y , v) = (2, 2) , this goal isachieved by the open-loop control

u (t) =

−1 if 0 ≤ t ≤ 41 if 4 ≤ t ≤ 60 if t ≥ 6.

A direct calculation shows that (x1 (t) , x2 (t)) = (0, 0) for t ≥ 6.However, the above control would not accomplish the same task inconnection with (y , v) 6= (2, 2) (exercise: verify!)


Example 2

A related problem is that of asymptotic stabilization: we seek afeedback control function u = u (x1, x2) such that for every initialdata (y , v) the corresponding solution of the Cauchy problem

(x1, x2) = (x2, u (x1, x2)) , (x1, x2) (0) = (y , v)

approaches the origin as t → ∞, that is,

limt→∞

(x1, x2) (t) = (0, 0) .

There are several feedback controls which accomplish this task.

One isu (x1, x2) = −x1 − x2.


Questions of interest

In connection with a control system of the general form (3) severalmathematical questions can be formulated.

A first set of problems is concerned with the dynamics of the system.

Given an initial state x , one would like to determine which other statesx ∈ Rn can be reached using the various admissible controls u ∈ U .More precisely, given a control function u = u (t), call x (., u) thesolution of the Cauchy problem

x (t) = f (x (t) , u (t)) , x (0) = x ,

and define the reachable set at time t as

R (t) = x (t, u) : u ∈ U .



The closure, boundedness, convexity and the dimension of the setR (t) provide useful information on the control system.In addition, it is interesting to study whether R (t) is a neighborhoodof the initial point x for all t > 0. In the positive case, the system issaid to be small time locally controllable at x .Another important case is when

⋃t>0

R (t) includes the entire space

Rn. We then say that the system is globally controllable.The dependence of the reachable set R (t) on the time t and on theset of admissible controls U is also a subject of investigation.One may ask whether the same points in R (t) can be reached byusing controls which are piecewise constant, or take values within theset of extreme points of U.Being able to perform the same tasks by mean of a smaller set ofcontrol functions, easier to implement, is quite relevant in practicalapplications.



A second very important area of control theory is concerned with theoptimal control.

In many applications, among all strategies which accomplish a certaintask, one seeks an optimal one, based on a given performancecriterion.

A performance criterion can be defined by an integral functional ofthe form

J =∫ T

0L (t, x , u) dt, (14)

and the values of J will have to be optimized among the admissibletrajectories of (3) , with a number of initial or terminal constraints.



For example, among all controls which steer the system from theinitial point x to some point on the target set T at time T , we mayseek the one that minimize the cost functional (14) .

This problem is formulated as

minu∈U

∫ T

0L (t, x , u) dt, (15)

subject tox = f (x , u) , x (0) = x , x (T ) ∈ T . (16)

Observe that if (3) takes the simplest form

x = u, u (t) ∈ U = Rn

and if T = y consists on just one point, then we do not have anyconstraint on the derivative x .



Our problem of optimal control thus reduces to the standard problemof the Calculus of Variations

minx (.)

∫ T

0L (t, x , x) dt, x (0) = x , x (T ) = y . (17)

Roughly speaking, the main difference between the problem(15)− (16) and (17) is that in (17) the derivative x is not restricted,while in (15)− (16) the derivative is constrained within the set

F (x) := y : y = f (x , u) for some u ∈ U .The basic mathematical theory of optimal control has been concernedwith the following main topics: existence of optimal controls;necessary conditions for the optimality of a control; suffi cientconditions for optimality.The next section will be dedicated to Carathéodory solutions toordinary differential equations which arise from control systems, andto which the classical notion of solution is not appropriate.


Normed vector spaces

A real vector space is a nonempty set X endowed with twooperations: + : X × X → X and · : R×X → X satisfying thefollowing conditions:

1 (x + y) + z = x + (y + z) , ∀x , y , z ∈ X ;2 x + y = y + x , ∀x , y ∈ X ;3 there exists 0 ∈ X such that x + 0 = x , ∀x ∈ X ;4 for every x ∈ X there exists −x ∈ X such that x + (−x) = 0∀x ∈ X ;

5 (α+ β) · x = α · x + β · x , ∀α, β ∈ R, ∀x ∈ X ;6 α · (x + y) = α · x + α · y , ∀α ∈ R, ∀x , y ∈ X ;7 1 · x = x , ∀x ∈ X (where 1 is the unit real number)8 α · (β · x) = (αβ) · x , ∀α, β ∈ R, ∀x ∈ X .

We denote by (X ,+, ·) a vector space with the operations + and ·.Vasile Staicu (University of Aveiro) An introduction to Mathematical Theory of Control UNICA, May 2018 26 / 120


A subset A of a vector space (X ,+, ·) is said to be convex if

λx + (1− λ) y ∈ A, , ∀x , y ∈ A and ∀λ ∈ [0, 1] .

Let (X ,+, ·) be a (real) vector space. A norm on X is a mapping‖.‖ : X → R with the following properties:(i) ‖x‖ = 0 if and only if x = 0;(ii) ‖α · x‖ = |α| ‖x‖ ∀α ∈ R, ∀x ∈ X ;(iii) ‖x + y‖ ≤ ‖x‖+ ‖y‖ ∀x , y ∈ X .If ‖.‖ is a norm on X then we say that (X , ‖.‖) is a vector normedspace, or simply a normed space.



Let (X , ‖.‖) be a normed space and for x0 ∈ X and r > 0 let

Br (x0) = x ∈ X : ‖x − x0‖ < rbe the open ball centered at x0 with radius r , and

B r (x0) = x ∈ X : ‖x − x0‖ ≤ rbe the closed ball centered at x0 with radius r .Both the open and the closed balls centered at x0 with radius r areconvex setIndeed, if x , y ∈ Br (x0) and 0 ≤ λ ≤ 1 then‖λx + (1− λ) y − x0‖ ≤ ‖λ (x − x0)‖+ ‖(1− λ) (y − x0)‖

= λ ‖x − x0‖+ (1− λ) ‖y − x0‖< λr + (1− λ) r = r

hence λx + (1− λ) y ∈ Br (x0) , and Br (x0) is a convex set. Theproof for B r (x0) is similar.


Normed vector spaces are metric spaces

Recall that if (X , ‖.‖) is a normed space then d : X × X → R

defined by d (x , y) = ‖x − y‖ is a distance (or metric) on X , that issatisfies the following properties:(a) d (x , y) = 0 if and only if x = y ;(b) d (x , y) = d (y , x) ∀x , y ∈ X ;(c) d (x , y) ≤ d (x , z) + d (z , y) ∀x , y , z ∈ X .If d : X × X → R satisfies (a) , (b) and (c) then d is called a metricon X and (X , d) is said to be a metric space.


Banach spaces

Let (X , ‖.‖) be a normed space.A sequence (xn)n≥1 ⊂ X is said to be convergent if there existsx ∈ X such that: for every ε > 0 there exists nε ∈N so that‖xn − x‖ < ε whenever n ≥ nε.

A sequence (xn)n≥1 ⊂ X is said to a Cauchy sequence if for everyε > 0 there exists nε ∈N so that ‖xn − xm‖ < ε whenever n,m ≥ nε.

We say that a normed space is complete if every Cauchy sequence isconvergent.

A complete normed vector space is called a Banach space.


Examples of Banach spaces

The finite dimensional space Rn with the Euclidian norm

‖x‖ =√

n

∑i=1xi , if x = (x1, ..., xn) ∈ Rn;

The space C ([a, b] ,Rn) of all continuous functions f : [a, b]→ Rn,with norm

‖f ‖∞ = supt∈[a,b]

‖f (t)‖ ;

Given a subset Ω ⊂ Rm , we say that a map f : Ω→ Rn is Lipschitzcontinuous if there exists a constant L such that

‖f (x)− f (y)‖ ≤ L ‖x − y‖ , ∀x , y ∈ Ω.

The space Lip (Ω,Rn) of all these Lipschitz continuous mappings is aBanach space with norm

‖f ‖Lip = supx∈Ω‖f (x)‖+ sup

x 6=y

‖f (x)− f (y)‖‖x − y‖ .


Examples of normed space which is not Banach spaces

The space X = P ([0, 1]) of polynomial functions p : [0, 1]→ R withthe norm

‖p‖∞ = supt∈[0,1]

‖p (t)‖

is a normed space but not a Banach space

Indeed, the sequence of polynomials pn (t) =n

∑k=0

tkk ! is a Cauchy

sequence converges uniformly on [0, 1] hence also with respect to‖.‖∞ to the continuous function f (t) = et .

Hence it is a Cauchy sequence in (C ([0, 1] ,R) , ‖.‖∞), therefore alsoin (P ([0, 1]) , ‖.‖∞) .

But its limit f (t) = et , is not polynomial, that is,it has no limit inthe space X = P ([0, 1]).


Banach contraction mapping principle

The next Banach’s theorem show that for an equation, a uniquesolution exists and depends continuously on the parameters thatdescribe the problem if the equation can be written in the form

x = Φ (λ, x) (18)

and the map x → Φ (λ, x) is a contraction for each given value ofthe parameter λ, that is, for some κ < 1,

‖Φ (λ, x)−Φ (λ, y)‖ ≤ κ ‖x − y‖ , ∀λ ∈ Λ, ∀x , y ∈ X .


Banach contraction mapping principle

Theorem (contraction mapping principle) Let (X , ‖.‖) be a Banachspace, Λ be a metric space and let Φ : Λ× X → X be a continuousmapping such that, for some κ < 1,

‖Φ (λ, x)−Φ (λ, y)‖ ≤ κ ‖x − y‖ , ∀λ ∈ Λ, ∀x , y ∈ X . (19)

Then:(i) For each λ ∈ Λ there exists a unique x (λ) ∈ X such that

x (λ) = Φ (λ, x (λ)) (20)

(ii) The map λ→ x (λ) is continuous(iii) For any λ ∈ Λ, y ∈ X one has

‖y − x (λ)‖ ≤ 11−κ ‖y −Φ (λ, y)‖ . (21)


Proof of Banach contraction principle

Fix any point y ∈ X . For each λ ∈ Λ, consider the sequenceyk = yk (λ) defined by

y0 = y , yk+1 = Φ (λ, yk ) for k ≥ 0.

By induction, for every k ≥ 0 one checks that

‖yk+1 − yk‖ ≤ κk ‖y1 − y0‖ = κk ‖y −Φ (λ, y)‖ . (22)

Since κ < 1, the sequence (yk )k≥0 is Cauchy in the Banach space X ,hence it converges to some limit point, which we call x (λ) .



By the continuity of Φ (λ, .) we have

x (λ) = limk→∞

yk+1 = limk→∞

Φ (λ, yk ) = Φ (λ, x (λ)) ,

hence (20) holds.

The uniqueness of x (λ) is proved observing that, if

x1 = Φ (λ, x1) , x2 = Φ (λ, x2) ,

then by (19) it follows that

‖x1 − x2‖ = ‖Φ (λ, x1)−Φ (λ, x2)‖ ≤ κ ‖x1 − x2‖ .

The assumption κ < 1 implies

x1 = x2.



Next, observe that (22) yields

‖yk+1 − y‖ ≤k

∑i=0‖yi+1 − yi‖ ≤ ‖y −Φ (λ, y)‖

k

∑i=0κi

≤ ‖y −Φ (λ, y)‖∞

∑i=0κi = ‖y −Φ (λ, y)‖

1−κ .

and letting k → ∞ we obtain (21) .

To show the continuity of λ→ x (λ) let (λn)n≥1 be a sequence ofparameters converging to λ∗.



Using (21) with λ = λn and y = x (λ∗) we obtain

‖x (λ∗)− x (λn)‖ ≤1

1−κ ‖x (λ∗)−Φ (λn, x (λ∗))‖

=1

1−κ ‖Φ (λ∗, x (λ∗))−Φ (λn, x (λ∗))‖ (23)

and since λ→ Φ (λ, x) is continuous, the right-hand side of (23)tends to zero as n→ ∞.Hence x (λn)→ x (λ∗) proving the continuity of λ→ x (λ) .


Elements of Lebesgue measure theory

If I = (a, b) is an open interval of R then the Lebesgue measure of Iis defined by

µ (I ) = b− a.For any open set A ⊂ R the Lebesgue measure is defined by

µ (A) = sup

∑j∈J

µ (Ij ) :⋃j∈JIj ⊂ A, Ij open int., Ij ∩ Ik = ∅ if j 6= k

.

If F ⊂ R is compact (that is, closed and bounded), then there existsan open and bounded interval I = (α, β) such that F ⊂ I . Definethen the Lebesgue measure of F by

µ (F ) = µ (I )− µ (I\F ) .



If X ⊂ R is any subset, then we define

µ∗ (X ) = sup µ (F ) : F compact ⊆ X

and

µ∗ (X ) = inf µ (A) : A compact open, bounded and with X ⊆ A .

We say that X is Lebesgue measurable if µ∗ (X ) = µ∗ (X ) and thecommon value is called the Lebesgue measure of X and denoted byµ (X ) :

µ (X ) := µ∗ (X ) = µ∗ (X ) .

One has the following:Proposition: A subset X ⊆ R is Lebesgue measurable if and only iffor every ε > 0 there exists a compact Kε ⊆ R and an open setAε ⊆ R such that

Kε ⊆ X ⊆ Aε and µ (Aε\Kε) < ε.



Example: A finite or countable set A ⊆ R is Lebesgue measurableand has a null measure.We denote by L the family of all Lebesgue measurable subsets of R

and if I is an interval, we denote by L (I ) the family of all measurablesubsets of I :

L (I ) = A ∈ L : A ⊆ I .Let I ⊆ R be an interval. A function f : I → Rn is said to beLebesgue measurable if for every open set V ⊆ Rn the preimagef −1 (V ) = t ∈ I : f (t) ∈ V is a Lebesgue measurable subset of I .Recall that f : I → Rn is said to be continuous if at each t0 ∈ I , onehas

f (t0) = limt→t0

f (t) .

f : I → Rn is said to be lower semicontinuous if at each t0 ∈ I , onehas

f (t0) ≤ lim inft→t0

f (t) .



Remark: Every continuous or lower semicontinuous function, ismeasurable.

We say that a property P holds almost everywhere (a.e. for short) ona set A ⊆ R if there exists a set N ⊆ R such that µ (N) = 0 andevery point of A\N has the property P.Let (fn)n≥1 be a sequence of measurable functions, fn : I → Rn and if

fn (x)→ f (x) for a.e. x ∈ I

then the function f is measurable.

If f : [a, b]→ Rn is measurable and if ‖f (t)‖ ≤ φ (t) for allt ∈ [a, b] for some integrable scalar function φ, then f itself isintegrable.



If f and f differ only on a set of measure zero are identified. With thisequivalence relation, the space of integrable functions f : [a, b]→ Rn

is written L1 ([a, b] ,Rn) and it is a Banach space with norms

‖f ‖L1 =∫ b

a‖f (t)‖ dt.

Lebesgue dominated convergence theorem: If (fn)n≥1 is asequence of measurable functions which converges a.e. to f and ifthere exists an integrable function ψ such that ‖fn (t)‖ ≤ ψ (t) for alln ≥ 1 and t ∈ [a, b] , then∫ b

af (t) dt = lim

n→∞

∫ b

afn (t) dt



Characterization of measurable functions: For any functionf [a, b]→ Rn, the following statements are equivalent:(i) f is measurable(ii) For every ε > 0 there exists a compact set Kε ⊆ [a, b] with

µ ([a, b] \Kε) < ε

and such that the restriction

f |Kε is continuous.



Absolutely continuous functions: A function f : [a, b]→ Rn is saidto be absolutely continuous if for every ε > 0 there exists δ > 0 suchthat for any finite family of disjoint intervals ]si , ti ] : 1 ≤ i ≤ Nwith total length

N

∑i=1|ti − si | < δ

one hasN

∑i=1|f (ti )− f (si )| < ε.



If f : [a, b]→ Rn is absolutely continuous, then its derivative f ′ isdefined almost everywhere and it satisfies

f (t) = f (t0) +∫ t

t0f ′ (s) ds for all t, t0 ∈ [a, b] .

Every Lipschitz continuous function f defined on [a, b] is absolutelycontinuous.

The set of all absolutely continuous functions f : [a, b]→ Rn isdenoted by

AC ([a, b] ,Rn) .


Elements of multivalued analysis

Assume (X , d) is a metric space. For x ∈ X and A ⊆ X let d (x ,A)be the distance from x to A defined by

d (x ,A) = inf d (x , y) : y ∈ A .

If A ⊆ X and ε > 0 then B (A, ε) denotes the ε− neighborhood of Adefined by

B (A, ε) = x ∈ X : d (x ,A) ≤ ε .If A = x0 then B (A, ε) is the closed ball centered at x0 with radiusε :

B (x0, ε) = x ∈ X : d (x , x0) ≤ ε .



If A, B are two closed, bounded and nonempty subsets of X then theHausdorff-Pompeiu distance between A and B is defined by

dH (A,B) = maxsupa∈A

d (a,B) , supb∈B

d (b,A).

One has that

dH (A,B) = inf

ε > 0 : B ⊂ B (A, ε) and A ⊂ B (B, ε)

= max d (a,B) , d (b,A) : a ∈ A and b ∈ B .

The space C (X ) of all closed, bounded and nonempty subsets of Xendowed with the Hausdorff-Pompeiu distance dH is a metric space.



Let X and Y be two sets and let 2Y = A : A ⊆ Y be the family ofall subsets of Y .

By multifunction or multivalued map or set-valued map from X to Ywe mean a map or correspondence which associate to each x ∈ X aunique subset F (x) of Y .

The set F (x) is called the value of F at x .

A multifunction F from X to Y will be denoted by F : X → 2Y .

The setGr (F ) = (x , y) : x ∈ X , y ∈ F (x)

is called the graph of the multifunction F : X → 2Y .



Let X , Y be metric spaces. We say that F is closed, bounded,compact valued if its values are all closed, bounded or compact.

We say that F : X → 2Y has closed graph if Gr (F ) is a closed subsetof the product metric space X × Y , which is equivalent to:

if: xn ∈ X , xn → x , yn ∈ F (xn) , yn → y then y ∈ F (x) .



Let F : X → 2Y be a multivalued map with closed and boundedvalues. We say that F is Hausdorff continuous at x0 ∈ X if

limx→x0

dH (F (x) ,F (x0)) = 0,

that is, for every ε > 0 there exists δε > 0 such thatdH (F (x) ,F (x0)) < ε whenever x ∈ B (x0, δε) .

Let I ⊆ R and let F : I → 2Rnbe a multifunction. A function

f : I → Rn is said to be a selection of F if

f (t) ∈ F (t) for all t ∈ I .

If f : I → Rn is a selection of F and if it is a measurable functionthen we say that f is a measurable selection of F .



In order to introduce the lexicografic measurable selection, we need todefine the lexicographic order relation on Rn : if x = (x1, x2, ..., xn)and y = (y1, y2, ..., yn) then we say that x < y if one of the followingsituations happens:

x1 < y1 or

x1 = y1 and x2 < y2 or

.....

x1 = y1, ...,xn−1 = yn−1, and xn < yn.

One has the following:Proposition: If K ⊆ Rn is compact subset of Rn then there thereexists a first element ξ of K with respect to the lexicografic order,that is, there exists ξ ∈ K such that ξ < x for all x ∈ K .



If F : I → 2Rnis a multifunction with compact values then we can

define the lexicographic selection t → ξ (t) where ξ (t) ∈ F (t) is thefirst element of the compact set F (t) with respect to thelexicographic order.

One has the following:Proposition If I = [a, b] and if F : I → 2Rn

is a compact valuedmultifunction, with closed graph and if t → ξ (t) is the lexicograficselection of F then t → ξ (t) is a measurable selection.


Carathéodory solutions

As we already seen in the basic model of a control system a controlsystem,

x = f (x , u)

as soon as a control function u = u (t) from the family U ofadmissible controls is assigned, the evolution can be determined bysolving the ordinary differential equation (ode, for short)

x = g (t, x) (24)

whereg (t, x) := f (x , u (t)) . (25)


Classical solutions

In the classical theory of ordinary differential equations like (24) weassume that the function g is continuous with respect to both thevariable and the appropriate notion of solution is the one of "classicalsolution".

Definition: Let Ω ⊆ R×Rn be an open set and let g : Ω→ Rn bea function which define the equation (24) . A functionϕ : I ⊆ R→ Rn, with I an interval, is called a classical solution of(24) if ϕ ∈ C 1 (I ,Rn) , G (ϕ) := (t, ϕ (t)) : t ∈ I ⊆ Ω and

ϕ′ (t) = g (t, ϕ (t)) for all t ∈ I .


Carathéodory solutions

For the case of control systems, the function g defined by (25)follows to be only measurable with respect to t variable, so the notionof classical solution is not appropriate. We will use instead the notionof Carathéodory solution.

Definition: Let Ω be an open set in R×Rn and let g : Ω→ Rn bea function which define an equation like (24) . A functionϕ : I ⊆ R→ Rn, with I an interval, is called a Carathéodory solutionof (24) if ϕ ∈ AC (I ,Rn) , G (ϕ) := (t, ϕ (t)) : t ∈ I ⊆ Ω and

ϕ′ (t) = g (t, ϕ (t)) a.e t ∈ I . (26)


Carathéodory solutions: some remarks

(i) An absolutely continuous function ϕ : [t0, t1] ⊆ Rn, is aCarathéodory solution of (24) ifG (ϕ) := (t, ϕ (t)) : t ∈ [t0, t1] ⊆ Ω and

x (t) = x (t0) +∫ t

t0g (s, x (s)) ds for every t ∈ [t0, t1] .

(ii) Any classical solution is a Carathéodory solution(iii) The function ϕ : [−1, 1]→ R defined by

ϕ (t) =

t if t ≥ 0−t if t < 0

is a Carathéodory solution but not a classical solutions of∣∣x ′∣∣ = 1, x (0) = 0.Vasile Staicu (University of Aveiro) An introduction to Mathematical Theory of Control UNICA, May 2018 57 / 120

Existence of Carathéodory solutions

We assume that the function g : Ω ⊂ R×Rn → Rn satisfies thefollowing conditions:

(A) For every x the function t → g (t, x) defined on the sectionΩx = t : (t, x) ∈ Ω is measurable.For every t, the function x → g (t, x) defined on the sectionΩt = x : (t, x) ∈ Ω is continuous

(B) For every compact K ⊂ Ω there exist constants CK , LK such that

‖g (t, x)‖ ≤ CK , ‖g (t, x)− g (t, y)‖ ≤ LK ‖x − y‖ ,for all (t, x) , (t, y) ∈ K . . (27)

For some (t0, x0) ∈ Ω we consider the Cauchy problem (g , t0, x0) :

x = g (t, x) , x (t0) = x0. (28)


Existence of Carathéodory solutions

Theorem (existence) Given a map g : Ω→ Rn, consider the Cauchyproblem (g , t0, x0) in (28) for some (t0, x0) ∈ Ω.(i) If g satisfies (A) , (B) , then there exists ε > 0 such that (28) hasa local solution x (.) defined for t ∈ [t0, t0 + ε](ii) Assume, in addition that the function g is defined in the entirespace R×Rn and there exist constants C , L such that

‖g (t, x)‖ ≤ C , ‖g (t, x)− g (t, y)‖ ≤ L ‖x − y‖ (29)

for all (t, x) , (t, y) ∈ R×Rn.Then for every T > t0, the Cauchy problem (28) has a globalsolution x (.) defined for all t ∈ [t0,T ]. Moreover, the solutiondepends continuously on the initial data x0.


Proof of the Existence of Carathéodory solutions

Assume that g satisfies (A) in R×Rn and (29). Let T > t0. Weshall construct a forward solution x = xx0 : [t0,T ]→ Rn.

In view of applying Theorem 1 (the Banach contraction mappingprinciple) we define Λ = Rn (the initial value x0 ∈ Rn plays here therole of a parameter). Let X be the space C ([t0,T ] ,Rn) endowedwith the weighted norm

‖x‖+ = supt∈[t0,T ]

e .−2Lt ‖x (t)‖ ,

which is equivalent to the usual C ([t0,T ] ,Rn) norm defined by‖x‖∞ = supt∈[t0,T ] ‖x (t)‖ ,and define Φ : Λ× X → X by setting

Φ (x0,w) (t) = x0 +∫ t

t0g (s,w (s)) ds, t ∈ [t0,T ] . (30)



To prove that Φ is well defined, for each w ∈ X we need to show thatthe composite function s → g (s,w (s)) is integrable. Letδ = T − t0.Given a function w ∈ X we consider the sequence of piecewiseconstant functions (wn)n∈N with

wn (t) = w(t0 + k

δ

n

), t ∈

[t0 + k

δ

n, t0 + (k + 1)

δ

n

], 0 ≤ k ≤ n− 1.

By assumption (A) , the maps t → g (t,wn (t)) are all measurable.

Moreover, (29) implies that

limn→∞‖g (t,w (t))− g (t,wn (t))‖ ≤ lim

n→∞L ‖w (t)− wn (t)‖ = 0,

hence t → g (t,w (t)) is measurable, being a pointwise limit of asequence of measurable functions.



Since‖g (s,w (s))‖ ≤ C for all s,

it follows that the integral in (30) is well defined and dependscontinuously on t.

Hence Φ is well defined and takes values inside X .

The continuity of the map x0 → Φ (x0,w) is obvious. To prove thatw → Φ (x0,w) is a contraction in

(X , ‖.‖+

), let w , w ′ ∈ X and let

θ =∥∥w − w ′∥∥

+. (31)

By the definition of ‖.‖+ and (31) , we have∥∥w (s)− w ′ (s)∥∥ ≤ θe .2Ls for all s ∈ [t0,T ] .



Moreover, by (29) , for all t ∈ [t0,T ] ,

e .−2Lt∥∥Φ (x0,w (t))−Φ

(x0,w ′ (t)

)∥∥= e−2Lt

∥∥∥∥∫ t

t0

[g (s,w (s))− g

(s,w ′ (s)

)]ds

∥∥∥∥≤ e−2Lt

∥∥∥∥∫ t

t0L∥∥w (s)− w ′ (s)∥∥ ds∥∥∥∥

≤ e−2Lt∫ t

t0Lθe2Lsds <

δ

2.

Therefore, ∥∥Φ (x0,w)−Φ(x0,w ′

)∥∥+≤ 12

∥∥w − w ′∥∥+.



We can now apply the Banach contraction mapping principle,obtaining the existence of a unique continuous mapping x0 → xx0such that

xx0 = Φ (x0, xx0)

that is,

xx0 (t) = x0 +∫ t

t0g (s, xx0 (s)) ds, t ∈ [t0,T ] .

This means that xx0 is the required solutions, and this achieves theproof of (ii) .



To prove (i) concerning local existence of solution without theadditional assumption (29) , choose ε > 0 small enough so that theset

K = (t, x) : |t − t0| ≤ ε, ‖x − x0‖ ≤ εis entirely contained in Ω.Then consider a smooth cut-off function φ : R×Rn → [0, 1] suchthat φ ≡ 1 on K , while φ ≡ 0 outside some larger compact set K ′,with K ⊂ K ′ ⊂ Ω. Observe that the function

g+ (t, x) =

φ (t, x) g (t, x) if (t, x) ∈ Ω0 if (t, x) /∈ Ω

satisfies (A) and (B) , together with the extra assumption (29) ,because it vanishes outside the compact set K ′.



By the previous steps, there exists a solution x (.) to the Cauchyproblem

x ′ = g+ (t, x) , x (t0) = x0

defined on arbitrary large interval [t0,T ] .

We now recall that K is a neighborhood of (t0, x0) . Therefore forsome ε > 0 suffi ciently small, the point (t, x (t)) remains in K ast ∈ [t0, t0 + ε] .

Since g and g+ coincide on K , the function x (.) thus provides a localsolution to the original problem (g , t0, x0) on the interval [t0, t0 + ε] .


Uniqueness of Carathéodory solutions

The next lemma represents a main ingredient in several uniquenessproofs and it is a simplified version of Gronwall’s lemma:Lemma (Gronwall) Let z (.) be an absolutely continuous nonnegativefunction such that

z (t0) ≤ γ, z (t) ≤ α (t) z (t) + β (t) for a.e. t ∈ [0,T ] , (32)

for some integrable functions α, β and some constant γ ≥ 0. Then zsatisfies the inequality

z (t) ≤ γe∫ tt0

α(s)ds+∫ t

t0β (s) e

∫ ts α(u)duds for all t ∈ [0,T ] . (33)

Theorem (uniqueness) Let g : Ω→ Rn satisfies (A) and (B) , as inthe existence Theorem. Let x1 (.) , x2 (.) be solutions of (28) , definedon the intervals [t0, t1] , [t0, t2] respectively. If T = min t1, t2 , then

x1 (t) = x2 (t) for all t ∈ [t0,T ] .


Proof of the uniqueness of Carathéodory solutions

Since g satisfies assumption (B), for the compact set

K = (t, x1 (t)) : t ∈ [t0,T ] ∪ (t, x2 (t)) : t ∈ [t0,T ]there exists LK ≥ 0 such that‖g (t, x)− g (t, y)‖ ≤ LK ‖x − y‖ for all (t, x) , (t, y) ∈ K .

For all t ∈ [t0,T ] = [t0, t1] ∩ [t0, t2] letz (t) = ‖x1 (t)− x2 (t)‖ .

One has that z (.) is absolutely continuous and

z (t) ≤ ‖x1 (t)− x2 (t)‖ ≤ LK z (t) for almost all t ∈ [t0,T ] .Applying Gronwall’s lemma with α = LK , β = γ = 0 we obtain thatz (t) ≤ 0 for all t ∈ [t0,T ] .Hence

z (t) = ‖x1 (t)− x2 (t)‖ = 0 for all t ∈ [t0,T ] .Vasile Staicu (University of Aveiro) An introduction to Mathematical Theory of Control UNICA, May 2018 68 / 120

Characterization of maximal solutions

Let g : Ω ⊂ R×Rn → Rn. In general, a solution to the Cauchyproblem

x = g (t, x) , x (t0) = x0 (34)

is defined only locally, for t in a neighborhood of the initial time t0.

If a solution cannot be extended beyong a certain time T , two casesmay arise:(i) as t → T−, the point (t, x (t)) approaches the boundary ∂Ω ofthe domain Ω;(ii) as t → T−, the solution blows up, that is ‖x (t)‖ → ∞.



Theorem Let Ω ⊂ R×Rn be an open set, g : Ω→ Rn satisfies thebasic assumptions (A) and (B) , (t0, x0) ∈ Ω and letT = sup τ : (34) has a solution defined on [t0, τ] . Then eitherT = +∞ or

limt→T −

[‖x (t)‖+ 1

d ((t, x (t)) , ∂Ω)

]= +∞, (35)

where d ((t, x) , ∂Ω) = inf |t − s |+ ‖x − y‖ : (s, y) ∈ ∂Ω .



Proof: Assume that T < ∞. If (35) does not hold, then there existM > 0, ε > 0 and a sequence tn → T− as n→ ∞, such that

‖x (tn)‖ ≤ M, d ((tn, x (tn)) , ∂Ω) ≥ ε.

By possibly taking a subsequence, we can assume that x (tn)converges to a point x∞ as n→ ∞, with (T , x∞) ∈ Ω.Let ρ > 0 so small that the cylinder

K = (t, x) : |t − T | ≤ ρ, ‖x − x0‖ ≤ ρ

is entirely contained in Ω.



Consider a smooth cut-off function φ : R×Rn → [0, 1] such thatφ ≡ 1 on K , while φ ≡ 0 outside some larger compact set K ′, withK ⊂ K ′ ⊂ Ω, and define

g+ (t, x) =

φ (t, x) g (t, x) if (t, x) ∈ Ω0 if (t, x) /∈ Ω

(36)

Then g+ ≡ g on K , and in addition g+ satisfies the global bounds C ,L such that∥∥g+ (t, x)∥∥ ≤ C , ∥∥g+ (t, x)− g+ (t, y)∥∥ ≤ L ‖x − y‖for all (t, x) , (t, y) ∈ R×Rn, for some constants L, C ≥ 1.



Fix δ > 0 small enough so that

(2C + 1) δ ≤ ρ,

and choose n so large that

‖x (tn)− x∞‖ ≤ δ, T − tn ≤ δ.

By (ii) of the existence Theorem, the Cauchy problem

y ′ = g+ (t, y) , x (tn) = x (tn)

has a solution y (.) defined on [tn,T + δ] .



We can now define an extension x of x by setting

x (t) =x (t) if t0 ≤ t ≤ tny (t) if tn ≤ t ≤ T + δ.

Since ‖y ′ (t)‖ ≤ C for t ∈ [tn,T + δ] , we have

‖y (t)− x∞‖ ≤ ‖y (t)− x (tn)‖+ ‖x (tn)− x∞‖≤ C (t − tn) + δ ≤ 2Cδ+ δ ≤ ρ.

Therefore, for all t ∈ [tn,T + δ] , the point (t, y (t)) remains insidethe compact K where g and g+ coincide.

The function x (.) thus provide a solution of the original problem(34) , defined on a strictly larger interval [t0,T + δ]. This contradictsthe maximality of T , thus proving the theorem.


Nonlinear control systems

We consider the control system

x ′ = f (t, x , u) , u (.) ∈ U (37)

where the set of admissible controls is defined as

U := u : R→ Rm : u measurable, u (t) ∈ U for all t (38)

We shall assume the following basic hypothesis:(H) : The set U ⊂ Rm of control values is compact, Ω ⊂ R×Rn isan open set, the function f : Ω× U → Rn is continuous in allvariables and continuously differentiable with respect to x .


Trajectory of nonlinear control systems

Definition: An absolutely continuous function x : [a, b]→ Rn is saidto be a solution (or trajectory) of (37) if its graph(t, x (t)) : t ∈ [0,T ] is entirely contained in Ω, and if there existsa measurable control u : [a, b]→ U such that

x ′ (t) = f (t, x (t) , u (t)) for almost every t ∈ [a, b] .


An equivalent differential inclusion

In connection with (37) , define the multifunction F : Ω→ P (Rn) by

F (t, x) = f (t, x ,ω) : ω ∈ U (39)

and consider the differential inclusion

x ′ ∈ F (t, x) . (40)

Definition: We call solution or trajectory of the differential inclusion(40) any absolutely continuous function x : [a, b]→ Rn whose graph(t, x (t)) : t ∈ [a, b] is entirely contained in Ω, and

x ′ (t) ∈ F (t, x (t)) for almost every t ∈ [a, b] .


An equivalent differential inclusion

Observe that, for each (t, x) , the set of admissible velocities x ′ in(37) is given in parametrized form, as the image of a fixed setU ⊆ Rn.

On the other hand, when we study the differential inclusion (40) , wedo not assume any parametrization.

The next result due to Filippov, shows that the two approaches areessentially equivalent.

Filippov’s lemma An absolutely continuous function x : [a, b]→ Rn

is a trajectory of the control system (37) if and only if it is a trajectoryof the differential inclusion (40) with F (t, x) defined by (39) .


Proof of Filippov’s lemma

If x : [a, b]→ Rn is a trajectory of the control system (37) then itsgraph is entirely contained in Ω, and if there exists a measurablecontrol u : [a, b]→ U such that

x ′ (t) = f (t, x (t) , u (t)) for almost every t ∈ [a, b] .

Then

x ′ (t) = f (t, x (t) , u (t)) ∈ F (t, x (t)) for almost every t ∈ [a, b] ,

hence it is a trajectory of the differential inclusion (40) .

Assume now that x : [a, b]→ Rn is a trajectory of the differentialinclusion (40) . Then

x ′ (t) ∈ F (t, x (t)) for almost every t ∈ [a, b] .


Proof of Filippov’s lemma

Fix any ω ∈ U and define the multifunction W : [a, b]→ 2Rnby

W (t) =ω ∈ U : f (t, x (t) ,ω) = x ′ (t) if x ′ (t) ∈ F (t, x (t))ω otherwise.

Remark that W (t) is a compact subset and W (t) = ω for t in aset of null Lebesgue measure.

Let u (t) be the first element of W (t) . Then by Proposition 2 ofSection 2 one has that t → u (t) is measurable, u (t) ∈ W (t) hence

x ′ (t) = f (t, x (t) , u (t)) ,

and x (.) is a trajectory of the control system (37) .


Fundamental properties of trajectories

Let f , U satisfy the basic assumption (H) and let (0, x) ∈ Ω.For any measurable control u : [0,T ]→ U, the Cauchy problem

x ′ = f (t, x , u (t)) , x (0) = x (41)

is equivalent tox ′ = g (t, x) , x (0) = x . (42)

where g (t, x) = f (t, x , u (t)) is suffi ciently regular to have a localsolution x (., u) : [0, δ]→ Rn of (42).



To study how the solution x (., u) depends on u, we first consider theglobally bounded case, assuming(H∗) : The set U ⊂ Rm of control values is compact, the functionf : R×Rn × U → Rn is continuous in all variables and continuouslydifferentiable with respect to x and satisfies

‖f (t, x ,ω)‖ ≤ C , ‖Dx f (t, x ,ω)‖ ≤ L, (43)

for some constants C , L ≥ 0 and all (t, x ,ω) ∈ R×Rn × U.Then we have the following global existence and continuousdependence result:



Theorem (imput-output map): Let the assumption (H∗) be satisfied.Then, for every T > 0 and u ∈ U the Cauchy problem (41) has aunique solution x (., u) : [0,T ]→ Rn.The imput-output map u → x (., u) is continuous fromL1 ([0,T ] ,Rm) to C ([0,T ] ,Rn) (the space of continuous functionsfrom [0,T ] into Rn.

The continuous dependence of trajectories on the control is a usefulbasic result.

Consider now a sequence of admissible controls (un)n∈N such thatthe sequence of the corresponding solutions (x (., un))n∈N convergeto x (.) uniformly on [0,T ] .

A natural question is whether the function x is a solution of theoriginal problem.



Example: Consider the system

x ′ (t) = u (t) , x (0) = 0, u (t) ∈ −1, 1 a.e. (44)

and for t ∈ R define

un (t) =

1 if sin (nt) ≥ 0−1 if sin (nt) < 0.

Then x (., un) converges to x ≡ 0 uniformly for all t ∈ R but x ≡ 0 isnot a trajectory of (44) .

We can conclude that the set of trajectories of (44) is not closed.



The closure of the set of trajectories of a control system is beststudied within the framework of differential inclusions.Let F : R×Rn → 2Rn

,

F (t, x) = f (t, x ,ω) : ω ∈ U (45)

be the multifunction associated to the control system (41) and let

x ′ ∈ F (t, x) (46)

be the differential inclusion defined by F (which is equivalent to tothe control system (41) , by Filippov’s lemma).We have the following resultTheorem (Closure of the set of trajectories): Assume themultifunction F : R×Rn → 2Rn

is Hausdorff continuous withcompact convex values. Then the set of trajectories of (46) is closedin C ([0,T ] ,Rn) .



Corollary: Let the basic assumption (H) hold. Let (xn (.))n∈N be asequence of solutions to

x ′ = f (t, x , u) , u (t) ∈ U (47)

converging to x (.) uniformly on [0,T ] . If the graph(t, x (t)) : t ∈ [0,T ] is entirely contained in Ω and the sets

F (t, x) = f (t, x ,ω) : ω ∈ U

are convex, then x (.) is a trajectory of the control (47)



The question here is what happens when the set U of admissiblecontrols is changed.

For example, assume that instead of using all measurable controlsu : [0,T ]→ [−1, 1] we can only use piecewise constant controls, orsay, control taking only the two values +1 , −1.Do these limitations substantially reduce our ability to control thesystem?Some results in this direction are presented below,

Theorem (density of trajectories with piecewise constant controls)Let f , U satisfy the basic hypothesis (H) . Then the family oftrajectories of (47) corresponding to piecewise constant controls isdense in the set of all solutions of (47) (with measurable controls).



Proof. Indeed, given any control u ∈ U assume that thecorresponding solution x (., u) of

x ′ = f (t, x , u (t)) , x (0) = x

is defined on [0,T ] . Construct a sequence of piecewise constantcontrols un ∈ U converging to u in L1. If f satisfies the global bounds(H∗) , by the continuity of the imput-output map u → x (., u) thecorresponding trajectories x (., un) converge to x (., u) uniformly on[0,T ] .To prove that the uniform convergence holds also in the general case,it suffi ces to consider an auxiliary function f + defined by

f + (t, x) =

φ (t, x) f (t, x , u) if (t, x) ∈ Ω0 if (t, x) /∈ Ω

which satisfies (H∗) and coincide with f for (t, x) in a neighborhoodof the graph

(t, x (t, u)) : t ∈ [0,T ] .Vasile Staicu (University of Aveiro) An introduction to Mathematical Theory of Control UNICA, May 2018 88 / 120

The result then holds for the system

x ′ = f + (t, x , u (t)) , x (0) = x

hence for the original system

x ′ = f (t, x , u (t)) , x (0) = x

as well.Theorem (approximation) Let f , U satisfy the basic hypothesis(H) . Let U ′ be a closed subset of U such that

f (t, x ,ω) : ω ∈ U ⊂ cof (t, x ,ω) : ω ∈ U ′

∀ (t, x) ∈ Ω.

(48)Then every trajectory of

x ′ = f (t, x , u (t)) , x (0) = x , u (t) ∈ U

can be uniformly approximated by trajectories of

x ′ = f (t, x , u (t)) , x (0) = x , u (t) ∈ U ′.


Remark. An important case where the key assumption (48) holds isthe following. Assume that the control system is linear with respectto u :

x ′ = h (t, x) + A (t, x) · u,each A (t, x) being a n×m matrix.Let U ′ be a compact subset of U and call F and F ′ the correspondingmultifunctions:

F (t, x) = f (t, x ,ω) : ω ∈ U andF ′ (t, x) =

f (t, x ,ω) : ω ∈ U ′

.

Then by linearity,

U ⊂ coU ′ =⇒ F (t, x) ⊂ coF ′ (t, x) .

This implication, however, is usualy false when f is nonlinear.


Example: Let h, g be smooth vector fields on Rn. Then the set oftrajectories of

x ′ = h (x) + u (t) g (x) , x (0) = 0, u (t) ∈ −1, 1 a.e.

is dense on the set of trajectories of

x ′ = h (x) + u (t) g (x) , x (0) = 0, u (t) ∈ [−1, 1] a.e.

becauseco −1, 1 = [−1, 1] .


Example: On R2 consider the systems(x ′1, x

′2

)=(u, 1− u2

), u (t) ∈ [−1, 1] , (x1, x2) (0) = (0, 0) (49)

(x ′1, x

′2

)=(u, 1− u2

), u (t) ∈ −1, 1 , (x1, x2) (0) = (0, 0) .

(50)Then coU ′ = U, but the set of solutions of (50) is not dense on theset of solutions of (49) .Indeed, taking u (t) = 0 one obtain a solution of (49)

(x1, x2) (t) = (0, t) ,

This solution cannot be approximated by trajectories of (50) , becauseu (t) ∈ −1, 1 implies x2 (t) = 0


Reachable sets, controllability

For Ω ⊆ Rn, U ⊆ Rm and f : Ω× U → Rn consider the nonlinearcontrol system

x ′ = f (t, x , u) , u (.) ∈ U (51)

where the set of admissible controls is defined as

U := u : [0, tu ]→ U : u measurable . (52)

For u ∈ U and (0, x0) ∈ Ω let x (., x0, u) be the solution of theCauchy problem

x ′ = f (t, x , u (t)) , x (0) = x0. (53)

Definition. Let x1, x2 ∈ Rn. We say that x2 is reachable from x1 orthat x1 is transferable to x2 at time t if there exists an admissiblecontrol u : [0, t]→ U such that

x2 = x (t, x1, u) .



The set

R (t, x0) = x ∈ Rn : x = x (t, x0, u) for some u ∈ U

is called the reachable set from x0 at time t.

For K ⊆ Rn we define

R (t,K ) =⋃x0∈K

R (t, x0) (54)

the set of points reachable at time t from a point of K .



Definition We say that the system (51) is locally controllable at x0 ifx0 is an interior point of R (t, x0) for each t > 0 :

x0 ∈ int R (t, x0) for all t > 0,

We say that the system (51) is globally controllable at x0 if⋃t>0

R (t, x0) = Rn.


Minimum time function

Let R (0) =⋃t>0

R (t, 0) be the set of all points reachable from the

origin 0 ∈ Rn.

The function T : Rn → [0,∞] defined by

T (x) =inf t ≥ 0 : x ∈ R (t, 0) if x ∈ R (0)+∞ if x ∈ Rn\R (0)

is said to be the "minimum time function" function.


Minimum time problem

The minimum time problem consists in:(i) Determine T (x) for each x ∈ R (0) ,(ii) Determine u∗ ∈ U such that

x = x (T (x) , 0, u∗) . (55)

A control u ∈ U satisfying (55) is called an optimal control withrespect to x .


The Bellman principle of optimality

Theorem (The Bellman principle of optimality) If u∗ ∈ U is anoptimal control with respect to x1 ∈ R (0) and if T2 < T (x1) andx2 = x (T2, 0, u∗) then u∗ ∈ U is an optimal control also with respectto x2.

Proof: Assume that is possible to reach x2 from the origin at timeT < T2, following the trajectory corresponding to a control u ∈ U ,that is

x2 = x(T , 0, u

).


The Bellman principle of optimality

Define the control

u (t) =

u (t) if t ∈[0, T

]u∗(t + T2 − T

)if T ≤ t ≤ T (x1)−

(T2 − T

)and observe that the trajectory corresponding to u allows to reach x1from the origin 0 at time

T (x1)−(T2 − T

)< T (x1) ,

in contradiction with the definition of T (x1) .


Compactness of the reachable sets

The following result concerning the compactness of reachable set willbe useful to prove existence of optimal controls.

Theorem (Compactness). Let Ω ⊆ R×Rn, U ⊆ Rm andf : Ω× U → Rn satisfy the basic hypothesis (H) . Assume that thereexists a compact set Ω ⊆ Ω such that the graphs of all trajectoriesdefined on [0,T ] of the control system (51) starting from x0 ∈ Rn

are contained in Ω and that all the setsF (t, x) = f (t, x ,ω) : ω ∈ U are convex, then, for everyτ ∈ [0,T ] , the reachable set R (τ, x0) is compact.


Mayer optimal control problem

We consider the control system

x ′ = f (t, x , u) , u (.) ∈ U (56)

where

U := u : R→ Rm : u measurable, u (t) ∈ U ∀t . (57)

Given an initial state x0, a set of admissible terminal conditionsS ⊆ R×Rn, and a cost function φ : R×Rn → R we consider theoptimization problem

min φ (T , x (T , u)) : u ∈ U defined on [0,T ] ,T > 0 (58)

with the initial and terminal constraints

x (0) = x0, (T , x (T )) ∈ S . (59)


Mayer optimal control problem

When as in (58) , the performance criterion depends only on terminaltime T and on the terminal point x (T ) of the trajectory, we say thatthe problem is in Mayer form.

Remark that the maximization problem

max ψ (T , x (T , u)) : u ∈ U defined on [0,T ] ,T > 0

is equivalent to (58) choosing φ = −ψ.


Existence of optimal controls for Mayer problem

Under suitable hypotheses, we shall prove the existence of anadmissible control u∗ ∈ U whose corresponding trajectoryx∗ (.) = x (., u∗) satisfies the constraints (58) , that is,

x∗ (0) = x0, and (T , x∗ (T )) ∈ S ,

and yields the minimum in (58) , that is,

φ (T , x∗ (T )) = min φ (T , x (T , u)) : u ∈ U defined on [0,T ] ,T > 0 .

The key assumption will be the convexity of the sets

F (t, x) = f (t, x ,ω) : ω ∈ U (60)

which guarantees the closure of the set of trajectories of (56) .



We start by considering the simplest case, assuming that the terminaltime T > 0 is fixed, and the problem (58)− (59) takes the form

min φ (x (T , u)) : u ∈ U , x (0) = x0, x (T ) ∈ S ,

where φ is continuous and S is a closed subset of Rn.

If the assumptions of Theorem giving the compactness of reachableset hold and the set of trajectories reaching S is nonempty, then anoptimal control exists.



Indeed, by Theorem on compactness of reachable sets, the set R (T )is compact, hence there exists a point xmin ∈ R (T ) ∩ S where φ isminimal, that is,

φ (xmin) = min φ (x (T , u)) : u ∈ U .

Any control u∗ ∈ U that steers the system to the point xmin, i.e.,such that

x (T , u∗) = xmin

is clearly optimal.



A similar result holds for the more general problem (56)− (59) .On the control system (56) we make the following assumptions,somewhat stronger than the assumption (H) . We assume:(H)

: The set U ⊂ Rm of control values is compact, the function

f : 0,∞)×Rn ×U → Rn is continuous in all variables, continuouslydifferentiable with respect to to x and satisfies

‖f (t, x , u)‖ ≤ C (1+ ‖x‖) , (61)

for some constants C ≥ 0 and all (t, x , u) ∈ R×Rn × U.



Then we have the following existence result:

Theorem. Let the assumption(H)hold. Assume that the sets of

velocities in (60) are convex, the cost function φ is continuous, andthe target set S is compact and contained in the strip [0,T ]×Rn. Ifsome trajectory x (.) satisfying the constraints (59) exists, then theproblem (56)− (59) has an optimal solution.Proof: By assumption, there exists at least one admissible trajectoryreaching the target set S .Therefore, we can construct a sequence of controls un : [0,Tn ]→ Uwhose corresponding trajectories xn (.) starting at x satisfy

(Tn, xn (Tn)) ∈ S , limn→∞

φ (Tn, xn (Tn)) = infu,T

φ (T , x (T , u)) . (62)



Since S ⊆[0,T

]×Rn, we have Tn ≤ T for every n. We can now

prolong each function xn to the entire interval[0,T

]by setting

xn (t) = xn (Tn) for t ∈[Tn,T

].

By the assumption(H), all trajectories satisfy the uniform bound

|xn (t, u)| ≤(eCt − 1

)+ eCt ‖x0‖ for t ∈

[0,T

]. (63)

Since f is uniformly bounded on bounded sets, the sequence xn (.) isuniformly Lipschitz continuous. Using Ascoli’s compactnesstheorem,by possible taking a subsequence, we can assume that

Tn → T ∗ for some T ∗ ≤ T and xn (.)→ x∗ (.) uniformly on [0,T ∗] .



Because if the convexity of the sets F (t, x) , we have that x∗ (.) is anadmissible trajectory of (56) , that is, there exists a controlu∗ : [0,T ∗]→ U such that

dx∗ (t)dt

= f (t, x∗ (t) , u∗ (t)) for a.e. t ∈ [0,T ∗] .

Clearly x∗ (0) = x0. Since S is closed, the first relation in (62) implies

(T ∗, x∗ (T ∗)) = limn→∞

(Tn, xn (Tn)) ∈ S .

Finally, from (62) and the continuity of φ it follows

φ (T ∗, x∗ (T ∗)) = limn→∞

φ (Tn, xn (Tn)) = infu,T

φ (T , x (T , u))

therefore, the control u∗ is optimal.


Remark: The above proof is a typical example of the Direct Methodfor proving existence of optimal solutions. The basic steps are:

1 Construct a minimizing sequence xn (.) ;2 Show that some subsequence converge to a function x∗ (.) ;3 Prove that x∗ (.) is an admissible trajectory and satisfies the bondaryconditions;

4 Prove that x∗ (.) attainsthe minimum value of the optimizationprobem.


Remarks:1 The assumption that S is compact is used to ensure that the domains[0,Tn ] are uniformly bounded. The theorem still holds if S is closedand φ (T , x)→ ∞ as T → ∞.

2 The continuity of φ can be replaced by lower semicontinuity.


Necessary conditions

The aim of this section is to derive necessary conditions in order thata trajectory x∗ = x (., u∗) be optimal for the Mayer problem

max ψ (x (T , u)) : u ∈ U (64)

subject tox ′ = f (t, x (t) , u (t)) , x (0) = x (65)

the set of admissible controls being

U := u : [0,T ]→ U : u measurable . (66)

As usual Ω ⊆ R×Rn is an open set, U ⊆ Rm , f : Ω× U → Rn iscontinuous on Ω× U with continuous Jacobian matrix Dx f of firstorder derivatives with respect to x . The function ψ : Rn → R isdifferentiable. No assumption is made on the set U ⊆ Rm . Inparticular we may well have U = Rm .


Pontryagin maximum principle

For the above problem, where the terminal time T is fixed and noconstraints are imposed on the terminal point the Pontryaginmaximum principle (for short, PMP)can be stated in a particularlysimple form.Theorem (PMP, free terminal point) In connection with themaximization problem (64)− (66), let u∗ be a bounded admissiblecontrol whose corresponding trajectory x∗ = x (., u∗) is optimal. Callp : [0,T ]→ Rn be the solution of the adjoint linear equation

p′ (t) = −p (t) ·Dx f (t, x∗ (t) , u (t)) , p (T ) = ∇ψ (x∗ (T )) . (67)

Then the maximality condition

p (t) f (t, x∗ (t) , u∗ (t)) = max p (t) f (t, x∗ (t) ,ω) : ω ∈ U(68)

holds at almost every time t ∈ [0,T ] .In the above theorem x , f , v represent column vectors, Dx f is a n× nmatrix, while p is a row vector.


In coordinates, the equations (67) and (68) can be rewritten as

p′i (t) = −n

∑j=1pj (t)

dfj (t)dxi

(t, x∗ (t) , u∗ (t)) , pi (T ) =dψ

dxi(x∗ (T ))

(69)

n

∑i=1pi (t) fi (t) (t, x∗ (t) , u∗ (t)) =

= max

n

∑i=1pi (t) fi (t) (t, x∗ (t) ,ω) : ω ∈ U

(70)

Remark. The pevious theorem remain valid if one replace max bymin .


The previous result motivates a practical method for finding optimalsolutions for the problem (64)− (66) :

1 First, define the function u = u (t, x , p) ∈ U by mean of the equality

pf (t, x , u (t, x , p)) = max pf (t, x ,ω) : ω ∈ U (71)

2 Then solve the system of 2n equationsx ′ = f (t, x , u (t, x , p))p′ = −pDx f (t, x , u (t, x , p))

(72)

with boundary conditions

x (0) = x , p (T ) = ∇ψ (x (T )) . (73)

3 By the previous theorem, if a bounded optimal control exists, it mustbe found among the solutions of (72)− (73) .


Remarks

In general this method encounters two main dificulties:

1 The map (t, x , p)→ u (t, x , p) , implicitely defined by the maximalitycondition (71) , may be multivalued or discontinuous.

2 The problem (72)− (73) is not a Cauchy problem but a (usuallyharder) two points boundary value problem. Indeed, the initial valuep (0) is not explicitely known. Instead, an equation is given involvingthe terminal values p (T ) and x (T ).

3 In particular cases, the equations for p and x can be uncoupled, and asolution is easily found


Example: linear pendulum with external force

Let q be the position of a linearized pendulum, controlled by anexternal force u, with magnitude constraint

|u (t)| ≤ 1, ∀t.Assuming that the initial position and velocities are both zero, themotion is determined by the quations

q′′ (t) + q (t) = u (t) , q (0) = q′ (0) = 0.

We wish to maximize the displacement q (T ) at a fixed terminal timeT .Introducing the variables x1 = q, x2 = q′, the opimization problemcan be formulated as

max x1 (T , u) : u ∈ Uwhere the dynamics is described by

x ′1 = x2 x1 (0) = 0x ′2 = −x1 + u x2 (0) = 0


The set of admissible controls is

U = u : [0,T ]→ [−1, 1] : u measurable

In this case, the adjoint equations (70) take the formp′1 = p2 p1 (T ) = 1p′2 = −p1 + u p2 (T ) = 0.

These equations can be solved for p independently of x , yielding

p1 (t) = cos (T − t) , p2 (t) = sin (T − t) .

By (71) , the optimal control u∗satisfies

p1x2 + p2 (−x1 + u∗) = max p1x2 + p2 (−x1 +ω) : |ω| ≤ 1 .

Therefore, the optimal control is given by

u∗ (t) = sign (p2 (t)) = sign (sin (T − t)) .


References

J. P. Aubin and A. Cellina, Differential inclusions, Springer-Verlag,1984;

A. Bressan and B. Picoli, Introduction to the Mathemtical Theory ofControl, AIMS Series in Applied Mathematics, 2007;

F. H. Clarke, Yu. S. Ledyaev, R. J. Stern, P. R. Wolenski, Nonsmoothanalysis and Control Theory, Springer-Verlag, New York, 1998.

A. F. Filippov, Differential equations with discontinuous right-handside, Kluwer, Doldrecht, 1988.

L. Cesari, Optimisation - Theory and Applications, Springer, 1983;


References

E. B. Lee and L. Marcus, Foundations of Optimal control theory,Willey, 1967;

P. D. Loewen, Optimal control via nonsmooth analysis, C. R. M.Proceedings & Lecture Notes, Amer. Math. Soc., Providence, 1993.

J. Macki, A. Strauss, Introduction to optimal control theory,SpringerVerlag, New York, 1982.

L. S. Pontryagin, V. Boltyanskii, R. V. Gamkrelidze, E. F. Mishenko,The Mathematical Teory of Optimal processes, Wiley, 1962


Documents

Vasile Staicu - dottorati.unica.it · Vasile Staicu (University of Aveiro) An introduction to Mathematical Theory of Control UNICA, May 2018 15 / 120. Example 2: cart on a rail Consider