Universidad del Paıs Vasco - Euskal Herriko UnibertsitateaBCAM-Basque Center for Applied Mathematics

Master in Computational Engineering, andIntelligent Systems

Master Graduation Project

Optimal control and turnpikeproperties of the Lotka Volterra

model

Aitziber Ibanez

Supervised by

Enrique Zuazua

Acknowledgements

First, I want to thank strongly my tutor Enrique Zuazua for the integration withinthe team, the availability he showed and the advice he gave me throughout this workexperience. Also I wish to express my sincere gratitude to all the PDEs, Control andNumerics team for their assistance and their warm hospitality.

This research is supported by the Advanced Grants NUMERIWAVES/FP7-246775of the European Research Council Executive Agency, FA9550-14-1-0214 of the EOARD-AFOSR, PI2010-04 program of the Basque Government and the MTM2011-29306-C02-00Grant of the MINECO. Also by the Basque Government through the BERC 2014-2017program and by Spanish Ministry of Economy and Competitiveness MINECO: BCAMSevero Ochoa excellence accreditation SEV-2013- 0323.

Contents

1 Introduction 51.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.3 Objetive of this work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 The Lotka-Volterra model 72.1 D′Ancora’s paradox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Generalizations of the Lotka-Volterra model . . . . . . . . . . . . . . . . . 122.3 Different controls for the model . . . . . . . . . . . . . . . . . . . . . . . . 132.4 Another applications of the Lotka-Volterra model . . . . . . . . . . . . . . 15

3 Optimal Control applied to the Lotka-Volterra system 163.1 Problem 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.1.1 Existence of solutions . . . . . . . . . . . . . . . . . . . . . . . . . . 183.1.2 Controllability of the system . . . . . . . . . . . . . . . . . . . . . . 203.1.3 The optimal control problem . . . . . . . . . . . . . . . . . . . . . . 203.1.4 Numerical resolution . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.2 Problem 2: Time optimization and the bang-bang property . . . . . . . . 233.2.1 Numerical resolution . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4 Turnpike property 274.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.2 Turnpike theorem for nonlinear systems . . . . . . . . . . . . . . . . . . . . 28

4.2.1 Example: Linear case . . . . . . . . . . . . . . . . . . . . . . . . . . 304.3 Aplication to Lotka-Volterra model . . . . . . . . . . . . . . . . . . . . . . 304.4 A variant of an indirect method using the turnpike property . . . . . . . . 35

4.4.1 A brief comparison between indirect and direct methods . . . . . . 354.4.2 The indirect shooting method . . . . . . . . . . . . . . . . . . . . . 364.4.3 Variant of the shooting method . . . . . . . . . . . . . . . . . . . . 374.4.4 The turnpike modification applied to the Lotka-Volterra model . . . 37

5 Conclusions 385.1 Comparison with the results obtained for Lotka-Volterra system with IPOPT 395.2 A possible improvement of the indirect method using the turnpike . . . . . 42

6 Annex 436.1 Problem 1 of section 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436.2 Problem 2 of section 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.3 Classic shooting method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476.4 Turnpike theory: linear example . . . . . . . . . . . . . . . . . . . . . . . . 50

2

6.5 IPOPT code to solve the Lotka-Volterra static problem . . . . . . . . . . . 516.6 IPOPT Codes to solve the turnpike problem of Lotka-Volterra model . . . 526.7 Implementation of the variation of indirect shooting method . . . . . . . . 54

3

Abstract

The Lotka-Volterra equations, also known as predator-prey equations, are a differentialnonlinear system of two equations, and are used to model biological systems where twospecies interact: a prey and a predator. In this work, in spite of analysing it, we treatit as an optimal control problem and we study how to apply the Potryagin MaximumPrinciple to it and, on the other hand, solve it using the routine IPOPT. Our final goalis to study the Turnpike property of the Lotka-Volterra model and to use it to improvethe classic indirect shooting method.

4

1 Introduction

1.1 Preliminaries

The Control Theory is an interdisciplinary research area which studies the behaviourof dynamic systems ([10]), in which different mathematical methods and concepts appear,with many applications in our life.

Suppose we want to control some system modeled by the following state equation:

A(y) = f(v)

In this case, y is the state vector and v is the control which belongs to the set ofadmissible controls Uad. Suppose that A : D(A) ⊂ Y −→ Y and f : Uad −→ Y are twomappings. A represents the equations y have to satisfy and f reflects the way in whichthe control affects the system.

For example, let imagine a building, where, for temperature control, we have severalradiators, and some sensors for measuring the temperature. Suppose we have model thisproblem through the following linear system:{

X(t) + AX(T ) = Bu(t)X(0) = X0

where A ∈ RN×N , B ∈ RN×M , X(t) ∈ RN , u(t) ∈ RM . The X(t) state vector representsthe temperature registered by the N sensors of the building in time instant t, and thematrix B represents the way the control u(t) influences the temperature by the regulationof the M radiators. If none control is applied, the system is reduced to X(t)+AX(T ) = 0,a linear differential system whose solution (if it exists) gives the evolution of the temper-ature in the building with all the radiators switched off. Once we choose the control u(t),when we solve the differential system (if the solution exists) we get the trajectory asso-ciated to the control u(t), which can be interpreted as the evolution of the temperaturealong the time.

Suppose the initial temperature of the building is 0, is it possible to get the tem-perature to 30 grades manipulating the radiators? That is to say, given an initial pointX1 ∈ RN , is there a control u(t) such that the associated trajectory to the control joinsX0 and X1? This issue is a control problem.

If that is true, is there a control u whose associated trajectory joins X0 and X1

and also minimize some given function C(u) (called cost function)? That would be anoptimal control problem. For example, in the case of the building, C(u) may representthe economical cost of maintaining the radiators switched on.

In recent years many books about Control Theory have been written, and for adeeper introduction the reader can look to [15], [27], [30], [38] or [48].

5

1.2 History

The progress in Control Theory has always been linked to the different problemsthat have appeared throughout history. Maybe, the most significant events in its evolutionhave been the following ones:

• The concern of the Arabs and Greeks for measuring the time (300BC-1200AC)

• The beginning of the Industrial Revolution in Europe

• The rise of communication systems and the two World Words

• The beginning of the spatial and digital era, in 1957

In ancient times, before this discipline had an own name, Greeks and Romans usedseveral elements of control: from the water clock to the construction of aqueducts. Per-haps, one of the most curious applications of control of this time period was the onecarried out by Heron of Alexandria, who built automatic doors for a temple. The inven-tion worked as follows: when the fire of the shrine was lit, the water of an undergrounddeposit was heated and it passed to tub, which, as it weight increased, descended andopened the door pulling on some rope. For closing the door, they only have to extinguishthe fire. Because the system was hidden, the mysticism surrounded the temple and peo-ple attributed the phenomenon to the enormous power of gods (for more details of thishistory [47]).

Before the Industrial Revolution, the design of control systems were made using“trial and error” method. However, in mid-1800 they began to apply the mathematicalanalysis to this problems, and it became into the discipline today known as ControlTheory.

The centrifugal regulator (or the steam engine) invented by James Watt is, withoutany doubt, the most classic example of an automatic control system. This is mainlydue to the diffusion its use reached, and also because it was the first to be analysedmathematically, which was done by Maxwell in his work “On governors” [32].

During the thirties, the Control Theory had a significant development regarding toautomatical control and analysis techniques, because of the large number of applicationsit had in telephonical systems, distribution systems in power plants, aircraft development,chemical industry and other areas. At the end of that decade, two very different methodsemerged, the first one based on the use of differential systems and the second one on theamplitude and “input” and “output” phases analysis.

At this time, several institutions began to work in this discipline, as ASME (Amer-ican Society of Mechanical Engineers) or the British IEE (Institution of Electrical Engi-neers).

6

The methods developed before 1960 are known now as Classic Control Theory. Bell-man’s work about dynamic systems, Kalman filtering techniques, the algebraic approachfor lineal systems and Pontryagin Maximum Principle for nonlinear Optimal ControlProblems established the foundations of Modern Control Theory.

References for this secion: [15] and [47].

1.3 Objetive of this work

As it has been mentioned, the Control Theory has many applications in variousareas of science, like the Biology (see [29] for different examples of those applications). Inthis work, we show an application of Optimal Control Theory to a biological model: theLotka-Volterra model, also known as predator-prey model.

In section 2, we review different works about this model, as well as some of itsgeneralizations and we mention some different ways to apply a control on it. In section3, first we study how to apply Pontryagin Maximum Principle to two different problems,and then we solve them using the IPOPT routine, which uses a direct method.

In section 4 we study the Turnpike phenomenon in a concrete Lotka-Volterra prob-lem. Finally, the Turnpike property is used to developed the variation of the classicindirect shooting method proposed in [50] to solve the Lotka-Votlerra problem.

2 The Lotka-Volterra model

The Lotka-Volterra equations, also known as predator-prey equations, are a differ-ential nonlinear system of two equations, and they are used to model biological systemswhere two species interact: a prey and a predator. Both species grow according to thefollowing system:

dx1

dt= x1(α− βx2)

dx2

dt= −x2(γ − δx1)

(1)

where x1 represents the prey population, x2 the predator one, and α, β, γ and δ are realpositive parameters that describe the interactions of both species.

2.1 D′Ancora’s paradox

7

Alfred J. Lotka developed the model (1) in 1926 to describe certain chemical re-actions, (see [31]), applying the chemical law for mass action. The same model wassimultaneously developed by the mathematician Vito Volterra, but with another objec-tive: to explain the evolution of the fish population of the Adriatic Sea. After finishingthe First World War (1914-1918), the biologist Umberto D′Ancora found that the pop-ulation of mackerels and tunas of that zone had decreased, while what he expected wasthe opposite, because during the war the fishing level had greatly reduced. On the otherhand, he also found that the population of selachians (sharks, rays...) had growth. Whatdistinguishes the selachians from the other fish is that the selachians are predators, whilethe tuna and mackerels are their food, (that is, they are the prey species).

In the following table we can see the data collected by D′Ancora in the port ofFiume, Italy, during the years 1914-1923. The data gives the percentage-of-total-catch ofselachians.

1914 1915 1916 1917 1918 1919 1920 1921 1922 192311.9% 21.4% 22.1% 21.2% 36.4% 27.3% 16.0% 15.9% 14.8% 10.7%

Table 1:

After exhausting all possible biological explanations of this phenomenon, D′Ancoraturned to his college Vito Volterra, who formulated the model of the growth of the selachi-ans (the predator population) and the mackerels and tunas (the prey population). Hiswork was published in 1926 [51]. As a curiosity, the correspondence between both scien-tists can be read in the chapter Letters between Umberto D′Ancora and Vito Volterra of[25].

From now, x1(t) will denote the population of tunas and mackerels, which are thepreys, and x2(t) the sellachians, which are the predators. Volterra developed the model(1) basing on the following hypothesis:

• The preys don’t compete very intensively among themselves for their food or space,so in the absence of predators, the prey population grows exponentially (Accordingto the Malthusian law of population growth):

dx1

dt= x1α, α > 0

x1(t) = x1(0)eαt

• In absence of prey, the size of the predator population decreases in proportion to itsize:

8

dx2

dt= −x2γ, γ > 0

x2(t) = x2(0)e−γt

• The predator population depends on the number of contacts per unit of time betweenpredator and prey, which is proportional to x1 and x2, so the following term mustbe added to the predator equation:

δx1x2, δ > 0

• On the other hand, such contacts are unfavourable for the prey, so the followingterm must be added to the prey equation:

−βx1x2, β > 0

The resulting system is (1), which has two equilibrium points: the origin (0, 0) andx = (γ/δ, α/β), the first one is a saddle-point, while the second one is a stable equilibriumpoint (see [14] and [24] chapter 1). On the other hand, the trajectories of the first quadrant(which are the interesting ones, x1(t) > 0, x2(t) > 0, because the rest are not biologicallyinterpretable) are closed orbits, except the point x and the axes. In Figure 1, we can seesome of those trajectories for α = β = γ = δ = 1.

Figure 1: Curves solution of (1), for (x1(0), x2(0)) = (1/2, 1/2), (x1(0), x2(0)) =(1/4, 1/4), and (x1(0), x2(0)) = (1/8, 1/8)

9

As it is proved in [7] (chapter 4) and [14] (Theorem 4.1), the solutions of the trajec-tories of the first quadrant are periodic.

Nevertheless, how can this model explain the data collected by D′Ancora? First, itmust be taken into account that the data of Table 1 is really an average over each yearperiod of the proportion of predators of the total volume of fishing:

x2

x1 + x2

To calculate the average values of the populations x1 e x2 in system (1) we applythe Lemma 2 of chapter 4.9 in [8]:

x1 =1

T

∫ T

0

x1(t)dt,

x2 =1

T

∫ T

0

x2(t)dt.

where T is the period of the system. Integrating, we get:

x1 =γ

δ, x2 =

α

β(2)

These averages correspond to the first five years: 1914-1918, the years of the War,when the fishing level was practically zero.

To consider the factor of the fishing, we must introduce a new constant εi, propor-tional to the fishing intensity, which is going to decrease ε1x1(t) the prey population andε2x2(t) the predator population, so we have a new system to modelling the evolution ofboth population during the years 1919-1923:

dx1

dt= x1(α− βx2)− ε1x1 = x1((α− ε1)− βx2)

dx2

dt= −x2(γ − δx1)− ε2x2 = −x2((γ + ε2)− δx1)

(3)

In [30] (chapter 4, Theorem 4.4) this problem is studied for the case ε1 = ε2, andthey solve that if α < ε1 both species extinguish after some time, while if α > ε1, thefollowing average values of x1 and x2 are got (see [30] and [8] (section 4.9)):

x1 = γ+ε2δ

x2 = α−ε1β

So, to understand the data of Table 1, it must be taken into account first, that wesuppose that the period of the system T is a year, and second, that before 1914 the fishing

10

data was adjusted to the model with fishing, while during war years, from 1914 to 1919the fish population is modeled by the model without fishing, the original Volterra model(1). After 1920, once again the data adjust to the fishing model, i. e., the average of bothpopulation were adjusted gradually to the averages of that model.

Consequently, a moderate amount of fishing ( α > ε1) actually increases the numberof prey on the average, and decreases the number of predators. Conversely, a reducedlevel of fishing increases the number of predators on the average. This result, which isknown as Volterra’s principle, explains the data of D’Ancora, and it is briefly expresed inTheorem 4.4 of [30]:

If two species are exterminated in direct proportion to the number of persons, thenthe mean value of preys increases and the mean value of predator decreases.

In the graphic we can see on blue the solutions of the system (1) (without fishing,that is, the model for the years 1914-1918), and on red the solutions corresponding to themodel with fishing (3)):

Figure 2: Blue curves: solution of (1), for (x1(0), x2(0)) = (1/2, 1/2), (x1(0), x2(0)) =(1/4, 1/4), and (x1(0), x2(0)) = (1/8, 1/8). Red curves: solutions of (3) for(x1(0), x2(0)) = (1/2, 1/2), (x1(0), x2(0)) = (1/4, 1/4), and (x1(0), x2(0)) = (1/8, 1/8).

Volterra’s principle explains also why some insecticide treatments, which destroyboth insect predators and their insect prey, actually increase the population of thoseinsects which are kept in control by other predatory insects.

11

2.2 Generalizations of the Lotka-Volterra model

In the case that in the ecosystem more than a single prey or predator populationinteracts, the Lotka-Volterra model can be generalized to a predator-prey system withseveral prey and predators:

dx1i

dt= x1i(αi −

k∑j=1

βijx2j)

dx2i

dt= −x2i(γi −

k∑j=1

δijx1j), i = 1, ..., k

In [37] (section 3.1) the equilibrium points of the system are studied, and they arealso calculated for which values those points are stable or unstable.

On the other hand, it is obvious that the Lotka-Volterra model (1) is not veryrealistic, since, among other things, the prey population grows exponentially in absenceof predators. In [14] (section 5) and [37] (section 3.3) a more realistic model can be found,where is taken into account that both species compete with themselves for food and space,and they add a new term in each equation:

dx1

dt= x1(α− βx2)− λx2

1

dx2

dt= −x2(γ − δx1)− µx2

2

The behaviour of the solutions of this system is more complex. Its equilibrium pointsare X0 = (0, 0), Xx0 = (α

λ, 0), X0y = (0, δ

µ), and Xxy = ( δβ+µα

γβ+λµ, γα−λδγβ+λµ

). Two cases mustbe distinguished:

• αγλ− δ < 0, so the point Xxy does not belong to the first quadrant and Xx0 is an

attractor point (graphic of the left of Figure 3)

• αγλ− δ > 0, then Xx0 is a saddle-point and Xxy belongs to the first quadrant and it

is a global attractor (graphic of the right of Figure 3)

In both cases, this system hasn’t got periodic orbits in the first quadrant (proves in[14] section 5).

12

Figure 3: Left: Case αγλ− δ < 0. Right: case αγ

λ− δ > 0, then Xx0

2.3 Different controls for the model

Once we have studied the system which models the evolution of both species, thenext question is how we can influence it, i. e., if by choosing a good mechanism we areable to lead the system from an initial state (x1,0, x2,0) to a certain prefixed objective, or adesired final point (x1,f , x2,f ) in T time. Shortly, we can consider it as a control problem.

In the case of D′Ancora’s problem, for example, fishing can be considered as a controlmechanism in Volterra’s system.

In general, for the predator-prey system we have the following control system, wherethe control u(t) is the hunting (or fishing):

dx1

dt= x1(α− βx2)− x1cx1ux1(t)

dx2

dt= −x2(γ − δx1)− x2cx2ux2(t)

(4)

where the constants cx1 , cx2 represent the maximum level of hunting or fishing in a timeinstant t for each population (cx1 = 1 means that there is no hunting limit).

Regarding ux1(t), ux2(t) control functions of this model, the most natural thing isask them to satisfy: ux1(t), ux2(t) ∈ [0, 1], where ux1(t) = 0 means that there is nohunting, and ux(t) = 1 hunting the maximum number of individuals allowed (in case ofcx1 = 1, it suppose to hunt the entire population in time instant t). It is natural to askto the control to be bounded, there will always be a maximum height of fishing.

Within this range of values, different restrictions can be imposed to the controlfunctions ux1 , ux2 : R −→ [0, 1], like asking them to be constants, or just being continuous.For example, in [43] a similar problem is proposed, but in that work the controls are only

13

allowed to take the values of 0 and 1, ux1 , ux2 : R −→ {0, 1}, that is to say, it is abinary control problem. Binary controls like this are interesting and very realistic in someapproaches, for example, if we fish, it is possible that our only two options are not fish,or cast the nets and fish as much as possible.

In the system (4), the control functions depend only on t (i. e., it is a open-loopcontrol), but in general they can depend on other factors, like on the population himself(i. e., a “feedback” or close-loop control). Feedback control is a control mechanism thatuses information from the state or the control to manipulate a variable to achieve thedesired result. In [42] (chapter 8, section 3) they analysed a generalized Lotka-Volterrasystem from the point of view of dynamical systems and they impose a feedback controlon the predator population:

x1 = x1(1− x1)− ax1x21+mx1

x2 = x2(−d+ bx21+mx2

)−H(x2)

where the function H determines the hunting, and has the form:

H(x2) =

{0 if x2 < C

h(x2−C)h+(x2−C)

, if x2 ≥ 0

here, C is the threshold population size that determines when harvesting starts or stops,and h is the rate of harvesting limit, and a, b, d and m are positive parameters, representingthe capteure rate of the prey, the prey conversion rate, the natural death rate of thepredator and the half-saturati

On the other hand, the problem (4) can be approached from another perspective, andbe treated as a optimization problem, defining an cost function representing the objectivewe want to reach and trying to find the control which get the state closer to that objectivein time T , under certain restrictions.

That objective can be to maximize one of the populations, or both (see [55] and [3])in final time T:

min− (x1(T ) + x2(T ))

Or, like is done in [43], try to get the solutions close to equilibrium point, adding acost function that penalizes the deviation to the equilibrium state X = (x1, x2):

minx1,x2,u∫ Tt0

(x1(t)− x1)2 + (x2(t)− x2)2dt

Another possibility is to want that in the final time T , both populations maintaina certain proportion:

minx1,x2,u|x2(T )− cx1(T )|2, c > 0

14

or to want that they maintain such proportion along the time interval [0, T ]:

min∫ T

0|x2(t)− cx1(t)|2dt, c > 0

It is also possible to desire to get close to some fixed final state minimizing the time(for example, as they do in [13]).

2.4 Another applications of the Lotka-Volterra model

Finally, it should be mentioned that the Lotka-Volterra model, besides being a classicexample of nonlinear dynamical systems, it has applications in other areas of science.

In epidemiology, the Lotka-Volterra system has been used to model the evolutionof certain microorganisms, such as the experiments Gause made in 1934 ([7] y [1]). Thisbiologist performed several experiments with some protozoos (the predator specie in thiscase) and yeast fungi (prey species), aiming to model the evolution of both species, andhe estimated the following system:

dx1dt

= x1(0.65− 0.0108x2)dx2dt

= x2(−0.32 + 0.0058x1)

However, as Gause recognized, the system was not perfect, and after doing somecycles, the protozoos destroyed the fungi population and died then because of lack of food.In other words, it was necessary to include some type of control in order to the survivalof both species. This is known today as Principle of competitive exclusionor Gause law ,postulate which states that two species in biological competition for the same resourcescannot coexist in a stable way if any other factor remains constant, and, therefore, thefate of the less efficient specie is the local extinction.

Chemistry is another field with several applications for the Lotka-Volterra model,as it was expected, since, as it has been mentioned, Lotka developed this system to modelcertain chemical reactions, like the following one [31]:

E + C1 = 2C1

C1 + C2 = 2C2

C2 = G

with the following system:

dn1

dt= k1En1n2 − k2n1k2

dn2

dt= k2n1n2 − k3n2

15

where the concentrations of E and G are supposed to be fixed, and n1 and n2 representthe concentration of C1 and C2 respectively. As in biology, this model can be generalizedin different ways from a chemical point of view, to make it more realistic ([26]).

In biology, this model has several applications (see [37] for an overview of thoseapplications, and [46] for a concrete case of an application of the Lotka-Volterra model),besides explaining D′Ancora’s paradox.

3 Optimal Control applied to the Lotka-Volterra sys-

tem

The goals of this section are double:

• See how to apply Pontryagin Maximum Principle to Lotka-Volterra model

• Solve two different problems using a direct method to see the solutions, and in caseof Problem 2, to illustrate the bang-bang property in a time optimal control problem

The general notation for an optimal control problem is:

x = f(x(t), u(t)) (5)

with f : Rn × Rm −→ Rn is C2, y u(t) ∈ L∞([0, T ];Rm). Let f 0 : Rn × Rm −→ R, a C2

function, for T > 0 the following cost function is defined:

CT (u) =

∫ T

0

f 0(x(t), u(t))dt (6)

and R : Rn × Rn −→ Rk a C2 mapping, which represents de initial and finalconditions the problem has to satisfy:

R(x(0), x(T )) = 0 (7)

There are several books about Optimal Control Theory. In this work [4], [27], [28],[35] and [48] have been used as reference.

To prove the controllability of a linear control system, Kalman theorem is used,which can be extended to prove the local controllability of a nonlinear system. But if wewant to solve the global controllability of a nonlinear control system, we have to use othertechniques, like the theory of Lie Algebra.

On the other hand, to analyse an optimal control problem, the most importanttheoretical result is the Pontryagin Maximum Principle, which completes the differential

16

system with the adjoin vector, which takes into account the cost function. The adjoinvector has a similar purpose as Lagrange multipliers in multivariate calculus, which addconstraints to the function of several variables to be minimized (or maximized). The Pon-tryagin Maximum Principle gives us the necessary conditions that the optimal trajectoryand its adjoin vector must satisfy. However, this principle does not give always a way todescribe the optimal solutions and we sometimes have to use technique based on geomet-rical analysis. This theorem can be found in chapter 7 of [48]. Also in [4] they explainhow to apply this theorem to different cases. Another good reference for the PontryaginMaximum Principle is [28].

In order to solve numerically an optimal control problem, there are mainly two kindsof approaches to solve optimal control problems:

• Direct methods: Consist in discretizing both the state and the control, and reducethe problem to a nonlinear optimization problem, in other words, in a dynamicprogramming problem ([5]), of the form:

minZ∈CF (Z)

where Z = {x1, ..., xN , u1, ..., uN}, C = {Z|gi(Z) = 0, i = 1, ..., r; gj(Z) ≤ 0, j =1, ...,m}, and gi functions represent the constraints for the state and the control

• Indirect methods: Consist in solving numerically the boundary problem obtainedfrom the Maximum Principle, and discretizing the system to solve it (for example,using a simple shooting method).

A good reference for the numerical methods in Optimal Control are [6], [16] and[19]. Also we can found general and brief explanations of them in chapter 9 of [48].

The problems below have been numerically solved with the optimization routineIPOPT [52], which uses a direct method, combined with the code AMPL [17].

For the resolution of any problem using the differentiation code AMPL, three filesmust be written: the first one, a .mod file, where the model is written; then a .dat one,specifying de data of the problem, and finally a .run file, which will be the executableone.

3.1 Problem 1

The problem we are going to study in this section is the following Lotka-Volerrasystem with fixed initial point.

17

dx1

dt= x1 − x1x2 − 0.4x1u

dx2

dt= −x2 + x1x2 − 0.2x2u

(x1(0), x2(0)) = (x1,0, x2,0) = (0.5, 0.7);

0 ≤ u(t) ≤ 1,

T = 30

(8)

along the time interval [0, T ], where x1(t) represents the prey population and x2(t) thepredator one. The control function u(t) represents the hunting, which, for this particularcase, has more influence on the prey population. We have chosen the parameters of thesystem like in [43].

3.1.1 Existence of solutions

For every control u ∈ L∞(0, T ), the Cauchy-Lipschitz theorem guarantees the exis-tence and uniqueness of a local solution, called maximal solution, in a maximal interval[0, tu] ⊂ [0, T ] (reference [23]). If tu < T , then the solution blows up in tu.

We want to prove that the solutions of (8) does not blow up in a finite time interval[0, T ] if the initial point verifies x1,0 > 0, x2,0 > 0. For that, first we are going to see thatthose solutions are positive in [0, tu], and second, that they are bounded.

To prove that the solutions remain positive if the initial point (x1,0, x2,0) are positive,we argue similarly to [8] for the case of Lotka-Volterra model without control (1). Thesystem has the particular family of solutions:

x1 = 0, x2 = x2,0 exp(−t− 0.2

∫ t

0

u(s)ds)

x1 = x1,0 exp(t− 0.4

∫ t

0

u(s)ds), x2 = 0

(9)

As 0 ≤∫ t

0u(s)ds ≤ t, both the x1 and the x2 axes are orbits of (8).

Let take a fixed control u(t) and let imagine that exists some trajectory (x1(t), x2(t))solution of (8), starting in the first quadrant, x1(0) > 0, x2(0) > 0, that it doesnot maintain positive in x2. That means that there exists some t1 > 0 such that(x1(t1), x2(t1)) = (x, 0). Is there another trajectory passing through the point (x, 0)in the same time instant t1? Yes, one of the form (9), because we always can choose x1,0

such that it satisfy:

18

x1,0 exp(t1 − 0.4

∫ t1

0

u(s)ds) = x

because exp(t1− 0.4∫ t1

0u(s)ds) is a well defined fixed number 6= 0 (as t1 6= 0 and u(t) are

fixed). Therefore, the uniqueness would not hold in that point, thus we have an absurd.The prove is analogous if we suppose the sign changes in the first coordinate.

So, by uniqueness every solution starting in the first quadrant: x1,0 > 0, x2,0 > 0,remains positive for every t.

Now we have to prove that every solution starting in the first quadrant is bounded.Let divide each equation in (8) by x1 and x2 respectively:

1

x1

dx1

dt= 1− x2 − 0.4u,⇒ d ln(x1)

dt= 1− x2 − 0.4u

1

x2

dx2

dt= −1 + x1 − 0.2u⇒ d ln(x2)

dt= −1 + x1 − 0.2u

Integrating in [0, t]:

ln(x1(t)) = ln(x1(0)) + t−∫ t

0

x2(s)ds− 0.4

∫ t

0

u1(s)ds

ln(x2(t)) = ln(x2(0))− t+

∫ t

0

x1(s)ds− 0.2

∫ t

0

u2(s)ds

So:

x1(t) = x1(0) exp(t−∫ t

0

x2(s)ds− 0.4

∫ t

0

u1(s)ds)

x2(t) = x2(0) exp(−t+

∫ t

0

x1(s)ds− 0.2

∫u2(s)ds)

We can deduce that:

x1(t) ≤ x1(0) exp(t)

x2(t) ≤ x2(0) exp(

∫ t

0

x1(s)ds) ≤ x2(0)t exp(x1(0) exp(t))

So we can assure that, if x1(0), x2(0) > 0, then the solutions are positive and fort <∞ the solutions do not blow up in [0, t].

19

3.1.2 Controllability of the system

Let imagine that we want to arrive to the final point (x1(T ), x2(T )) = (1; 1.4). Wecan treat it like a control problem, and add the final fixed end point, or we can solve itas an optimal control problem, and just try to get close to that final point. The last oneis a more practical way to solve the problem.

Nevertheless, to prove the controllability, we are going to use the property provedin [21] for controlled Lotka-Volterra model with n species and positive control. Accordingto it, once we have the system of n differential equations on the form:

x =

x1 0 ... 00 x2 ... 0... ... ... ...0 0 ... xn

(Ax+ a+ bu)

(where x ∈ Rn+, a ∈ Rn, A ∈ Rn×n, b ∈ Rn, u ∈ R+), if the matrix A is of rank n, the

system is controllable.In our case:

A =

(0 −11 0

); a =

(1−1

)so the condition is verified.

The controllability and the existence of solutions guarantees us the existence of atrajectory joining (0.5, 0.7) and (1, 1.4).

3.1.3 The optimal control problem

Even if we have controllability for the system (8), we are going to solve the problemfrom the point of view of the optimal control problem, and we are going to consider thefollowing cost functional:

J(u) =1

2

∫ T

0

u(t)2dt+ 2((x1(T )− 1)2 + (x2(T )− 1.4)2) (10)

with T = 30.Here,

∫ T0u(t)2dt represents the quadratic cost of travelling along xu trajectory, ap-

plying u control, and (x1(T ) − 1)2 + (x2(T ) − 1.4)2 function penalizes the deviation ofthe final state (x1(T ), x2(T )) from the chosen state (1, 1.4). The constant multiplying(x1(T ) − 1)2 + (x2(T ) − 1.4)2 means that more importance is given to get close to thedesired end point.

20

Proof of the existence of minimum of J(u)

We want to prove that for U = {u : [0, T ] −→ [0, 1]} set of controls, which aremeasurable in L2(0, T ) , it exists some u ∈ U ⊂ L2(0, T ) that the functional J(u) isminimum.

J(u) ≥ 0, ⇒ ∃ I = infu∈U J(u)Because of the supreme axiom, there exists a minimizing sequence (un)n∈N such that

J(un) ↓ I.On the other hand, as (un) is bounded, ∃ subsequence (un′) ∈ U such that (un′) ⇀ u

(weak convergence)We want to prove that that u is the minimum of J(u), i. e., that lim infn−→∞ J(un) ≥

J(u).

• lim infn−→∞∫ T

0u2n(s)ds =

∫ T0u2(s)ds because it is the norm of L2(0, T )

• The bounding of the solutions of (8) has been proved in section 3.1, so we candeduce that also x′ is it, so we are in the Sobolev space H1(0, T ). Let (xn) bethe sequence of the first component of the solutions corresponding one to each un,and x the corresponding to u. As we have said, (xn) is bounded, so there exists asubsequence (xn′) such that:

(xn′) ⇀ x in H1(0, T )

Because of the Rellich–Kondrachov Theorem ([9], Theorem 9.16) we now thatH1(0, T ) ⊂ C(0, T ) (compact injection), so:

xn → x in C0(0, T ) and it follows that xn(T )→ x(T ). We have proved:

lim infn−→∞(xn(T )− 1)2 = (x(T )− 1)2

• Analogous for lim infn−→∞(yn(T )− 1.4)2 = (y(T )− 1.4)2

�

Application of Pontryagin Maximum Principle

To analyse this optimal control problem (8) with the cost (10), as we have said,we are going to apply the Pontryagin Maximum Principle. According to it, there existsan absolutely continuous mapping p(.) : [0, T ] −→ Rn, called adjoin vector, and a realnumber p0 ≤ 0, with (p(.), p0) 6= (0, 0), such that, for every almost t ∈ [0, T ]:

x =∂H

∂p(t, x(t), p(t), p0, u(t))

p = −∂H∂x

(t, x(t), p(t), p0, u(t))

(11)

21

where H(t, x(t), p(t), p0, u(t)) =< p, f(t, x, u) > +p0f 0(t, x, u) is the Hamiltonian of thesystem, which for almost every t ∈ [0, T ] verifies the following maximization condition:

H(t, x(t), p(t), p0, u(t)) = maxv∈UH(t, x(t), p(t), p0, v) (12)

The Hamiltonian for the problem (8) with cost function (10)

H = p1(x1 − x1x2 − 0.4x1u) + p2(−x2 + x1x2 − 0.2x2u) +1

2p0u

2

Adjoin vector:

p0 = 0, (we take p0 = −1, without lost of generality)

p1 = (−1 + 0.4u)p1 + x2(p1 − p2)

p2 = (1 + 0.2u)p2 + x1(p1 − p2)

Besides that, deduced from the transversality conditions of the Pontryagin MaximumPrinciple, we get the following constraints for the extremes of the adjoin vector:

p1(T ) = 0, p2(T ) = 0

From the maximization condition of the Hamiltonian (12), we get:

u =

0 if −0.4x1p1 − 0.2p2x2 < 0

−0.4x1p1 − 0.2p2x2 if 0 ≤ −0.4x1p1 − 0.2p2x2 ≤ 11 if −0.4x1p1 − 0.2p2x2 > 1

(13)

So we have the following differential system:

x1 = x1 − x1x2 − 0.4x1u

x2 = −x2 + x1x2 − 0.2x2u

p1 = (−1 + 0.4u)p1 + x2(p1 − p2)

p2 = (1 + 0.2u)p2 + x1(p1 − p2)

(14)

where u can be replaced using (13), with the following boundary conditions:

x1(0) = 0.5;x2(0) = 0.7

p1(T ) = 0; p2(T ) = 0(15)

From the application of the Pontryagin Maximum Principle we know that any op-timal solution of the problem (8) with the cost (10) has to verify the boundary valueproblem (14) and (15). However, is not easy to know how the optimal solution is fromthe equations the Pontryagin Maximum Principle gives us. From here, we should use thenumerical methods to get a solution.

22

3.1.4 Numerical resolution

Now we are going to solve the problem numerically to illustrate the solution.In this section we are going to solve the problem using a direct method, using the

routine IPOPT. So, first the problem must be discretized.As it have been mentioned, to solve the problem using a direct method, the first

thing to do is to discretize the differential system (8). To do it, we use Euler implicitmethod (with N number of dicretizations):

We have used Euler implicit method to discretize this problem because it is easy toimplement in the AMPL language (whose rigidity difficults to implement a higher-ordermethod).

x1,i+1 = x1,i + ∆tx1,i+1(1− x2,i+1 − 0.4ui+1)

x2,i+1 = x2,i + ∆tx2,i+1(−1 + x1,i+1 − 0.2ui+1)

Discretization of objective function (10):

2((x1(N)− 1)2 + (x2(N)− 1.4)2) +1

2

N∑i=1

∆t(u(i)2)

And finally, the restrictions:

x1(0) = 0.5

x2(0) = 0.7

0 < u(i) < 1

We have to take into account that direct methods may diverge (see [49] section 2.4.3for a review of this issue). In [22] they prove the convergence of this discretization schemeto the continuous model (for a sufficiently small ∆t) .

We can see the computational results for these parameters in Figure 4. The minimumcost obtained is 0.0341461, and the final point is: (x1(T ), x2(T )) = (1.01464, 1.43263).

3.2 Problem 2: Time optimization and the bang-bang property

Let consider the system:

23

Figure 4: Solution of (8) with (10), with N=100000 time steps

dx1

dt= x1 − x1x2 − 0.4x1u

dx2

dt= −x2 + x1x2 − 0.2x2u

(x1,0, x2,0) = (0.5, 0.7), (x1,f , x2,f ) = (1, 1.4);

0 ≤ u(t) ≤ 1,

(16)

In this case, we consider the problem of stearing the initial condition (x1,0, x2,0) andthe final condition (x1,f , x2,f ) in the shortest possible time. So now the cost function is:∫ tf

0

dt; (17)

The prove of existence of solutions and the controllability of this system is analogousto the previous one.

The Maximum Principle is applied below for (16) with cost (17).Hamiltonian:

24

H = p1(x1 − x1x2 − 0.4x1u) + p2(−x2 + x1x2 − 0.2x2u) + p0

Adjoin vector:

p0 = 0,

p1 = (−1 + 0.4u)p1 + x2(p1 − p2)

p2 = (1 + 0.2u)p2 + x1(p1 − p2)

From the Hamiltonian maximization condition (12) we conclude:

u =

{1 if − 0.4x1p1 − 0.2p2x2 > 00 if − 0.4x1p1 − 0.2p2x2 < 0

Therefore, the time optimal control is switching between 0 and 1. This is known asbang-bang principle.

The bang-bang principle:This phenomenon appears in time optimal control problems (but not only in time optimalcontrol problems, it may appear also when the objective function is linear in u), becausethe Hamiltonian has the form:

H = uφ(t, x, p) + rest(not depending on u), a ≤ u(t) ≤ b

here, φ is called switching function , an thus the Hamiltonian maximization condition, weget:

u =

a if φ(t, x, p) < 0? if φ(t, x, p) = 0b if φ(t, x, p) > 0

where a and b are the bounds on the control u. If φ(t, x, p) = 0 is not sustained overan interval, then the control is bang-bang, and almost everywhere takes the extremalvalues of the control set. Otherwise, the problem is called singular . The points wherethe control value changes, i.e., when φ(t, x, p) = 0, are the switch times or switch points,but there are not usually easy to calculate.As reference, a bang-bang theorem can be found in [4].

From [45] we can conclude that the number of switching points of our problem isfinite.

25

When −0.4x1p1 − 0.2p2x2 = 0, the maximization conditions gives no information.When that happens in a point, we are going to have a switching point, but if it happensover an interval, the control would be singular.

Then:

u =1

2(1 + sg(2x1p1 + p2x2))

3.2.1 Numerical resolution

In regard to the computational resolution, this problem needs different formulation,because the final time is not fixed. A new variable must be defined: tf , which is asked tobe strictly positive (we maintain N number of discretizations fixed), and the corresponding

extra differential equationdtfdt

= 0.

x1,i+1 = xi +tf

Nx1,i+1(1− x2,i+1 − 0.4ui+1)

x2,i+1 = x2,i +tf

Nx2,i+1(−1 + x1,i+1 − 0.2ui+1)

Cost function:

min tf

And finally, the restrictions:

x1(0) = 0.5

x2(0) = 0.7

x1(N) = 1

x2(N) = 1.4

u(i) > 0

tf > 0

The optimal time obtained is 4.08948, and we can see the corresponding optimaltrajectories in Figure 5:

So this problem has a unique switching point, and once the trajectory chains, itmaintains for the rest of the time in the trajectory corresponding to the solution of (16)with u(t) ≡ 1.

26

Figure 5: Solution of (16) with (17), with N=100000 time steps

4 Turnpike property

4.1 Preliminaries

The Turnpike phenomenon appears in several variational and optimal control prob-lems, arising in engineering and economic growth. We said that a problem has a turnpikeproperty when the solutions converge to certain path during most of the time. This pathis known as the turnpike of the problem.

The first ones to observe this phenomenon were the mathematicians John vonNeu-mann and Paul Samuelson (the last one winner of the Nobel Prize for Economics in 1970)in the field of economics. The origin of the term Turnpike is in the interpretation thatSamuelson did of this phenomenon in [44] (chapter 12): Suppose we want to travel fromA city to B city by car, the best way to do it, the optimal way, is to take the highway(namely, the turnpike) as near as we can from city A, and leave it when we are close to B.So, except nearby A and B, we are expect to be on the highway: that is it, the turnpikeof the problem. In econometry such steady-states are known as VonNeumann points.

27

The turnpike properties are studied on [T1, T2] intervals, where T1 < T2, and T1, T2

real numbers in the case of continuous problems and integers in the discrete problems (itcan also be [0, T ] or [0,∞]). The solutions of these problems (the trajectories) dependon a optimality criterion, which is determined by an objective function (that can be anintegrand), by the interval [T1, T2], and by other data that usually are a set of initialconditions. The Turnpike Theory studies the structure of the solution when we fix theobjective function, while T1, T2 and the data varies. To have a turnpike property meansthat the solution is basically determined by the objective function (i. e. the optimallycriterion), and that it is essentially independent of the other conditions of the problem,except on the vicinities of the extremes of the interval [T1, T2]. In other words, if aninstant t doesn’t belong to such vicinities, the value of the solution in t will be close tothe turnpike.

After the work of [44], first complete demonstrations for VonNeumann model weregiven by McKenzie [33] [34], Morishima [36] and Radner [41]. In Mckenzie’s works, theywere also proved some turnpike theorems for Leontief model. Several turnpike results canbe found in [56] and [12] for continuous-time problems.

Basing on Hamilton-Jacobi approach, the turnpike property is proved in [53] forlinear quadratic problems under Kalman condition, and for nonlinear control-affine systemin [2]. On the other hand, in [50] they prove a turnpike theorem basing on PontryaginMaximum Principle.

4.2 Turnpike theorem for nonlinear systems

In this section, results of the paper [50] are going to be summarized. There, aturnpike theorem is proved for the next nonlinear system:

x = f(x(t), u(t)) (18)

with f : Rn × Rm −→ Rn is C2, y u(t) ∈ L∞([0, T ];Rm). Let f 0 : Rn × Rm −→ R, a C2

function, for T > 0 the following cost function is defined:

CT (u) =

∫ T

0

f 0(x(t), u(t))dt (19)

the R : Rn × Rn −→ Rk a C2 mapping, which represents de initial and final conditionsthe problem has to satisfy:

R(x(0), x(T )) = 0 (20)

As already mentioned, the optimal solution (x, p, u) of this system has to verify thenecessary conditions of Maximun Principle in addition to the transversallity conditions:

28

(−p(0)p(T )

)=

k∑i=1

γi∇Ri(x(0), x(T )))

Let consider now the static problem:

minf(x,u)=0f0(x, u) (21)

whose solution is (x, u). Therefore, there are (p, p0) ∈ Rn ×R− {(0, 0)}, with p0 ≤ 0 (wetake p0 = −1 without lost of generality), such that they verify:

∂H

∂p(x, p,−1, u) = 0

∂H

∂x(x, p,−1, u) = 0

∂H

∂u(x, p,−1, u) = 0

(22)

So the point (x, p, u) is an equilibrium point of (22), therefore, is natural to expectthat the optimal solution (x, p, u) of the problem (18) for the cost (19) keeps close to thatpoint most of the time if T > 0 large

The hessian of the Hamiltoian of the system is:

Hess(x,p,−1,u)H =

Hxx Hxp Hxu

Hpx 0 Hpu

Hux Hup Huu

Theorem 4.1 Suppose that the Huu matrix is symmetric negative definite, the matrixW = −Hxx + HxuH

−1uuHux is symmetric positive definite, and the pair of matrices A =

Hpx −HpuH−1uuHux and B = Hpu satisfies de Kalman condition:

rank(B,AB, ..., An−1B) = n

Assume also that the point (x, x) is not a singular point of R, that the norm of theHessian of R at the point (x, x) is small enough. Then, there exits constants ε > 0, C1 >0, C2 > 0, T0 > 0, such that if:

D = ||R(x, x)||+ ||(−p, p)t −k∑i=1

γi∇Ri(x, x)|| ≤ ε

then, ∀T > T0, the problem (18) for the cost (19) has at least one optimal solution (x, p, u)satisfying that ∀t ∈ [0, T ] verifies:

29

||x(t)− x||+ ||p(t)− p||+ ||u(t)− u|| ≤ C1(e−C2t + eC2(T−t)) (23)

Remark: The closeness of the optimal solution to the equilibrium point (x, p, u)depends on the constants C1 and C2. C1 depends in a linear way on D and e−C2T . Onthe other hand:

C2 = −max{R(µ)|µ ∈ Spec(A−BH−1uuB

tE−)}

where E− is the minimal symmetric negative definite matrix solution of the Riccati equa-tion:

XA+ AtX −XBH−1uuB

tX −W = 0

4.2.1 Example: Linear case

Suppose that our system is:

f(x, u) = Ax+Bu

with:

A =

(0 1−1 0

); B =

(01

)with the cost function:

min1

2

∫ T

0

((x− 1)2 + y2 + u2)dt

Is easy to see that this system satisfies the conditions of the theorem, and theturnpike of this problem is (x, y) = (1

2, 0) and u = 1

2.

As we can see in Figure 6, the curves x(.), y(.), u(.) oscillate around their steady-statevalue along the time interval [0, 60]

4.3 Aplication to Lotka-Volterra model

It is easy to see that both problems of section 3 do not satisfy the conditions of theturnpike theorem. In the case of Problem 1, the resulting W is not positive definite, andin case of problem 2, Huu = 0.

So, let apply the previous results to the system below:

30

Figure 6:

dx1

dt= x1 − x1x2 − 0.4x1u

dx2

dt= −x2 + x1x2 − 0.2x2u

(x0, y0) = (0.5; 0.7);

T = 60;

(24)

with the following cost function to minimize:

min1

2

∫ T

0

((x1 − 1)2 + x22 + u2)dt (25)

(we take now T = 60 instead of T = 30 as in the cases of section 3, just to see better theturnpike phenomenon).

The proofs of the existence of solutions, controllability and existence of minimumare analogous to Problem 1 of Section 3.

The R function representing the boundary conditions is:

31

R(x(0), x(T )) = (x1(0)− 0.5, x2(0)− 0.7)

Application of Potryagin Maximum Principle

First, we have to apply the Maximum Principle, and we get:Hamiltonian: H = p1(x1−x1x2− 0.4x1u) + p2(−x2 +x1x2− 0.2x2u)− 1

2((x1− 1)2 +

x22 + u2)

Adjoin vector:

p1 = (−1 + 0.4u)p1 + x2(p1 − p2) + (x1 − 1)

p2 = (1 + 0.2u)p2 + x1(p1 − p2) + x2

Hamiltonian maximization condition :

u =

0 if −0.4x1p1 − 0.2p2x2 < 0

−0.4x1p1 − 0.2p2x2 if 0 ≤ −0.4x1p1 − 0.2p2x2 ≤ 11 if −0.4x1p1 − 0.2p2x2 > 1

(26)

We deduce from the transversallity condition that (p1(T ), p2(T )) = (0, 0).

The Turnpike

Therefore, to find the Turnpike (x, p, u) of the system, first we have to solve thestatic problem:

min((x1 − 1)2 + x22 + u2)

subject to:

0 ≤ u(t) ≤ 1

x1(1− x2 − 0.4u) = 0

x2(−1 + x1 − 0.2u) = 0

whose solution is (x1, x2, u) = (1615, 13

15, 1

3) (obtained using IPOPT, the corresponding code

can be found in the annex). Then, we can deduce the value of (p1, p2) and we get:

(x1, x2, p1, p2, u) = (16

15,13

15,−0.8125, 0.07692308,

1

3)

Such solution is the point called turnpike of the problem.

32

Let verify that the hypothesis of the Theorem (1.1) are satisfied. The Hessian of His:

Hess(H) =

−1 −p1 + p2 1− 0.4u− x2 x2 −0.4p1

−p1 + p2 −1 −x1 −1− 0.2u+ x1 −0.2p2

1− 0.4u− x2 −x1 0 0 −0.4x1

x2 −1− 0.2u+ x2 0 0 −0.2x2

−0.4p1 −0.2p2 −0.4x1 −0.2x2 −1

• Huu = −1

• W =

(0.8943750 0.89442310.8944231 0.9997633

)symmetric positive definite matrix

• Kalman condition

A =

(−0.1386667 −1.06010260.810333333 0.002666667

);B =

(−0.426667−0.173333

)rank(B|AB) = 2

So the hypothesis of the theorem (4.1) are satisfied, thus:

||(x1(t), x2(t))−(16

15,13

15))||+||(p1(t), p2(t))−(−0.8125, 0.07692308)||+||u(t)−1

3|| ≤ C1(e−C2t+e−C2(T−t))

(27)However, how large the constants C1 and C2 are?

C2 = 0.3143442

D = ||(x1 − 0.5, x2 − 0.7)||+ ||(p1, p2)|| = 1.4068

As we have said, C1 depends in a linear way on D = 1.4068 and e−C2T ' 0, soC1 = O(1).

When t is small, the term of the right of (27) ' C1. So during a first interval [0, τ ]the distance between the optimal solution and the turnpike depends on C1, which dependson the distance between the initial point and the turnpike. As long as t becomes larger,but no as much as T (that is when T − t ' 0), the optimal solution gets closer to theturnpike of the problem, until t ' T , when the right term of (27) becomes again ' C1.

In Figure 7 the turnpike phenomenon of this problem is showed. The black line ineach graphic represents the value of the turnpike, whereas the blue one is the optimalsolution obtained with de IPOPT for this problem.

As we have said, the approximation of the solution to the turnpike depends only onthe cost function and the system itself. We have solved this system for different initial

33

Figure 7: Blue: Solutions of (24) with the cost (25). Black: The turnpike of the problem

Figure 8: Different solutions of (24) with the cost (25) deppending on the initial point:(x1(0), x2(0)) = (0.5, 0.7) → blue, (x1(0), x2(0)) = (1.2, 0.6) → green, (x1(0), x2(0)) =(1, 1)→ purple

points (see Figure 8), to see that independently of them, when we move away of t = 0,

34

the solution converges to the turnpike.Remark: The AMPL code to solve this problems is included in an annex, Section

6.6.

4.4 A variant of an indirect method using the turnpike property

As it has been mentioned, there are mainly two kinds of numerical methods inoptimal control theory: indirect and direct methods; until now, a direct method has beenused to solve numerically each problem.

4.4.1 A brief comparison between indirect and direct methods

The direct methods have the following advantages:

• Their implementation is simpler because they do not require a prior theoretical studyas indirect methods; in particular, it does not have to study the adjoin vector.

• They are robust

• They are not very sensitive to the initial condition

• It is easy to take into account eventual constraints on the state

• They allow to calculate feedback-controls.

and the following disadvantages:

• They are less accurate than the indirect methods.

• Because of the direct discretization, the method may converge to a local maximum

• Direct methods usually use a lot of memory, so they can become ineffective if thespace dimension is too large

On the other hand, the advantages of the indirect methods are:

• They are extremely precise

• The multiple shooting method is parallelizable, and its implementation can be doneusing a network of computers connected in parallel.

and their disadvantages:

• They compute the optimal controls in the form of open-loop

35

• They are based on the use of the Pontryagin Maximum Principle, which is a neces-sary condition, so it is necessary to verify the optimality of the obtained trajectory.

• Rigidity of the method: the switching structure should be known in advance, andit is not easy to introduce restrictions in the state.

• Besides that, it is important to get a good intilization point both for the state vectorand for the adjoin vector.

In general, the direct method are more used, since there are more practical. Abrief discussion between both type of methods can be found in [49] section 2.4.4 and [48]section 9.3. For a deeper explanation of the comparison this two type methods and alsoexplanations and comparisons of concrete methods, see [6]

4.4.2 The indirect shooting method

This is the most classic indirect method (see [11] chapter 7, [48] chapter 9, or [49]for a detailed explanation of the method). It is based on applying the The PontryaginMaximum Principle, to get a differential system, where the control has been expressed asa function of the state vector and the adjoin vector, so:

z(t) = F (z(t)) (28)

where z = (x, p). From the initial and final conditions and from the transversality condi-tions, we get the boundary conditions written as:

G(z(0), z(T )) = 0 (29)

Then, the method consist on suiting z(0) in order to satisfy (29) (usually using aNewton method).

For example, in the problem enunciated in this section, we have the initial pointfixed, and the endpoint is free, so (x1(0), x2(0)) is already known, and in z(0) the un-knowns are the last 2 coordinates, which correspond to the initial point of the adjoinvector (p1(0), p2(0)). So the shooting method is use to tuned these two unknowns so that(p1(T ), p2(T )) = (0, 0) (concluded from the transversality condition). So:

G((p1(0), p2(0))) = (p1(T ), p2(T ))

Starting on an initial point (p1(0), p2(0))0, we apply the Newton method in order toget ¯(p1(0), p2(0)) such that G( ¯(p1(0), p2(0))) = (0, 0), iterating in the following way:

(p1(0), p2(0))i = (p1(0), p2(0))i−1 − (G′(p1(0), p2(0)i))−1G(p1(0), p2(0)i)

36

where G′ is the Jacobian of the function, which is usually approximated by:

∆G(si) =G(si + ∆si)

∆si

where ∆si is chosen sufficiently small.Remark: Note that every time we want to calculate G(s) we have to solve the

initial-value problem (28).

4.4.3 Variant of the shooting method

As it has been mentioned, the handicap of the indirect methods is that there arehard to initialize, and they sometimes diverge. In [50], a variant of the previous shootingmethod is proposed, using the turnpike property to avoid the initialization problem.

The turnpike property ensures that in some interval [ε, T − ε], the solution is closedto the turnpike. However, the initial point z(0) does not have to be close to the turnpike,so the property cannot be used directly as initial point for the method. The proposedidea is to use a point of [ε, T − ε], like z(T/2) as initial point, in the following way:

1. The unknown is z(T/2) = (x(T/2), p(T/2)), and we approximate it using the turn-pike (x, p)

2. (a) We integrate backwards the system (28) in the interval [0, T/2] to get z(0)

(b) We integrate the system (28) in the interval [T/2, T ] to get z(T )

3. The value of z(T/2) is tuned using Newton method to satisfy (29).

4.4.4 The turnpike modification applied to the Lotka-Volterra model

In this work, it has been implemented the shooting method modified as it has beenexplained before, to solve the Lotka-Volterra problem of section 4. The algorithm for ageneral case is the following one:

G((x(T/2), y(T/2), p1(T/2), p2(T/2)) = (x(0)− 0.5, x2(0)− 0.7, p1(T ), p2(T ))

∆G((x1(T/2), x2(T/2), p1(T/2), p2(T/2)) =G((x1(T/2)+ε,x2(T/2),p1(T/2),p2(T/2))−G((x1(T/2),x2(T/2),p1(T/2),p2(T/2))

εG((x1(T/2),x2(T/2)+ε,p1(T/2),p2(T/2))−G((x1(T/2),x2(T/2),p1(T/2),p2(T/2))

εG((x1(T/2),x2(T/2),p1(T/2)+ε,p2(T/2))−G((x1(T/2),x2(T/2),p1(T/2),p2(T/2))

εG((x1(T/2),x2(T/2),p1(T/2),p2(T/2)+ε)−G((x1(T/2),x2(T/2),p1(T/2),p2(T/2))

ε

T

37

• Input: G(s) (boundary function),F (s) (differential system), ε, maxiter (maximalnumber of iterations), T , tol1 (tolerance for the method used to solve the differentialsystem), tol2 (tolerance for Newton method), (x1, x2, p1, p2) (the turnpike previouslycalculated)

• Initialize z(T/2)(0) = (x1(T/2), x2(T/2), p1(T/2), p2(T/2)) = (x1, x2, p1, p2), i = 1

• Calculate (x1(0), x2(0))(0) integrating backwards the system z = F (z) in [0, T/2]using the initial point z(T/2)(0)

• Calculate (p1(T ), p2(T )) integrating the system z = F (z) in [T/2, T ] using the initialpoint z(T/2)(0)

• While the i < maxiter and error < tol2 do:

– Calculate G(z(T/2)(i)) and ∆G(z(T/2)(i))

– Calculate z(T/2)(i+1):

z(T/2)(i+1) = z(T/2)(i) − (∆G(z(T/2)(i)))−1.G(z(T/2)(i))

– error = |G(z(T/2)(i))| and i = i+ 1

• Calculate the optimal solution using the point z(T/2)(i) integrating backwards andforwards.

• Calculate the control u(t) using the Hamiltonian maximization condition

• Output: (x(t), u(t)), the error and the number of iteration

This algorithm has been implemented using the free software R [40], and it can beseen in the annex of this work. The method used to solve the differential system is theexplicit Runge-Kutta method of order four ([18], [20]) with tolerance=10−9. We have usedthe following values:

tol2 = 10−9, ε = 10−10,maxiter = 100The error of this solution is 2.378833.10−11, and it has been necessary to do six

iterations.

5 Conclusions

In this work we show that the variation of the classic shooting method proposed in[50] is feasible. Such application of the turnpike property is very useful, as it representsa good solution to the divergence and initialization problems of the classic method.

38

Figure 9: Solutions of (24) with the cost (25), using the variant of the indirect method

Besides that, the work done here for the Lotka-Volterra model can be easily gener-alized to another problems.

However, the application of this variation is not always possible. This is the caseof the two problems studied in section 3, they don’t verify the hypothesis of the turnpiketheorem (Theorem 1.1), i. e. they haven’t got a turnpike. It must be also taken intoaccount that, even if the use of the turnpike solves the initialization problem and itimproves the convergence, it also has the other disadvantages characteristic of the indirectmethods, as the rigidity of the method when we want to add restrictions to the state.

5.1 Comparison with the results obtained for Lotka-Volterrasystem with IPOPT

Let compare the two different solutions obtained in section 4, for the same Lotka-Volterra problem, both for x1(t) and for x2(t) (Figure 10 and Figure 11 respectively). Ingreen, we can see the one obtained with the variation of the indirect shooting method(implement in R). In blue, we have the solution obtained using the IPOPT. On the

39

right, we can see the graphics of the difference between both solutions, for x1(t) andx2(t) respectively. To do this comparison, we have execute again the turnpike.mod,

turnpike.dat and turnpike.run with N=60000 (codes in section 6.6), in order to havethe same discretization in both methods. It is also important to mention that we haveintroduced the same tolerance error in both algorithms (10−9).

Figure 10: Difference in x1(t) (prey)

Figure 11: Difference in x2(t) (predator)

As we can see in Figure 10 and in Figure 11, the solution obtained with the newalgorithm is nearly the same than the one obtained with the IPOPT. Both IPOPT andR give us the CPU time for the execution of each algorithm (Table 2)

Even if the computational time of the IPOPT is less than the half of the new one,we must take into account that the algorithms were run in different computers. The first

40

IPOPT CPU time (in seconds) R CPU time (in seconds)360.28 771

Table 2:

one (using AMPL and IPOPT) was run in a server with 8 processors of 4 cores each,(a total of 32 cores)and 256 GB of RAM memory. On the other hand, the one we havedeveloped in this work was run in a personal laptop of 2 cores and 4 GB of RAM memory.So the comparison of the CPU time does not necessary imply that the second method isworse.

In order to have a better perception of the accuracy of the algorithm, we ran bothagain with tolerance= 10−11, take those two solutions as the “real” ones in each case, andcompare the previous solutions with the ones obtained now (i.e. the solutions obtainedwith IPOPT between them (Figure 12), and the ones obtained with R between them(Figure 13)). In Table 3, there are the sum of the absolute errors done for each methodboth for x1 and for x2.

Figure 12: Difference between the two solutions obtained with IPOPT

IPOPT: error Variant of the indirect shooting method: errorError in x1 Error in x2 Error in x1 Error in x2

2.774519 2.431568 6.103355e− 08 5.784397e− 08

Table 3:

It seems like indirect method developed using the turnpike property approximatesbetter the solution than the direct method used by the IPOPT. However, it is probably

41

Figure 13: Difference between the two solutions obtained using the variation of the indirectshooting method

because we have used a higher order method in the indirect method (a fourth ordermethod, while the one used in the direct method was of order one).

It should be noted that the algorithm using the turnpike property has been developedonly for solving this concrete problem of Lotka-Volterra, but is easy to be generalizedin order to solve any problem satisfying the hypothesis for having a turnpike. Thosehypothesis are easy to check computationally. It may be interesting to develop suchalgorithm, as effectively as possible from the point of view of computational cost , use itto solve different problems, and check if it is really competitive.

5.2 A possible improvement of the indirect method using theturnpike

The variation proposed in [50] for the indirect shooting method, it also can be useto improve the indirect multiple shooting method.

The idea of this method is similar to the shooting method, but dividing the timeinterval [0, tf ] in M subintervals [ti, ti+1], being now the unknowns z(ti), and adding someboundary conditions in ti in order to ensure the continuity of the trajectory. After applyingthe Pontryagin Maximum Principle we get the following system:

z(t) = F (t, z(t)) =

F0(t, z(t)) if 0 ≤ t < t1F1(t, z(t)) if t1 ≤ t < t1

...FM(t, z(t)) if tM−1 ≤ t < tM

where z = (x, p) ∈ R2n and t1, ..., tM are the switching time where we are going to havemore constraints z(t−i−1) = z(t+i ). So we are going to have the following boundary problem:

42

z(t) = F (t, z(t)) =

F0(t, z(t)) if 0 ≤ t < t1F1(t, z(t)) if t1 ≤ t < t1

...FM(t, z(t)) if tM−1 ≤ t < tM

∀j ∈ {2, ...,M − 1}, rj(tj, z(t−j ), z(t+j ))) = 0rM(tf , z(0), z(tf )) = 0A good reference for this method is [11].The idea would be to apply the turnpike variation proposed in [50] to this method,

which is most stable than single shooting method. Since the turnpike property assuresthat in the middle part of the time interval [0, T ] the solutions gets close to certaintrajectory, the partition of it for applying multiple shooting method can be chosen to bemore thin in the extremes.

6 Annex

In this section, there are the codes used in this work.

6.1 Problem 1 of section 3

LotVol.mod file:

#-------------------------------------------------------------------------

#

#Model for Lotka Volterra problem

#

#-------------------------------------------------------------------------

#Parameters

param N; #Number of time discretizations

param T; #final time instant

param x0;

param y0;

#Parameters of the system

param a;

param b;

param c;

param d;

param cx;

param cy;

43

##Variables

var x {0..N} >=0.001;#prey

var y {0..N} >=0.001;#predator

var u {0..N} >=0, <=1; #control

var coste {0..N} >=0; #cost function

#Objective function

minimize cost: 2*((x[N]-1)^2+(y[N]-1.4)^2)+(1/2)*(sum{i in 0..N} (T/N)*(u[i]^2));

#Restrictions

#System ( implicit Euler method )

subject to presa {i in 0..N-1}: x[i+1]=x[i]+(T/N)*x[i+1]*

(a-b*y[i+1]-cx*u[i+1]);

subject to predador {i in 0..N-1}: y[i+1]=y[i]+(T/N)*y[i+1]*

(-c+d*x[i+1]-cy*u[i+1]);

#Initial conditions

subject to xinicial: x[0]=x0;

subject to yinicial: y[0]=y0;

LotVol.dat file:

#----------------------------------------------------

#Data

#----------------------------------------------------

param N:=100000;

param T:=30;

param x0:=0.5;

param y0:=0.7;

param a:=1;

param b:=1;

param c:=1;

param d:=1;

param cx:=0.4;

param cy:=0.2

Executable file:

44

#------------------------------------

#solve for LotkaVolterra problem

#------------------------------------

reset;

model LotVol.mod;

data LotVol.dat;

option solver '/opt/ipopt/build/bin/ipopt';

solve;

display x[N], y[N], cost;

print{i in 0..N}: i*(T/N), x[i], y[i], u[i], coste[i] >> lotvol.txt;

6.2 Problem 2 of section 3

LotVol.mod file:

#--------------------------------------------------------------------

#

#Lotka Volterra, time optimal problem

#

#--------------------------------------------------------------------

#Parameters

param N;

param T; #bound for final time tf

param x0;

param y0;

param xf;

param yf;

#Parameters of the system

param a;

param b;

param c;

param d;

param cx;

45

param cy;

##Variables

var tf >=0.01, <=T; #final time, we want to minimize

var x {0..N} >=0.001;#prey

var y {0..N} >=0.001;#predator

var u {0..N} >=0, <=1; #control

#Objective function

minimize cost: tf;

#Restrictions

#System (implicit Euler method)

subject to presa {i in 0..N-1}: x[i+1]=x[i]+(tf/N)*x[i+1]*

(a-b*y[i+1]-cx*u[i+1]);

subject to predador {i in 0..N-1}: y[i+1]=y[i]+(tf/N)*y[i+1]*

(-c+d*x[i+1]-cy*u[i+1]);

#Initial conditions

subject to xinicial: x[0]=x0;

subject to yinicial: y[0]=y0;

#Final conditions

subject to xfinal: x[N]=xf;

subject to yfinal: y[N]=yf;

LotVol.dat file:

#----------------------------------------------------

#Data

#----------------------------------------------------

param N:=100000;

param T:=120;

param x0:=0.5;

param y0:=0.7;

param xf:=1;

param yf:=1.4;

46

param a:=1;

param b:=1;

param c:=1;

param d:=1;

param cx:=0.4;

param cy:=0.2

Executable file:

#------------------------------------

#solve for tima optimal problem

#------------------------------------

reset;

model LotVol.mod;

data LotVol.dat;

option solver '/opt/ipopt/build/bin/ipopt';

option ipopt_options "linear_solver=mumps";

#option ipopt_options "mumps_mem_percent=10000";

solve;

display x[N], y[N], tf;

print{i in 0..N}: i*(1/N), x[i], y[i], u[i] >> lotvol.txt;

6.3 Classic shooting method

Here, the classic shooting method implemented in R, to solve the Lotka-volterra problemof section 3.

library("deSolve")

sistema<-function(t,v,parametros){

x<-v[1]

y<-v[2]

p1<-v[3]

p2<-v[4]

47

res<-c()

aux<-(-0.4*p1*x-0.2*y*p2)

if(aux>0&&aux<1){

u<-aux

}

else{

if(aux<0){

u<-0

}

else{

u<-1

}

}

dx<--(x-x*y-0.4*x*u)

dy<--((-y)+x*y-0.2*y*u)

dp1<--((-1+0.4*u)*p1+y*(p1-p2))

dp2<--((1+0.2*u)*p2+x*(p1-p2))

list(c(dx,dy,dp1,dp2))

}

tol<-10^(-10)

parametros<-c()

u0<-c(0.5,0.7,0.6,0.6)

tiempos<-seq(0,30,0.001)

out<-ode(y=u0,times=tiempos,parms=parametros,func=sistema,method="ode45",

atol=tol,rtol=tol)

plot(out[,1],out[,2],col="red")

#shooting method clasico

i<-1

iter<-100

error<-10

maxerror<-10^(-4)

48

while(i<iter&&error>maxerror){

fp1<-out[30001,4]

fp2<-out[30001,5]

eps<-10^(-5)

aux1<-c(u0[1],u0[2],u0[3]+eps,u0[4])

aux2<-c(u0[1],u0[2],u0[3],u0[4]+eps)

faux1<-ode(y=aux1,times=tiempos,parms=parametros,func=sistema,method="ode45",

atol=tol,rtol=tol)

faux2<-ode(y=aux2,times=tiempos,parms=parametros,func=sistema,method="ode45",

atol=tol,rtol=tol)

f1<-(faux1[30001,4]-fp1)*(1/eps)

f2<-(faux1[30001,5]-fp2)*(1/eps)

f3<-(faux2[30001,4]-fp1)*(1/eps)

f4<-(faux2[30001,5]-fp2)*(1/eps)

deltaf<-matrix(c(f1,f2,f3,f4),nrow=2,byrow=TRUE)

deltafinv<-solve(deltaf)

var<-(c(fp1,fp2))%*%deltafinv

u0<-c(u0[1],u0[2],u0[3]+var[1],u0[4]+var[2])

out<-ode(y=u0,times=tiempos,parms=parametros,func=sistema,method="ode45",

atol=tol,rtol=tol)

plot(out[,1],out[,2])

print("Iteracion")

print(i)

i<-i+1

error2<-c(out[30001,4],out[30001,5])

print(error2)

error<-max(abs(error2))

print(error)

}

plot(out[,1],out[,2],type="l",col="green",xlab="t",ylab="x(t), prey")

plot(out[,1],out[,3],type="l",col="green",xlab="t",ylab="y(t), predator")

49

6.4 Turnpike theory: linear example

TurLineal.mod file

#----------------------------------------------------------------------------------------

#

#Ejemplo turnpike lineal-Turnpike linear example

#

#----------------------------------------------------------------------------------------

#Parametros- Parameters

param N; #Numero de discretizaciones en el tiempo-Number of tima discretizations

param T; #instante final- final time

param x0;

param y0;

##Variables

var x {0..N} ;#presa-prey

var y {0..N} ;#predador-predator

var u {0..N} >=0, <=1; #control-control

#Funcion objetivo-cost function

minimize cost: sum{i in 0..N} (T/N)*(1/2)*((x[i]-1)^2+(y[i])^2+u[i]^2);

#Restricciones-constraints

#System

subject to presa {i in 0..N-1}: x[i+1]=x[i]+(T/N)*(y[i]);

subject to predador {i in 0..N-1}: y[i+1]=y[i]+(T/N)*(-x[i]+u[i]);

#Condiciones iniciales- INitial point

subject to xinicial: x[0]=x0;

subject to yinicial: y[0]=y0;

TurLineal.dat file

#----------------------------------------------------

#Datos para ejemplo lineal turnpike-

#Data for inear turnpike problem

#----------------------------------------------------

50

param N:=100000;

param T:=60;

param x0:=0;

param y0:=0;

TurLineal.run file

#------------------------------------

#solve para ejemplo turnpike lineal

#solve for linear turnpike problem

#------------------------------------

reset;

model TurLineal.mod;

data TurLineal.dat;

option presolve_eps 1e-10;

option solver '/opt/ipopt/build/bin/ipopt';

solve;

display x[N], y[N], cost;

print{i in 0..N}: i*(T/N), x[i], y[i], u[i] >> turlineal.txt;

6.5 IPOPT code to solve the Lotka-Volterra static problem

We just need a .mod file for this case:

#########################################################

#

# Lotka volterra static problem

#

#########################################################

reset;

##Variables

var x >=0.1;#prey

var y >=0.1;#predator

var u >=0, <=1; #control

51

minimize cost: (x-1)^2+y^2+u^2;

subject to c1: (1-y-0.4*u)=0;

subject to c2: (-1+x-0.2*u)=0;

option solver '/opt/ipopt/build/bin/ipopt';

solve;

display x, y, u, cost;

6.6 IPOPT Codes to solve the turnpike problem of Lotka-Volterra model

The turnpike.mod file:

#----------------------------------------------------------------------------------------

#

#Simulation: Turnpike property of Lotka-Volterra model.

#

#----------------------------------------------------------------------------------------

#Parameters

param N; # Number of time discretization

param T; #instante final

param x0;

param y0;

#Parametros del sistema

param a;

param b;

param c;

param d;

param cx;

param cy;

##Variables

var x {0..N} >=0.1;#presa

var y {0..N} >=0.1;#predador

var u {0..N} >=0, <=1; #control

52

#Funcion objetivo

minimize cost: sum{i in 0..N} (T/N)*(1/2)*((x[i]-1)^2+(y[i])^2+u[i]^2);

#Restricciones

#El sistema (metodo de Euler implicito)

subject to presa {i in 0..N-1}: x[i+1]=x[i]+(T/N)*x[i+1]*

(a-b*y[i+1]-cx*u[i]);

subject to predador {i in 0..N-1}: y[i+1]=y[i]+(T/N)*y[i+1]*

(-c+d*x[i+1]-cy*u[i+1]);

#Condiciones iniciales

subject to xinicial: x[0]=x0;

subject to yinicial: y[0]=y0;

The tunrpike.dat file:

#----------------------------------------------------

#Data for the turnpike problem

#----------------------------------------------------

param N:=100000;

param T:=60;

param x0:=0.5;

param y0:=0.7;

param a:=1;

param b:=1;

param c:=1;

param d:=1;

param cx:=0.4;

param cy:=0.2;

The tunrpike.run file:

#------------------------------------

#RUN file for turnpike problem

#------------------------------------

reset;

model turnpike.mod;

data turnpike.dat;

option solver '/opt/ipopt/build/bin/ipopt';

53

solve;

display x[N], y[N], cost;

print{i in 0..N}: i*(T/N), x[i], y[i], u[i] >> turnpike.txt;

6.7 Implementation of the variation of indirect shooting method

The implementation is done in R [40].

library("deSolve")

#condiciones extremas: x(0)=0.5, y(0)=0.7, p1(T)=0, p2(T)=0

iter<-100 #numero maximo de iteracciones

sistema<-function(t,v,parametros){

x<-v[1]

y<-v[2]

p1<-v[3]

p2<-v[4]

res<-c()

aux<-(-0.4*p1*x-0.2*y*p2)

if(aux>0&&aux<1){

u<-aux

}

else{

if(aux<0){

u<-0

}

else{

u<-1

}

}

54

dx<-x-x*y-0.4*x*u

dy<-(-y)+x*y-0.2*y*u

dp1<-(-1+0.4*u)*p1+y*(p1-p2)+(x-1)

dp2<-(1+0.2*u)*p2+x*(p1-p2)+y

list(c(dx,dy,dp1,dp2))

}#sistema Lotka-Volterra

sistema2<-function(t,v,parametros){

x<-v[1]

y<-v[2]

p1<-v[3]

p2<-v[4]

res<-c()

aux<-(-0.4*p1*x-0.2*y*p2)

if(aux>0&&aux<1){

u<-aux

}

else{

if(aux<0){

u<-0

}

else{

u<-1

}

}

dx<--(x-x*y-0.4*x*u)

dy<--((-y)+x*y-0.2*y*u)

dp1<--((-1+0.4*u)*p1+y*(p1-p2)+(x-1))

dp2<--((1+0.2*u)*p2+x*(p1-p2)+y)

list(c(dx,dy,dp1,dp2))

}#sistema Lotka-Volterra para integrar hacia atras

#Primera solucion aproximada utilizando el turnpike

turnpike<-c(16/15,13/15,-0.8125,0.07692308)

55

tol<-10^(-9)

parametros<-c()

#Integracion del instante 30 al 60, hacia adelante, para obtener p1(T), p2(T)

tiempos<-seq(30,60,0.001)

out<-ode(y=turnpike,times=tiempos,parms=parametros,func=sistema,

method="rk4",atol=tol,rtol=tol)

plot(out[,1],out[,2])

p1T<-out[30001,4]

p2T<-out[30001,5]

#Integracion hacia atras del instante 30 a 0, para obtner x(0),y(0)

tiempos2<-seq(-30,0,0.001)

out2<-ode(y=turnpike,times=tiempos2,parms=parametros,func=sistema2,

method="rk4",atol=tol,rtol=tol)

plot(out2[,1],out2[,2],col="red")

x0<-out2[30001,2]

y0<-out2[30001,3]

u0<-turnpike

print(u0)

eps<-10^(-10)

maxerror<-10^(-9)

i<-1

error<-10

while(i<iter&&error>maxerror){

#metodo de newton

#calculamos la aproximacion de la matriz jacobiana

#perturbacion en x(t/2)

aux1<-c(u0[1]+eps,u0[2],u0[3],u0[4])

Gaux1atras<-ode(y=aux1,times=tiempos2,parms=parametros,func=sistema2,

method="rk4",atol=tol,rtol=tol)

Gaux1adelante<-ode(y=aux1,times=tiempos,parms=parametros,func=sistema

,method="rk4",atol=tol,rtol=tol)

g11<-(1/eps)*((Gaux1atras[30001,2]-0.5)-(x0-0.5))

g12<-(1/eps)*((Gaux1atras[30001,3]-0.7)-(y0-0.7))

g13<-(1/eps)*(Gaux1adelante[30001,4]-p1T)

g14<-(1/eps)*(Gaux1adelante[30001,5]-p2T)

#perturbacion en y(t/2)

56

aux2<-c(u0[1],u0[2]+eps,u0[3],u0[4])

Gaux2atras<-ode(y=aux2,times=tiempos2,parms=parametros,func=sistema2,

method="rk4",atol=tol,rtol=tol)

Gaux2adelante<-ode(y=aux2,times=tiempos,parms=parametros,func=sistema,

method="rk4",atol=tol,rtol=tol)

g21<-(1/eps)*((Gaux2atras[30001,2]-0.5)-(x0-0.5))

g22<-(1/eps)*((Gaux2atras[30001,3]-0.7)-(y0-0.7))

g23<-(1/eps)*(Gaux2adelante[30001,4]-p1T)

g24<-(1/eps)*(Gaux2adelante[30001,5]-p2T)

#perturbacion en p1(t/2)

aux3<-c(u0[1],u0[2],u0[3]+eps,u0[4])

Gaux3atras<-ode(y=aux3,times=tiempos2,parms=parametros,func=sistema2,

method="rk4",atol=tol,rtol=tol)

Gaux3adelante<-ode(y=aux3,times=tiempos,parms=parametros,func=sistema,

method="rk4",atol=tol,rtol=tol)

g31<-(1/eps)*((Gaux3atras[30001,2]-0.5)-(x0-0.5))

g32<-(1/eps)*((Gaux3atras[30001,3]-0.7)-(y0-0.7))

g33<-(1/eps)*(Gaux3adelante[30001,4]-p1T)

g34<-(1/eps)*(Gaux3adelante[30001,5]-p2T)

#perturbacion en p2(t/2)

aux4<-c(u0[1],u0[2],u0[3],u0[4]+eps)

Gaux4atras<-ode(y=aux4,times=tiempos2,parms=parametros,func=sistema2,

method="rk4",atol=tol,rtol=tol)

Gaux4adelante<-ode(y=aux4,times=tiempos,parms=parametros,func=sistema,

method="rk4",atol=tol,rtol=tol)

g41<-(1/eps)*((Gaux4atras[30001,2])-(x0))

g42<-(1/eps)*((Gaux4atras[30001,3])-(y0))

g43<-(1/eps)*(Gaux4adelante[30001,4]-p1T)

g44<-(1/eps)*(Gaux4adelante[30001,5]-p2T)

#Matriz deltaG y su inversa

deltaG<-matrix(c(g11,g12,g13,g14,g21,g22,g23,g24,g31,g32,g33,g34,g41,

g42,g43,g44),nrow=4, byrow=TRUE)

invdeltaG<-solve(deltaG)

variacion<-(c(x0-0.5,y0-0.7,p1T,p2T))%*%invdeltaG

u0<-u0-variacion

57

print(u0)

#calculamos la nueva solucion

out<-ode(y=u0,times=tiempos,parms=parametros,func=sistema,method="rk4",

atol=tol,rtol=tol)

p1T<-out[30001,4]

p2T<-out[30001,5]

out2<-ode(y=u0,times=tiempos2,parms=parametros,func=sistema2,method="rk4",

atol=tol,rtol=tol)

x0<-out2[30001,2]

y0<-out2[30001,3]

print(p1T)

print(p2T)

print(x0)

print(y0)

print("Iteracion")

print(i)

i<-i+1

error2<-c(x0-0.5,y0-0.7,p1T,p2T)

print(error2)

error<-max(abs(error2))

print(error)

}

#solucion

solucion<-matrix(data=NA,nrow=60001,ncol=5)

tiempo<-seq(0,60,0.001)

solucion[,1]<-tiempo

for(j in 2:30001){

j2<-30001-(j-1)

#solucion[j,1]<-(-out2[j2,1])

solucion[j,2]<-out2[j2,2]

58

solucion[j,3]<-out2[j2,3]

solucion[j,4]<-out2[j2,4]

solucion[j,5]<-out2[j2,5]

}

for(k in 1:30001){

k2<-k+30000

solucion[k2,2]<-out[k,2]

solucion[k2,3]<-out[k,3]

solucion[k2,4]<-out[k,4]

solucion[k2,5]<-out[k,5]

}

plot(solucion[,1],solucion[,2], type="l", col="green", ylab="x(t),

prey",xlab="time t")

plot(solucion[,1],solucion[,3], type="l", col="green", ylab="y(t),

predator",xlab="time t")

plot(solucion[,2],solucion[,3],type="l",col="green",ylab="y(t),predator",

xlab="x(t),prey", main=paste("Evolution of both species"))

#comparacion con la solucion obtenida en IPOPT

#error en x(t)

setwd("C:/Users/Aitziber/Desktop/Tur2")

datos<-read.table("turnpike.txt")

plot(datos[,1], datos[,2], type="l", col="blue",xlab="Time", ylab="x(t)

(prey) ", main=paste("Comparation between the two methods: x(t)"))

lines(solucion[,1],solucion[,2], type="l", col="green")

errorx<-c()

for(i in 1:60001){

errorx[i]<-abs(datos[i,2]-solucion[i,2])

}

tiempo<-seq(0,60,0.001)

plot(tiempo,errorx, main=paste("Difference in x(t)"),ylab="difference",

xlab="time", col="red",type="l")

59

#error en y(t)

plot(datos[,1], datos[,3], type="l", col="blue",xlab="Time", ylab="y(t)

(predator) ", main=paste("Comparation between the two methods: y(t)"))

lines(solucion[,1],solucion[,3], type="l", col="green")

errory<-c()

for(i in 1:60001){

errory[i]<-abs(datos[i,3]-solucion[i,3])

}

plot(tiempo,errory, main=paste("Difference in y(t)"),ylab="difference",

xlab="time", col="red",type="l")

#el control

u<-c()

for(i in 1:60001){

u[i]<-(-0.4*solucion[i,4]*solucion[i,2]-0.2*solucion[i,3]*solucion[i,5])

}

plot( tiempo,u, col="green", type="l", xlab="time",ylab="control u(t)",

main=paste("Comparation of controls"),ylim=c(-0.4,0.9))

lines(datos[,1], datos[,4],ylab="control u(t)",col="blue",type="l")

60

References

[1] Allen, L. J., Brauer, F., Van den Driessche, P., and Wu, J. Mathematicalepidemiology, vol. 1945. Springer, 2008.

[2] Anderson, B., and Kokotovic, P. V. Optimal control problems over large timeintervals. Automatica 23, 3 (1987), 355–363.

[3] Apreutesei, N., and Dimitriu, G. Optimal control for lotka-volterra systemswith a hunter population. In Large-Scale Scientific Computing. Springer, 2008,pp. 277–284.

[4] Athans, M., and Falb, P. Optimal control. 1966.

[5] Bertsekas, D. P., Bertsekas, D. P., Bertsekas, D. P., and Bertsekas,D. P. Dynamic programming and optimal control, vol. 1. Athena Scientific Belmont,MA, 1995.

[6] Betts, J. T. Practical methods for optimal control and estimation using nonlinearprogramming, vol. 19. Siam, 2010.

[7] Brauer, F., and Castillo-Chavez, C. Mathematical models in population biol-ogy and epidemiology. Springer, 2011.

[8] Braun, M., and Golubitsky, M. Differential equations and their applications.Springer, 1983.

[9] Brezis, H., Brezis, H., and Brezis, H. Functional analysis, Sobolev spaces andpartial differential equations. Springer, 2011.

[10] Broer, H., and Takens, F. Dynamical systems and chaos, vol. 172. Springer,2010.

[11] Bulirsch, R., and Stoer, J. Introduction to numerical analysis. Springer Hei-delberg, 2002.

[12] Carlson, D. A., Haurie, A., and Leizarowitz, A. Infinite horizon optimalcontrol: deterministic and stochastic systems. Springer-verlag, 1991.

[13] Crespo, L., and Sun, J. Optimal control of populations of competing species.Nonlinear Dynamics 27, 2 (2002), 197–210.

[14] Cruz, J. M. L., and Gonzalez, G. B. Modelo depredador-presa. Revista deCiencias Basicas UJAT 7, 2 (2008), 25–34.

61

[15] Fernandez-Cara, E., and Zuazua, E. Control theory: History, mathemati-cal achievements and perspectives. Notes, Universidad de Sevilla and UniversidadAutonoma Madrid, Spain (2003).

[16] Fletcher, R. Practical methods of optimization. John Wiley & Sons, 2013.

[17] Fourer, R., Gay, D., and Kernighan, B. Ampl: a modeling language formathematical programming. 2002. Duxbury Press .

[18] Gautschi, W. Numerical analysis. Springer, 2011.

[19] Gill, P. E., Murray, W., and Wright, M. H. Practical optimization.

[20] Griffiths, D. F., and Higham, D. J. Numerical methods for ordinary differentialequations. Springer, 2010.

[21] Grognard, F., and Gouze, J.-L. Positive control of lotka-volterra systems. InWorld Congress (2005), vol. 16, pp. 723–723.

[22] Hager, W. W. Runge-kutta methods in optimal control and the transformedadjoint system. Numerische Mathematik 87, 2 (2000), 247–282.

[23] Hale, J. K. Ordinary differential equations. Pure and Applied Mathematics, JohnWiley & Sons, New York (1969).

[24] Hinrichsen, D., and Pritchard, A. J. Mathematical systems theory I: mod-elling, state space analysis, stability and robustness, vol. 1. Springer, 2005.

[25] Israel, G., and Gasca, A. M. The biology of numbers: the correspondence ofVito Volterra on mathematical biology, vol. 26. Birkhauser, 2002.

[26] Keizer, J. An observation on the effect of fluctuations far from equilibrium: Thelotka-volterra model. Journal of Statistical Physics 15, 6 (1976), 477–483.

[27] Kirk, D. E. Optimal control theory: an introduction. Courier Dover Publications,2012.

[28] Lee, E. B., and Markus, L. Foundations of optimal control theory. Tech. rep.,DTIC Document, 1967.

[29] Lenhart, S., and Workman, J. T. Optimal control applied to biological models.CRC Press, 2007.

[30] Leonov, G. Mathematical Problems of Control Theory: An Introduction.

[31] Lotka, A. J. Elements of physical biology.

62

[32] Maxwell, J. C. On governors. Proceedings of the Royal Society of London 16(1867), 270–283.

[33] McKenzie, L. W. The turnpike theorem of morishima (1). Metroeconomica 15,2-3 (1963), 146–154.

[34] McKenzie, L. W. Turnpike theorems for a generalized leontief model. Economet-rica: Journal of the Econometric Society (1963), 165–180.

[35] Mesterton-Gibbons, M. A primer on the calculus of variations and optimalcontrol theory, vol. 50. American Mathematical Soc., 2009.

[36] Morishima, M. Proof of a turnpike theorem: The” no joint production” case. TheReview of Economic Studies (1961), 89–97.

[37] Murray, J. D. Mathematical biology i: An introduction, vol. 17 of interdisciplinaryapplied mathematics, 2002.

[38] Ogata, K., and Yang, Y. Modern control engineering.

[39] Potra, F. A., and Wright, S. J. Interior-point methods. Journal of Computa-tional and Applied Mathematics 124, 1 (2000), 281–302.

[40] R Development Core Team. R: A Language and Environment for StatisticalComputing. R Foundation for Statistical Computing, Vienna, Austria, 2008. ISBN3-900051-07-0.

[41] Radner, R. Paths of economic growth that are optimal with regard only to finalstates: A turnpike theorem. The Review of Economic Studies (1961), 98–104.

[42] Rebaza, J. A first course in applied mathematics. John Wiley & Sons, 2012.

[43] Sager, S., Bock, H. G., Diehl, M., Reinelt, G., and Schloder, J. P.Numerical methods for optimal control with binary control functions applied to alotka-volterra type fishing problem. In Recent Advances in Optimization. Springer,2006, pp. 269–289.

[44] Samuelson, P. A., Dorfman, R., and Solow, R. M. Linear programming andeconomic analysis, 1958.

[45] Sussmann, H. J. A bang-bang theorem with bounds on the number of switchings.SIAM Journal on Control and Optimization 17, 5 (1979), 629–651.

[46] Tainaka, K.-i. Stationary pattern of vortices or strings in biological systems: latticeversion of the lotka-volterra model. Physical Review Letters 63, 24 (1989), 2688.

63

[47] Torres Pineiro, M. O. Breve historia de la teorıa de control. Revista EstudiantilNacional de Ingenierıa y Arquitectura. RNPS 2359. ISSN 2307-471X 2, 1 (2011), 6.

[48] Trelat, E. Controle optimal: theorie & applications. Vuibert Paris, 2005.

[49] Trelat, E. Optimal control and applications to aerospace: some results and chal-lenges. Journal of Optimization Theory and Applications 154, 3 (2012), 713–758.

[50] Trelat, E., and Zuazua, E. The turnpike property in finite-dimensional nonlinearoptimal control. arXiv preprint arXiv:1402.3263 (2014).

[51] Volterra, V. Variazioni e fluttuazioni del numero d’individui in specie animaliconviventi. C. Ferrari, 1927.

[52] Wachter, A., and Biegler, L. T. On the implementation of an interior-pointfilter line-search algorithm for large-scale nonlinear programming. Mathematical pro-gramming 106, 1 (2006), 25–57.

[53] Wilde, R., and Kokotovic, P. A dichotomy in linear control theory. AutomaticControl, IEEE Transactions on 17, 3 (1972), 382–383.

[54] Wright, M. The interior-point revolution in optimization: history, recent develop-ments, and lasting consequences. Bulletin of the American mathematical society 42,1 (2005), 39–56.

[55] Yosida, S. An optimal control problem of the prey-predator system. Funck. Ekvacioj25 (1982), 283–293.

[56] Zaslavski, A. Turnpike properties in the calculus of variations and optimal control,vol. 80. Springer, 2006.

64