Convex Design Control for Practical Nonlinear Systems

0018-9286 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. Seehttp://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI10.1109/TAC.2014.2309271, IEEE Transactions on Automatic Control

1

Convex Design Control for Practical NonlinearSystems

Simone Baldi, Iakovos Michailidis, Elias B. Kosmatopoulos, Antonis Papachristodoulou and Petros A. Ioannou,Fellow, IEEE

Abstract—This paper describes a new control scheme forApproximately Optimal Control (AOC) of nonlinear systems,Convex Control Design (ConvCD). The key idea of ConvCD isto transform the approximate optimal control problem into aconvex Semi-Definite Programming (SDP) problem. Contrary tothe majority of existing AOC designs where the problem thatis addressed is to provide a control design which approximatesthe performance of the optimal controller by increasing the ‘con-troller complexity’, the proposed approach addresses a differentproblem: given a controller of ‘fixed complexity’ it provides acontrol design that renders the controller as close to the optimalas possible and, moreover, the resulted closed-loop system stable.Two numerical examples are used to show the effectiveness ofthe method.

Index Terms—Approximately Optimal Control (AOC), ControlLyapunov Function (CLF), Convex Control Design (ConvCD),Semi-Definite Programming (SDP)

I. I NTRODUCTION

Despite the impressive recent advances in the control designfor nonlinear systems, the problem of providing a scalable,practically feasible, optimal controller has not found yetadefinite answer even for nonlinear systems of small-scale.The “curse-of-dimensionality”, a notion introduced by RichardBellman [1] decades ago, still haunts control engineering.Withthe exception of the narrow class of nonlinear systems that canbe transformed into linear ones by means of transformationsand/or feedback, computing the optimal controller actionsis anNP-complete problem and thus impossible to be implementedin real-time. Despite the significant advances achieved over thelast years in formulating optimal nonlinear control theory[2],[3], [4], [5], [6], these methods have not yet progressed intocontrol applications because of the difficulties associated to thesolution of the Hamilton-Jacobi partial differential equation.

Model Predictive Control (MPC) for nonlinear systems, acontrol approach which has been extensively analysed and suc-cessfully applied in industrial plants during the latest decades

S. Baldi was with Computer Science Department, University of Cyprus,Nicosia, Cyprus, and now is with Informatics & Telematics Institute, Centerfor Research and Technology Hellas (ITI-CERTH), Thessaloniki, [email protected].

I. Michailidis is with Dept. of Electrical and Computer Engineering,Democritus University of Thrace, Xanthi, [email protected].

E. B. Kosmatopoulos is with Dept. of Electrical and Computer Engineer-ing, Democritus University of Thrace, Xanthi, Greece and with Informatics& Telematics Institute, Center for Research and Technology Hellas (ITI-CERTH), Thessaloniki, [email protected].

A. Papachristodoulou is with Department of Engineering Science, Univer-sity of Oxford, [email protected].

P. A. Ioannou is with Department of Electrical Engineering, University ofSouthern California, Los Angeles, CA, [email protected].

[7], [8], [9], faces similar dimensionality issues. In fact,in most cases, predictive control computations for nonlinearsystems amount to numerically solving online a non-convexhigh-dimensional mathematical programming problem. Whilefor linear and/or constrained linear systems the optimizationproblem expressing the performance index is a QuadraticProgram, or alternatively, a Linear Program, for which efficientnumerical routines have been developed, the solution of theoptimization problem for nonlinear systems may require aquite formidable computational burden if online solutionsarerequired.

Optimal solutions based on control Lyapunov functions(CLFs) guarantee closed-loop stability whenever a CLF can befound, and do not require the solution of possibly intractableoptimization problems [10], [11]; however, no systematic pro-cedure exists for finding CLFs of arbitrary nonlinear systems.

Although computing and realizing optimal controllers forgeneral nonlinear systems is practically infeasible, recentlythere has been a significant effort and research activity to-wards designing and implementing Approximately OptimalControllers (AOC), i.e., providing a control design that isboth practically feasible and approximates the performanceof the optimal controller at a satisfactory level, [12], [13],[14], [15], [16]. The key idea in all these approaches is touse approximation techniques in order to come up with apractically feasible control design that effectively approxi-mates the optimal controller. However, although the use ofapproximation techniques significantly reduces the burdeninvolved in the optimal control design, the majority of AOCdesigns still face the “curse-of-dimensionality” problemasthey require the solution of highly non-convex problems. This,in turn, may lead to either impractical designs or optimizationalgorithms that might get trapped into local minima.

Recently, different AOC designs have been proposed anddeveloped: such designstransform the problem of constructingan AOC into a convex problem[17], [18], [19], [20]. In[20] both the system dynamics and the optimal controller areapproximated using PieceWise Linear (PWL) approximators.Using such approximations the optimal control problem isapproximated as a piecewise Linear Quadratic (LQ) prob-lem which admits a PieceWise Quadratic (PWQ) Lyapunov(=optimal cost-to-go) function. In [20] the requirement for thePWQ Lyapunov function to be continuous at the switchingboundaries, along with the Hamilton-Jacobi-Bellman (HJB)equations, results in an overall design that requires the solutionof a convexoptimization problem involving the solution ofLinear Matrix Inequalities (LMIs).

Limited circulation. For review only

Preprint submitted to IEEE Transactions on Automatic Control. Received: February 24, 2014 01:13:59 PST



2

A different convex AOC design was proposed and analyzedin [17], [18], [19]. This design combines the transformationsproposed in [21] for the control design of polynomial non-linear systems using Sum-of-Squares (SoS) polynomials, withfunction approximation results and approximation of positivefunctions by SoS polynomials. The design of [17], [18], [19]involves polynomial approximations for the system dynamics,the controller and the optimal cost-to-go function. The result-ing control design requires the solution of a convex problem(a quadratic Semi-Definite Programming–SDP problem whichcan be transformed into a standard LMI problem using Schurcomplement techniques). There are several advantages of theapproach of [17], [18], [19] as compared to the approach of[20]: (a) the approach of [17], [18], [19] canemploy andhandle nonlinearities in the controller structure, while the oneof [20] is restricted to employing purely linear controllers; (b)it employs smooth controllers, (c) it avoids the requirement forrestricting the switching Lyapunov functions to be continuousat the switching boundaries and (d) most importantly, it canapproximate with arbitrary accuracy the performance of any–feasible – stabilizing controller for the system. However,theprice paid for these advantages is the significant increase ofthe computational requirements: while the approach in [20]requires the solution of LMIs that are of the same orderas the system dimension, the size of LMIs in [17], [18],[19] increases exponentially with the order of the polynomialapproximation used.

In this paper, we propose and analyze a newconvexcontroldesign methodology that combines the advantages of bothapproaches in [20] and [17], [18], [19]. The main featuresof this new control design – abbreviated as ConvCD (ConvexControl Design) – are:

1) A convex AOC methodology which requires the solutionof a convex problem (involving either SDP or LMIs).

2) Like the majority of existing AOC designs, it canapproximate with arbitrary accuracy the performanceof the respective optimal controller by increasing the“controller complexity” (e.g., by increasing the numberof switching linear controllers).

3) The controller complexity must not be “large” for theresulting ConvCD design to provide a stable and efficientperformance. This is particularly useful in practical ap-plications where the complexity of an efficient controlleris known and given (i.e., the number of switching linearcontrollers as well as the strategy for switching amongthe controllers is given). Given such a controller offixed complexity, ConvCD provides a controller whichis as close to the optimal as possible while guaranteeingclosed-loop system stability. Tuning parameters can beused to regulate the transient response (in terms of over-shoot and exponential decay) and to have an estimateof the “distance” of the resulting controller from theoptimal one.

4) The proposed ConvCD methodology overcomes thescalability and computational problems of [17], [18],[19]. In particular, similarly to [20], it involves SDPconstraints (or LMIs) of the same order as the system

state dimension.5) It can handle convex state and control constraints.6) The proposed scheme avoids the requirement of continu-

ity of the Lyapunov function at the switching boundaries.This is made possible by using smooth mixing signalsthat make sure that the switching among the spacepartition is performed smoothly.

7) Finally, the proposed approach is able to provide con-trol designs which use PieceWise NonLinear (PWNL)approximation control schemes instead of PWL ones.This property may prove to be very useful in practicalapplications, where a tedious and complicated design(e.g., for partitioning of the state space, etc.) is requiredfor efficient PWL approximation while the correspond-ing design for PWNL approximation emerges naturally.

The paper is organized as follows: in Sect. II a simpleexample is presented to introduce the ConvCD scheme. Theproblem formulation for the general case is presented in Sect.III. Sect. IV deals with the transformations and approximationin order to put the nonlinear system in a suitable form forthe ConvCD design. Sect. V exposes the ConvCD approach,together with the stability results. Sect. VI shows the effec-tiveness of the proposed method by solving a more complexnonlinear problem.

II. A TUTORIAL EXAMPLE

In this section, we use a simple example to introduce theConvCD approach. Consider the problem of designing a statefeedback controller for the inverted pendulum model of [20]

χ1 = χ2 (1)

χ2 = −0.1χ2 + sin(χ1) + u (2)

whereχ = [χ1 χ2]τ is the state of the pendulum, composed of

angular position and angular velocity, andu is the control inputtorque. We are interested in applying the ConvCD techniqueto find a state feedback control that brings the pendulum tothe upright unstable equilibrium(0, 0), while minimizing thecriterion

J =

∫ ∞

0

(

4χ21(s) + 4χ2

2(s) + u2(s))

ds (3)

The first step in the proposed approach is to impose specialtransformations that render the system dynamics and theobjective criterion in an appropriate format that is convenientfor our developments. In order to keep the example as simpleas possible, no constraints will be considered for this example.Input/state constraints will be considered in Section III,wherethe ConvCD formulation will be exposed for a more generalset of control problems.

After the addition of an integratoru = v we define theaugmented state vectorx = [χτ uτ ]

τ . The second step inthe proposed design is to approximate the system dynamicsas well as the objective criterion (3) using smooth mixingsignals. For the inverted pendulum case, a PieceWise Linear(PWL) model of (1)-(2) can be constructed by finding a PWLapproximation of the system nonlinearitysin(χ1). Similarlyto [20] such nonlinearity is approximated inside the convex





3

interval−4 ≤ χ1 ≤ 4. For reasons that will be clearer as theConvCD methodology is exposed, contrary to [20], where aswitching PWL approximation is considered, here we look fora smooth PWL approximation: this is made possible by usingsmooth mixing signalsβ = [β1 . . . βL]

τ that make sure that theswitching among the space partition is performed smoothly.Besides, by construction, the mixing signals must be non-negative and normalized,i.e,

∑Li=1 βi(χ1) = 1, ∀χ1 ∈ R.

The interval[−4, 4] is divided intoL = 5 overlapping subsetsand five mixing signals are constructed on the basis of the

Gaussian functionψ(χ1, χ0, σχ) = e−

(χ1−χ0)2

σ2χ . Consider the

pre-normalized weights,β = [β1 . . . βL]τ , where βi(χ) =

ψ(χ, χ0i, σχ), andχ0i is the center of the Gaussian function,which belong to a set of the five centers which have beenchosen as−3.2, −1.6, 0, 1.6, 3.2, and σχ = 0.2667 isthe bell width. Different widths can be selected for eachGaussian function: in this example the width is the samefor each bell, since this choice has lead to satisfying results.In order to obtain a normalized set of mixing signals, themixing signalsβi(χ1) are generated by normalizingβ, i.e.,βi(χ1) = βi(χ1)/

∑Lj=1 βj(χ1), i = 1, . . . , L. The mixing

signals resulting from this procedure are shown in Fig. 1(a).Once the mixing signals have been defined, it is possible toapproximate thesin(χ1) term by:

sin(χ1) = β1(χ1) · (−θ1χ1 − θ2) + β2(χ1) · (−θ3) +β3(χ1) · (θ4χ1) + β4(χ1) · (θ3) +β5(χ1) · (−θ1χ1 + θ2) (4)

Note that the anti-symmetry of thesin function has beentaken into account for the piecewise linear approximation.Thevector [θ1 θ2 θ3 θ4]

τ can be found using a Least Squaresmethod. Fig. 1(b) shows the PWL approximation ofsin(χ1)via (4) inside the convex interval[−4, 4]. For the invertedpendulum case, the partition (4) can be viewed as a partitionin the variableχ1, while the partition is independent ofχ2. InSect. III the mixing functionsβi will be generalized so as tobe functions of the entire statex. For this reason, from nowon, let us denote the mixing functions byβi(x), i = 1, . . . , L.

−4 −3 −2 −1 0 1 2 3 4

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

χ1

(a) Mixing signals βi(χ1), i =1, . . . , 5

−4 −3 −2 −1 0 1 2 3 4−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

χ1

(b) Resulting PWL approximation

Fig. 1. Piecewise Linear approximation ofsin(χ1)

After the described PWL approximation, we arrive at asystem of the form

x ≈L∑

i=1

βi(x) (Aix(x) +Bv) (6)

wherex(x) = [χ1 χ2 1 u]τ , B = [0 0 1]τ , andAi dependingon the coefficientsθ as shown in (5). The system (6) can berewritten as

x ≈ Φ(x)z(x) +Bv (7)

where

Φ(x) =[ √

β1(x)A1 . . .√

βL(x)AL

]

and the vectorz(x) is defined as follows:

z(x) =

√

β1(x)x(x)...

√

βL(x)x(x)

(8)

Accordingly, the objective function (3) to be minimized canbe written in the form

J =

∫ ∞

0

L∑

i=1

βi(x) (xτQix) ds (9)

by defining the weight matricesQi = blkdiag 4, 4, 0, 1, i =1, . . . , L.

By applying all the transformations, the optimal state feed-back design problem can be cast as

minimize J =

∫ ∞

0

(zτ (s)Qz(s))) ds (10)

subject to

x = Φ(x)z(x) +Bv + ν (11)

whereν stands for the approximation error due to the replace-ment of the actual system dynamics (1)-(2) by (7).

Application of the Hamilton-Jacobi-Bellman (HJB) equa-tion to the above problem results in the following equation

−zτQz =∂V

∂x

τ

(x)(

Φ(x)z(x) +Bv∗ + ν)

(12)

whereV is the optimal-cost-to-go function, i.e.,

V (x(t)) =

∫ ∞

t

(zτ (s)Qz(s)) ds,

v∗ denotes the optimal control andν is the error approximationterm that results from the earlier approximations.

The optimal cost-to-go functionV is a Control LyapunovFunction (CLF) for the controlled system: ifχ1 ∈ X , theCLF V can be approximated – with accuracyO(1/L) – usinga piecewise approximation of quadratic Lyapunov functions,as follows:

V (x) ≈L∑

i=1

βi(x)xτ (x)Pix(x) = zτ (x)Pz(x) (13)

whereP is a constantpositive definite matrixwith P beingsymmetric and having the followingblock diagonal form

P =

P1 . . . 0

0. . . 0

0 . . . PL

(14)

where Pi are dim(x)2-dimensionalsymmetric and positivedefinite matrices.





4

A1 =

[

0 1 0−θ1 −0.1 −θ2

]

, A2 =

[

0 1 00 −0.1 −θ3

]

, A3 =

[

0 1 0θ4 −0.1 0

]

,

A4 =

[

0 1 00 −0.1 θ3

]

, A5 =

[

0 1 0−θ1 −0.1 θ2

]

(5)

The approximation of the optimal-cost-to-go function via(13) can be shown mathematically also in the general case(e.g., X ⊂ R

n, with n > 1, andβi possibly depending on theentire statex) and it will be shown in Sect. V (cf. Lemma 1).

Similar to the use of approximations for the optimal cost-to-go function, the optimal controllerv∗ can be approximatedas follows:

v∗ ≈L∑

i=1

βi(x)Giz(x) = Γ(x)Gz(x) (15)

whereGi are constant matrices, and

G =

G1

...GL

, Γ(x) = [β1(x)I, . . . , βL(x)I]

Using the approximations (7), (13) and (15), the HJBEquation (12) can be written as follows:

0 = zτ(

[

Φ(x) +BΓ(x)G]τMτ

z P

+PMz

[

Φ(x) +BΓ(x)G]

+Q

)

z − ν (16)

≡ GP,G(x)− ν

whereMz ≡Mz(x) denotes the matrix whose(i, j)-th entry isgiven byMz,ij(x) = ∂zi(x)/∂xj and ν is theapproximationerror term that is inversely proportional to the numberL ofmixing signals, resulting from the approximations (7), (13) and(15). Moreover, as the optimal cost-to-go functionV (x) is aCLF for the system (7) we have that closed-loop stability ispreserved by the control scheme ifV < 0 for x 6= 0, or,equivalently using the approximations (7), (13) and (15) ifthefollowing inequality holds

LP,G(x)= zτ

(

[

Φ(x) +BΓ(x)G]τMτ

z P (17)

+PMz

[

Φ(x) +BΓ(x)G]

)

z < 0, ∀x 6∈ B(ν)

Equations (16) and (17) indicate that it suffices to chooseP,G so that the termGP,G is as small as possible subjectto the constraint thatLP,G(x) is – almost – negative definite.In other words, the problem of constructing an approximatelyoptimal performance can be cast as the following optimizationproblem:

min ‖GP,G(x)‖2 + γ (18)

s.t. P ≻ 0

LP,G(x) ≤ −γ, γ > c > 0, ∀x 6∈ B(ν)

wherec is a small positive user-defined constant and where1

B(ν) denotes a ball centered at the origin and having radiusproportional toν.

Unfortunately, as the above optimization problem is nonlin-ear with respect to the unknownsP,G, attempting to solve(18) is non-convex– and thus difficult to solve – even in thecase where the approximation-related termν is negligible. Tocircumvent this problem we work similarly to [21], [17]-[19]:by multiplying the terms inside the parenthesis of (16) byP−1

from left and right we obtain that

FP,F,Q(x)= zτ

([

P(Φ(x))τ + FτΓτ (x)Bτ]

Mτz

+Mz

[

Φ(x)P+BΓ(x)F]

+ Q)

z

= ¯ν (19)

where ¯ν = O(ν),

P= P−1, Q

= PQP ≡ P−1QP−1 (20)

andF is a matrix satisfying

F = GP (21)

Working similarly on (17) we obtain

HP,F(x)= zτ

([

P(Φ(x))τ + FτΓτ (x)Bτ]

Mτz (22)

+Mz

[

Φ(x)P+BΓ(x)F])

z < 0, ∀x 6∈ B(ν)The above transformations play a crucial role in our ap-

proach: instead of attempting to solve the non-convex opti-mization problem (18), we solve a relaxed version – which isconvex – that involves the functionsFP,F,Q andHP,F:

min∥

∥FP,F,Q(x)∥

∥

2+ γ (23)

s.t.

ǫ1I P ǫ2I, ǫ3Q Q

HP,F(x) ≤ −γ, γ > c > 0, ∀x 6∈ B(ν)where ǫi, i = 1, 2, 3 are some positive design constants(with ǫ2 > ǫ1). In the next section, the relationship betweenthe optimization problems (18) and (23) is investigated, andit is shown how the two problems can be made close byopportunely tuning the design parameters. Besides, even forthe sake of simplicity, scalar design parametersǫi, i = 1, 2, 3are considered, all the mathematical results are replicable byconsideringPl P Pu, QlQ Q, with Pl, Pu, Ql

matrices of appropriate dimension.Despite the fact that the optimization problem (23) is a

convex problem, its solution requires discretization of the

1Note that throughout this paper the notationB(·) is used in a similar wayas the notationO(·): B(a)is used to denotea ball of radius proportional toa and not a ball of radiusa.





5

state-space as it is aninfinite-dimensional, state-dependentproblem. Fortunately, as shown in Sect. V, due to the particularform of (23), the number of discretization points does nothave to be as large as it would be required in typical state-dependent optimization problems: as shown in Theorem 1presented below, the number of discretization points can beas few as the total number of free variables in the matricesP, Q,F.

So, instead of (23), it is possible to solve the convex problem

min

N∑

i=1

‖FP,F,Q(x[i])‖2 + γ (24)

s.t.

ǫ1I P ǫ2I, ǫ3Q Q

HP,F(x[i]) ≤ −γ, γ > c > 0, ∀x 6∈ B(ν)

whereN is any integer satisfyingN ≥ N andN denotes thetotal number of free variables of the matricesP, Q,F.

The stability properties of the proposed approach will bemathematically shown in Sect. V. We conclude this sectionby demonstrating how the proposed method performs on theinverted pendulum example. The ConvCD algorithm (24) isrun with N = 300, ǫ1 = 1, ǫ2 = 20, ǫ3 = 1, L = 5 andthe resulting controller is compared to three other controllers:the first one is the controller resulting from the local lin-earization of the pendulum dynamics around the equilibrium,via the Taylor-MacLaurin approximationsin(χ1) = χ1, andthe solution of the LQ problem associated to the resultinglinear system that minimize the performance objective (3).Wecall this controller “local-linearizing controller”. Thesecondcontroller is the one resulting from the optimization approachdescribed in [20]. The third controller is the one, in thesame form as in (15), which minimizes the cost (3), subjectto the actual plant (1)-(2) (and not the approximated one).This problem is solved by numerically optimizing, via thefmincon routine of Matlab, the trajectory from an initialcondition to the origin. The resulting controller has been called“simulation-optimized controller”. Also, two more controllers,resulting from the ConvCD algorithm after the approximationof the sin function withL = 1 andL = 3 mixing functions,has been considered. The approximation procedure is similarto the case withL = 5 and, for the sake of brevity, it is notexposed in details.

The six controllers are compared by starting from a com-mon initial conditionx0, and by calculating the performanceobjective (3). The results are shown in Table I. It can beseen how the proposed approach achieves better results thanthe local-linearizing controller (forL = 3 and L = 5)and a performance which is comparable to the approachexposed in [20] (forL = 5). Besides, the ConvCD ap-proach manages to find a performance which is close toone achieved by the simulation-optimized controller. Theperformance of the simulation-optimized controller is thebestone since the optimization procedure is not based on anapproximate model of the plant. Also note how the ConvCDperformance increases as the number of mixing functionsincreases. As a final comment, we found via simulations

an estimate of the region of attraction for semiglobal sta-bility. The region of attraction has been estimated usingan ellipsoid: call X0 such an estimate: it is observed thatthe size ofX0 increases by increasingL. In particular wehave X0 =

[χ1 χ2 u]τ : χ2

1/0.752 + χ2

2/2.252 + u2 ≤ 1

for L = 1, X0 =

χ21/2.25

2 + χ22/4.5

2 + u2 ≤ 1

for L = 3,and X0 =

χ21/4

2 + χ22/9.75

2 + u2 ≤ 1

for L = 5. Theseestimates are less conservative than the estimates obtainedusing the technique described in the Appendix, which are notreported for the sake of brevity.

III. G ENERAL PROBLEM FORMULATION

In this section, we consider the problem of designing astate feedback controller for a general nonlinear system whichassumes the following dynamics

χ = F (χ, u) (25)

whereχ(t) ∈ ℜn, u(t) ∈ ℜm denote the vectors of systemstates and control inputs, respectively andF is a nonlinearvector function (assumed to be continuous). The problem athand is to construct a state-feedback controller

u = k∗(χ)

which makes the zero equilibrium of the closed-loop systemstableand moreover, solves the followingconstrained optimalcontrol problem:

minuJ =

∫ ∞

0

Π(χ(s), u(s))ds (26)

s.t. χ = F (χ, u)

umin ≤ u ≤ umax (27)

C(χ, u) ≤ 0 (28)

where Π is a bounded-from-below, continuous function ofits arguments;umin, umax denote the vectors of minimumand maximum, respectively, allowable control signals;C :ℜn×m → ℜp is a smooth convex nonlinear vector function.Inequality (28) can represent a large class of constraints metin practical applications. Without loss of generality, we willassume that the minimum ofΠ(χ, u) is attained atΠ(0, 0) andthat all constraints are satisfied for(χ, u) = (0, 0).

Please also note that inequality (28) includes as a specialcase the limited control constraint (27). However, due to thespecial treatment of limited control constraints in the nextsection, we include (27) as a different set of constraints thanthose of (28).

In order to have a well-posed problem we will assume thatthe controller solving the optimal control problem (26)-(28)provides closed-loop stability, i.e., we will assume that

(A1) Let u∗ = k(χ) be the controller that solves theoptimal control problem (26)-(28). Then, for alladmissible initial statesχ(0), the closed-loop system(25) under the feedbacku∗ = k(χ) is stable.





6

Table I: Inverted pendulum perfomance results

local-linearizing ConvCD (L = 1) ConvCD (L = 3) Controller [20] ConvCD (L = 5) simulation-optimizedJ 16.6 16.6 15.9 15.4 15.4 15.2

IV. T RANSFORMATIONS& A PPROXIMATIONS

A. Adding a – nonlinear – integrator

The first step in the proposed approach is to imposespecial transformations that render the system dynamics, theconstraints and the objective criterion in an appropriate formatthat is convenient for our developments. To do so, we definea new fictitious “control” inputv that is calculated accordingto

˙u = v, u = S(u) (29)

with S(·) being a smooth and invertible function such that

umin ≤ S(u) ≤ umax (30)

It is worth noticing that by adding the integrator (29) andintroducing the functionS satisfying (30), the problem ofdesigning the control vectoru so that constraint (27) issatisfied is transformed into the problem of designingv thatdoes not need to satisfy a boundedness constraint like (27):while v can take “arbitrary” values, the actual control vectoru is restricted – due to the use of functionS – to satisfy (27).

We can rewrite system (25) and the constraints (28) asfollows:

x = f(x) +Bv (31)

C(x) ≤ 0 (32)

wherex = [χτ uτ ]τ , and

f(x) =

[

F (χ, S(u))0

]

, B =

[

0I

]

, C(x) = C (χ, S(u))

B. Approximations using Mixing Signals

The second step in the proposed design is to approximatethe transformed system dynamics (31) as well as the objectivecriterion using smooth mixing signals. More precisely, we letβi, i = 1, . . . , L denote a set of smoothmixing signalsthatsatisfy the following properties:

βi(x) ∈ [0, 1],

L∑

i=1

βi(x) = 1, ∀x (33)

Remark 1: The mixing functions described by (33) includea large number of descriptors for approximation of nonlinearsystems. For example, by using (33), it is possible to describePWL systems with partition of the state space with polytopiccells [22], [23], as well as many Fuzzy dynamical systems,like the Takagi-Sugeno ones [24]. In the first case the mixingfunctions will be a smooth version of the state space partition,i.e., if a state cell Si has the polytopic formGτ

i x ≥ 0,i = 1, . . . , L, then the corresponding mixing function can beobtained from the sigmoidal functionβi(x) = 1/(1+ e−Gix),and subsequent normalization with respect to the other statecells, i.e., βi(x) = βi(x)/

∑Lj=1 βj(x), i = 1, . . . , L. In such

a way, each mixing function is active when the correspondingcell will be active. Note that different mixing functions can be

used in place of the sigmoidal one, like the gaussian functionused in the tutorial example. In the Takagi-Sugeno case, themixing functions will have the form of fuzzy membershipfunctions,i.e., βi(x) = µSi(x) whereµSi(x) is the member-ship functions over the fuzzy setSi. Again, normalization isnecessary if the membership functions do not sum to one.The interested reader is referred to [22], [24], [25] andreferences therein for numerical techniques for generatingapproximations of nonlinear systems using PWL and Fuzzytechniques. PWL and Fuzzy techniques include a large but notexhaustive set of tools for descriptors of nonlinear systems:for example, the proposed method can also handle piecewisenonlinear approximations using Sum of Squares [26].

Remark 2: In the majority of existing AOC designs addressthe problem of providing a control design which approximatesthe performance of the optimal controller by increasing the“controller complexity” (e.g., by increasing the number ofswitching linear controllers). Typically, the resulting AOCdesign provides an efficient performance if the controllercomplexity is “large enough” which, in turn, may result incontrol designs that cannot be practically implemented. Theproposed ConvCD approach attacks a different problem: givena controller of a fixed complexity (e.g., given the number ofswitching linear controllers as well as the strategy for switch-ing among the controllers), ConvCD provides a controllerwhich is as close to the optimal as possible while guaranteeingclosed-loop system stability. The practical significance of suchan approach is the following: in many practical cases, wherea PWL or Fuzzy approximation of the nonlinear systems isavailable, the complexity of a controller leading to satisfactoryperformance is also known, without the need to look for anaccurate (and computationally expensive) fine approximationof the system.

Using the above design considerations for the mixing signalsβi, we consider the following approximations of the systemdynamics and the objective criterion2 as follows:

x ≈L∑

i=1

βi(x) (Aix(x) +Bv) (35)

Π(χ, u) ≡ Π(x) ≈L∑

i=1

βi(x) (xτQix) (36)

2In the approximation (35) we omitted, in order to to avoid lengthy andcomplicated formulations, theconstant terms. All the results of this paper canbe readily extended to the case where an approximation with constant termsis employed. In such a case the the approximation (35) is replaced by thefollowing one

˙x ≈L∑

i=1

βi(x) (Aix(x) + ai +Bv) (34)

which can be can be recast as in (35) via the state extensionx = [x(x)τ 1]τ

and

Ai =

[

Ai ai0 0

]





7

whereAi, Qi are constant matrices withQi being positivesemidefinite, and

x(x) =

[

(x)σ(x)

]

with (x) being any smooth (also nonlinear) function ofxsatisfying

(x) = 0 ⇐⇒ x = 0

andσ(x) is defined according to

σi(x) = eαCi(x)−η − eαCi(0)−η, i = 1, . . . , p (37)

andα is a large positive constant andη is a positive constant.The functionσi(x) in (37) has been chosen in such a waythat σi(0) = 0, and, if a particular constraintCi(x) ≤ 0 is –or is about to be – violated, then the respectiveσi(x) takes avery large value, while it is negligible when the constraintissatisfied.

Remark 3: Contrary to standard PWL approximation [inwhich case the vectorx(x) coincides withx, as in the invertedpendulum example of Sect. II], the PieceWise NonLinear(PWNL) approximations (35) and (36) might involve the use ofthe nonlinear vectorx(x) comprising the nonlinear subvectors(x) andσ(x). The use of these nonlinear subvectors withinx(x) is made for the following reasons:

• The use of a nonlinear vector (x) instead of a purelylinear one may be proven extremely useful in order tosimplify the design for the partitioning of the state space.Typically the introduction of nonlinear terms might helpto reduce the number of state-space partitions needed toapproximate the nonlinear systems.

• The nonlinear termσ(x) is included in the control signalso that it includes information on whether a particularconstraint is – or is about to be – violated.

As a final step, the constrained optimization problem (26)-

(28) is transformed into anunconstrainedone, by incorporat-ing the constraints (28) – or, equivalently, the constraints (32)– as penalty functions into the objective function. This is madepossible by making use of (37) and by replacing the objectivefunction (26) by the following one

J =

∫ ∞

0

( L∑

i=1

βi(x(s)) (x(s)τQix(s))

+

dim(C(x))∑

j=1

σ2j (x(s))

)

ds

By using the fact that∑L

i=1 βi(x) = 1 and the definition ofzin (8), the augmented objective functionJ can be written inthe following compact form:

J =

∫ ∞

0

(zτ (s)Qz(s)) ds (38)

where

Q =

Q1 +

[

0 00 I

]

. . . 0

0.. . 0

0 . . . QL +

[

0 00 I

]

(39)

C. HJB and Controller Approximations

After applying all transformations presented in the previoussection, we have that the optimal state feedback design prob-lem can be cast as an unconstrained optimal control problemof the form

minimize J =

∫ ∞

0

(zτ (s)Qz(s))) ds (40)

subject to

x = Φ(x)z(x) +Bv + ν (41)

whereν stands for the approximation error due to the replace-ment of the actual system dynamics (31) by (35).

As the proposed approach employs approximation tech-niques it is not possible to provide a globally stable feedbackcontrol design. Instead asemi-globally stablefeedback con-trol design can be provided which, given a compact subsetX0 ⊂ ℜn+m of admissible initial states constructs a controldesign that produces closed-loop stable solutions provided theinitial statex(0) belongs toX0. Therefore, additionally to (A1)we will assume that

(A2) The initial statesx(0) of the system belong to acompact subsetX0 ⊂ ℜn+m.

Associated to the subsetX0 of admissible initial conditionswe will define a “sufficiently large” compact subsetX whichcontainsX0, such that every trajectory starting insideX0 willbe contained inX . An estimate of the size of the region ofattraction is provided in the proof of the main Theorem on theAppendix.

Remark 4: Note that, in casex lies in the compact subsetX ⊂ ℜn+m, then the amplitude of the termν ≡ ν(x) canbecome arbitrarily small and is inversely proportional to thenumberL of mixing signals.

Moreover, in order for the approximators of the optimalcost-to-go function and the respective optimal controllerto beable to provide with arbitrary accuracy, the optimal cost-to-go function and the respective optimal controller continuous.In other words, we additionally have to impose the followingassumption.

(A3) The optimal cost-to-go functionV (x) and the respec-tive optimal controlv∗ which correspond to the HJBsolution are continuous functions (wrt.x).

V. THE CONVCD APPROACH

Following the same mathematical steps exposed after theHJB equation (12), we arrive to the convex optimizationproblem (23).





8

The following Lemma, which justifies the choice of (13)as an approximation of the optimal-cost-to-go function isintroduced.

Lemma 1: Let (A1)-(A3) hold and assume thatx ∈ X . Theoptimal-cost-to-go functionV can be approximated – withaccuracyO(1/L) – using a Sum-of-Squares (SoS) polynomial(13), withP being symmetric and having theblock diagonalform of (14) andPi are dim(x)2-dimensionalsymmetric andpositive definite matrices.

Proof: See the Appendix.Remark 5: Similarly to [17]-[19], the proof of Lemma

1 is based on the observation that the optimal cost-to-gofunction V , being a Control Lyapunov Function (CLF) forthe controlled system, can be approximated using a SoSpolynomial. However, that while in [17]-[19], the vectorzis vector of monomials, whose size increases exponentiallywhile increasing the order of the monomials, in (13) the sizeof z increases linearly by increasing the number of mixingelements. Due to the use of piecewise approximations, thematrix P in (13) is of block diagonal structure which reducessignificantly the computational burden of the ConvCD designas compared to the use of high order monomials in [17]-[19].

Remark 6: At this point we should emphasize that the twooptimization problems (18) and (23) are not equivalent, being(23) a convex relaxation of the non-convex problem (18). Thisis due to the fact that the optimal solution coming from (23)might not satisfy

Q = P−1QP−1 (42)

Condition (42) is a non-convex constraint which cannot beimposed without destroying the convexity of (23). To solve sucha problem, the condition (42) is “approximately” imposed byappropriately re-definingx(x), Q, P and Q. Considering forsimplicity the case where no constraints (28) are present, weaugment the vectorx by adding the element

√Π as its first

component. Then it suffices to chooseQ = diag(1, δ, . . . , δ)where δ is a small positive design constant and restrict thematricesP and Q to admit following block diagonal form

P =

1 0 0 . . . 00 P1 0 . . . 00 0 1 . . . 0

0 0 0. . . 0

0 0 0 . . . PL

, Q =

1 0 0 . . . 00 Q1 0 . . . 00 0 1 . . . 0

0 0 0. . . 0

0 0 0 . . . QL

where Pi, Qi are symmetric positive definite matrices con-strained to satisfyǫ1I Pi ǫ2I, ǫ3I Qi δI.Under such a choice it is not difficult to see thatQ =P−1QP−1 + O(δ) and thus condition (42) approximatelyholds. Therefore, by imposing the above described redefini-tions, the two optimization problems (18) and (23) becomeapproximately equivalent, i.e., the smaller isδ, the closer arethe solutions of the two problems. Similar – but lengthier –transformations can be imposed in the case where constraintsof the form (28) are present.

In the sequel and in order to avoid lengthy complications,we will assume that condition (42) holds for a particularchoice of the design constantsǫi and the set-up describedso far i.e., without needing to redefinex(x), Q, P and Q

as described above. All the arguments of the paper can bestraightforwardly extended in the case where (42) does nothold, in which case redesign/redefinition ofx(x), Q, P andQ

is required. Table II presents the proposed procedure for solving the op-

timization problem (23) and constructing the proposed controldesign scheme. Despite the fact that the optimization problem(23) is a convex problem, its solution requires discretization ofthe state-space as it is aninfinite-dimensional, state-dependentproblem. Fortunately, due to the particular form of (23), thenumber of discretization points does not have to be as large asit is required in typical state-dependent optimization problems:as it will be seen in Theorem 1 presented below, the numberof discretization points can be as few as the total numberof free variables in the matricesP, Q,F. This is contraryto alternative AOC approaches that typically require a densegrid covering the state-space, leading to a large number ofdiscretization points.

Remark 7: The matricesP, Q should be positive definite.As a result any approach that attempts to solve (43) shouldconstrain these matrices to be positive definite; SDP methodsshould be engaged towards such a purpose. Please note thatdue to the special structure of the matricesQ,P given in (39)and (14), respectively, the SDP constraints

ǫ1I P ǫ2I, ǫ3Q Q

in (43), can be broken down intoL independent smallerconstraints, thus decreasing the computational load. Finally,Schur complement techniques – see e.g. [27] – may be usedto transform the SDP subject to second-order cone constraints(43) into a SDP subject to Linear Matrix Inequality (LMI)constraints.

The next theorem establishes the stability properties of theoverall scheme presented in Table II.

Theorem 1:Fix the numberL of mixing signals and theconstantsǫi, i = 1, 2, 3, let P, G be constructed according tothe design procedure of Table II and let (A1)-(A3) hold. LetalsoM be any positive integer satisfyingM ≥ N , whereN– see also Table II – denotes the number of free variables ofthe matricesP, Q,F. Select3 randomlyM points x[j] ∈ Xand let

z[j] = z(x[j])

Finally, suppose that the positive design constantsǫi arechosen so that

ǫ1I P∗−1 ǫ2I, ǫ3Q P∗−1QP∗−1 (45)

whereG∗,P∗ denote the optimal solutions to the optimizationproblem (18).

Then, the following statements hold:(a) Let C1 > 0 be the smallest positive constant such that, forall j ∈ 1, . . . ,M,

‖z[j]‖ > C1 ⇒ LP,G(x[j]) < 0 (46)

3Note that although it is possible forN andx[i] of Table II to coincidewith M and x[j], respectively, it is advisable that they do not coincide sothat the performance of the proposed control design is evaluated at differentstate points than the ones used to construct the ConvCD solution.





9

Table II: The ConvCD Approach

Step 1.Calculate the matricesP, Q,F as follows: LetN denote the total number of free variables of these matrices. Select randomlyN pointsx[i] ∈ X ,whereN is any integer satisfyingN ≥ N and solve the following convex optimization problem:

minN∑

i=1

‖FP,F,Q(x[i])‖2 + γ (43)

s.t.

ǫ1I P ǫ2I, ǫ3Q Q

HP,F(x[i]) ≤ −γ, γ > c > 0, ∀x 6∈ B(ν)

with ǫi user-defined positive constants, andFP,F,Q(x[i]) andHP,F(x[i]) as defined respectively in (19) and (22).

Step 2.By using the solution of the above optimization problem, we canextract the estimates of the matricesP,G in (18) according to

P = P−1, G = FP−1 (44)

Step 3.The proposed control scheme is the controller given by (15) and (29) by settingG equal toG.

then, the closed-loop system is stable and its solutions con-verge to the subset

D =

x : ‖z(x)‖ ≤ C1 +O(

1

M

)

(47)

Moreover, the closed-loop solutions of the system satisfy

‖x(t)− xopt(t)‖ ≤ C2 +O(

1

M

)

(48)

wherexopt denotes the closed-loop solutions of the systemunder the optimal controlv∗ andC2 is a nonnegative constantsatisfying

C2 = O(

GP,F(x

[j]))

(b) For eachC1 > 0, C2 ≥ 0, there exists a lower bound onthe approximators’ sizeL so that (46) and (48) hold for allchoices of approximators’ sizeL satisfyingL ≥ L. ♦

Proof: See the AppendixSeveral remarks are in order:

• The optimization problem (23) is aconvex one: theoptimization criterion comprises a quadratic function withrespect to the decision variablesP,F, Q while all ofthe constraints are Semi-Definite constraints (and thusconvex).

• Most importantly, according to Theorem 1,the solutionto the optimization problem (23) is able to provide anefficient control design even in cases where the approx-imation error is not negligible. Theorem 1 provides aneasy-to-calculate formula – see relation (46) – to checkwhether a particular choice for the approximators sizeLprovides the required controller efficiency. In other words,even in cases where the particular choice forL is farfrom providing a close-to-the-optimal performance (i.e.,a significantly largerL – and thus a significantly morecomplicated controller – is required to get a close-to-optimal performance), the proposed scheme provides acontrol design that is efficient.

Furthermore to Theorem 1 and by using the same arguments asin [17] it can be seen that in case (46) holds, then the solutions

of the overall system satisfy the following inequality:

‖z(t)‖ ≤ α1 exp−α2t ‖z(0)‖+ α3 (49)

α1 =

√

ǫ2ǫ1, α2 =

ǫ1λmin(Q)

2ǫ3α3 = C = O(1/L)

What is important about (49) is that the design constantsǫiin the optimization problem (56) can serve as tuning/designparameters in a similar fashion as e.g., the LQ matrices inLinear-Quadratic control design applications: (49) can beusedto evaluate the effects and trade-offs of different choicesforǫi on the overshoot, convergence and steady-state closed-loopperformance and thus it can provide a guide on how to chooseǫi so that the desired performance is obtained.

Remark 8: We close this section by emphasizing the factthat, in the case wherex(x) ≡ x and no input/state constraintsare given, because of the block diagonal structure of theLyapunov matrix (13)-(14), the computational complexity ofthe control problem is comparable with the solution ofLRiccati equations,i.e., the computational requirements ofthe control scheme resulting using the proposed approachincreases linearly with the complexity of the controller.

VI. N UMERICAL EXAMPLE : FREEWAY MODEL

In this section, we demonstrate how the methodology wepresented can be applied in a second, more realistic example:congestion reduction on a freeway. Traffic on a freewaycan be described by a nonlinear system, composed of manyinterconnected subsystems, namely the sections by which thefreeway is divided into, see Fig. 2. One of the control strategiesproposed in the literature to control congestion on a freeway isthe use of variable speed limits across the sections [28], [29].The dynamics of a freeway withS sections are described bya nonlinear dynamical system, whose exact description can befound in [28], [29], and which takes the form

ρ = G(ρ, ν) (50)

whereρ = [ρ1, . . . , ρN ]τ are the mean densities across the sec-tions andν = [ν1, . . . , νS ]

′ the speed limits in each sections.





10

The system dynamics (50) are subject to the constraints:

0 ≤ ρi ≤ ρjam, i = 1, . . . , S (51)

vmin ≤ νi ≤ vf , i = 1, . . . , S (52)

with vf the free flow speed andvmin the minimum speed limit.The system (50) has a fixed point equilibriumρeqi = qd/vf ,i = 1, . . . , S and νeqi = vf , i = 1, . . . , S, which representsthe equilibrium in the free flow condition, to which we wouldlike to drive the system. So we defineχi = ρi − ρeqi , i =1, . . . , S andui = νi − νeqi , i = 1, . . . , S, and we apply thetransformations to obtain a system in the form of (31).

Fig. 2. Freeway stretch divided intoN section

For the control problem at hand we consider a freewaymodel with S = 4 sections, andρjam = 0.75[veh/m] ,vf = 29.17[m/s], vmin = 13.89[m/s]. The control objectiveis to chooseui, i = 1, 2, 3, 4, so as to optimize the criterion

J =

∫ ∞

0

[

4∑

i=1

χ2i (τ) + 10−4

4∑

i=1

u2i (τ)

]

dτ (53)

Such a criterion describe the desired to operate close to thefree flow conditions while keeping the control action small.The above criterion along with the constraints (51) and (52)were transformed into the unconstrained objective criterion Jusing the approach detailed in section III.

The ConvCD algorithm described in Table II was imple-mented withe1 = 1, e2 = 2, e3 = 10−2 and N = 2000discretization points. Two types of synthesis have been made,one employing just one mixing function,L = 1 (β1 ≡ 1),and a second one employing 2 mixing functions,L = 2. Inthe second case, the two mixing functions have been chosenin the following way. Consider the pre-normalized weights,i = 1, 2

βi(χ) = e−

∑4j=1(χj−χ0i)

2

σ2χ (54)

whereχ01 = 0, χ02 = 0.6, and σχ = 0.5. These weightsrepresent the freeflow freeway condition and the congestedfreeway condition. The mixing signalsβ(χ) are generated bynormalizing β = [β1 β2], i.e., βi(χ) = βi(χ)/

∑2j=1 βj(χ),

i = 1, 2. Different mixing functions can be considered, butthe following choice has been verified to approximate in asatisfactory manner the 4-section freeway.

Both controllers found from the ConvCD methodology withL = 1 and L = 2 have been tested in simulations under acongested scenario, and compared with the not controlled (oropen loop) strategy,i.e., whenνi(t) ≡ vf , ∀t. The objectivefunction (53) has been calculated, in order to evaluate theimprovement respect to the case with no control action. Theresults are summarized in Table III. Both controllers comingfrom the ConvCD procedure improves the performance indexcoming from the open loop strategy. The controller employing

2 mixing functions achieves a better performance than thecontroller with 1 mixing function.

In Fig. 3, 4, 5 the four mean densities across the section, aswell as the four variable speed limits are shown, respectivelyfor the open loop case, for the ConvCD controller withL = 1,and for the ConvCD controller withL = 2. It can be seen that,as soon as congestion appears in the fourth section, the twoConvCD controllers decrease the speed limits in the previoussections, in order to allow less cars to enter into the congestedzone (Fig. 4(b) and 5(b)): this strategy allows a decrease inthepropagation of the congestion wave to the previous sectionsof the freeway. An estimate of the region of attractionfor semiglobal stability for the two controllers is:X0 =[ρτ ντ ]τ : 0 ≤ ρi ≤ ρjam/4.5, vmin ≤ νi ≤ vf , i = 1, . . . , 4for L = 1 and X0 =[ρτ ντ ]τ : 0 ≤ ρi ≤ ρjam/3.25, vmin ≤ νi ≤ vf , i = 1, . . . , 4for L = 2, which are acceptably inside the practical operatingrange of the freeway system.

0 2000 4000 6000 8000 10000 12000 14000 16000 180000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

ρ(t)

time [s]

(a) Mean densitiesρi, i = 1, . . . , 4:ρ1 (dotted), ρ2 (dashed),ρ3 (dash-dotted),ρ4 (solid)

0 2000 4000 6000 8000 10000 12000 14000 16000 1800010

15

20

25

30

35

ν(t)

time [s]

(b) Speed limitsνi, i = 1, . . . , 4:ν1 (dotted), ν2 (dashed),ν3 (dash-dotted),ν4 (solid)

Fig. 3. Congestion reduction on a freeway: open loop strategy

0 2000 4000 6000 8000 10000 12000 14000 16000 180000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

ρ(t)

time [s]


0 2000 4000 6000 8000 10000 12000 14000 16000 1800010

15

20

25

30

35

ν(t)

time [s]


Fig. 4. Congestion reduction on a freeway: ConvCD controller (L = 1)

VII. C ONCLUSIONS& FUTURE RESEARCH

A control design methodology for Approximately OptimalControl (AOC) of nonlinear systems has been presented. Theproposed design, called ConvCD, transforms the problem ofconstructing an AOC into a convex problem, involving thesolution of a Semi-Definite Programming (SDP) problem.The ConvCD methodology, is scalable in the sense that itinvolves SDP constraints of the same order as the systemdimension. The stability properties of the proposed approachhave been analyzed: the proposed scheme guarantees semi-global stability of the nonlinear closed-loop system. Besides,





11

Table III: Freeway control results

No control ConvCD (L=1) Improv. ConvCD (L=2) Improv.J 1318.7 1215.5 7.8 % 1141.6 13.4 %

0 2000 4000 6000 8000 10000 12000 14000 16000 180000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

ρ(t)

time [s]


0 2000 4000 6000 8000 10000 12000 14000 16000 1800010

15

20

25

30

35

ν(t)

time [s]


Fig. 5. Congestion reduction on a freeway: ConvCD controller (L = 2)

given a user-defined optimality criterion, the approximatelyoptimal controller coming from ConvCD can approximatewith arbitrary accuracy the performance of the actual optimalcontroller. Two nonlinear numerical examples have been usedto show the effectiveness of the method. As ConvCD assumesperfect knowledge of the system dynamics it may becomeinefficient in cases of system uncertainties or variations or,even worse, in cases where minor or major faults or anomaliesaffect the system dynamics. In order for ConvCD to bepractically efficient, an adaptive self-tuning/re-designtool isrequired that will take care of all the above-mentioned factorsthat may affect the efficiency of ConvCD. This will be a topicof future research.

ACKNOWLEDGEMENTS

The research leading to these results has been partiallyfunded by the European Commission FP7-ICT-2013.3.4, Ad-vanced computing, embedded and control systems, undercontract #611538 (LOCAL4GLOBAL).

APPENDIX

PROOF OFLEMMA 1

It is quite straightforward to show similarly to [17]-[19] that– due to (A1) –V (x) is a positive definite function [in factV (x) is nothing but a Control Lyapunov Function (CLF)].SinceV (x) is positive definite andX is a compact subset,we have that there exists a positive constantp∗ such that thefollowing function

W (x) = V (x)− p

dim(C(x))∑

j=1

σ2j (x)

is also positive definite for allx ∈ X and all p ∈ (0,p∗)[please note that by assumption, all constraints are satisfiedfor (χ, u) = (0, 0) and thusσ2

j (0) = 0]. It can be seen –using similar arguments as in [17], [18], [19] and standardfunction approximation arguments – that the functionW (x)can be approximated – with accuracyO(1/L) – by the

function∑L

i=1 βi(x)(x)τ Pi(x) with Pi being symmetricand positive definite. The proof of (13) is established by setting

Pi = Pi +

[

0 00 pI

]

.

PROOF OFTHEOREM 1

Consider the following two optimization problems:

min∥

∥

∥F

P,F, ¯Q+ǫ3Q(x)

∥

∥

∥

2

+ γ (55)

s.t.

ǫ1I P ǫ2I, 0 ¯Q

HP,F(x) ≤ −γ, γ > c > 0, ∀x 6∈ B(ν)and

min

N∑

i=1

‖FP,F, ¯Q+ǫ3Q

(x[i])‖2 (56)

s.t.

ǫ1I P ǫ2I, 0 ¯Q

HP,F(x[i]) ≤ −γ, γ > c > 0, ∀x 6∈ B(ν)

The optimization problems (55) and (56) are equivalent to theoptimization problems (23) and (43), respectively. From nowon, we will consider instead of the optimization problems (23)and (43), their equivalent ones (55) and (56).

Let ϑ denote a vector which contains all non-zero entries ofthe matricesP, ¯Q,F; as the first matrices symmetric block-diagonal, the dimension of the vectorϑ is smaller than thenumber of non-zero entries ofP, ¯Q,F. Similarly, letθ denotea vector which contains all non-zero entries of the matricesP andG. Let alsoϑ∗ andθ∗ denote the global optimizers ofthe infinite-dimensional problems (55) and (18), respectively.Finally, letT denote the one-to-one transformation fromϑ to θdefined according toP = P−1,G = FP−1, Q = P−1QP−1.Then, if (45) holds, the following relation is true

θ∗ = T (ϑ∗) +O(ν) (57)

Using the definition ofϑ we have that

FP,F, ¯Q+ǫ3Q

(x) = ϑτφ(x) + ǫ3z(x)τQz(x) (58)

for some appropriately defined functionφ. Moreover, from(19) we have that

ϑ∗τφ(x) + ǫ3z(x)τQz(x) = O(ν) (59)

Let

Φ =

N∑

i=1

φτ (x[i])φ(x[i])

and

Z = −ǫ3N∑

i=1

φ(x[i])zτ (x[i])Qz(x[i])





12

Then, by using (59) we directly obtain

Φϑ = Z +O(ν) (60)

We will show that the matrixΦ is full-rank with probability1. The proof of the last claim can be established using similararguments as in [30], [18]: ifΦ is not full-rank then thereexists a constant non-zero vectorb such that

bτi Φ(x[i]) = 0, ∀i ∈ 1, . . . ,N (61)

which is equivalent to a set of nonlinear system of equationsin x[i]. The set of solutions to these system of equations haszero Lebesgue measure, see e.g. [30], [18], and as a resultunder a random choice forx[i] the probability that (61) holdsis zero. In other words, a random choice forx[i] guaranteesthatΦ is full-rank with probability 1.

Let ϑ denote the solution of the ConvCD optimizationproblem (56) and

J (ϑ) = ‖Φϑ− Z‖2

Sinceϑ∗ is a feasible solution to the optimization problem(56) and by using (60), we have that

J (ϑ) ≤ J (ϑ∗) = O(ν) (62)

As a result, sinceJ (·) is a quadratic function, we have that

J (ϑ) = J (ϑ∗) + 2(

ϑ∗ − ϑ)τ

Φτ (Φϑ∗ − Z) +(

ϑ∗ − ϑ)τ

ΦτΦ(

ϑ∗ − ϑ)

(63)

Combining (62) and (63) we obtain

0 ≥ 2(

ϑ∗ − ϑ)τ

Φτ (Φϑ∗ − Z) +(

ϑ∗ − ϑ)τ

ΦτΦ(

ϑ∗ − ϑ)

Using the above inequality together with the facts that (a)Φis full-rank and thusΦτΦ ≻ 0 and (b)‖Φϑ∗ − Z‖ = O(ν)we finally obtain that

ϑ = ϑ∗ +O(ν) (64)

that is, the solution of (56) is close cloe to the solution of(55), apart from a term of the order of magnitude of theapproximation errorν. The above equation in combinationwith (57) establishes that

θ∗ = T (ϑ) +O(ν) (65)

Combining (65) with (16) we finally obtain that

GP,G(x) = O(ν) (66)

whereP, G are generated by transforming the solution of theConvCD optimization algorithm (43) [or, its equivalent, (56)].

Besides, from (45), (58) and (66), by definingV (x) =zτ Pz, it possible to verify that

1/ǫ2‖z‖2 ≤ V (x) ≤ 1/ǫ1‖z‖2 (67)˙V (x) ≤ −1/ǫ3Q‖z‖2 +O(ν) (68)

Now, definek1 = 1/ǫ2, k2 = 1/ǫ1, k3 = λmin(Q)/ǫ3, whereλmin(Q) is the smallest eigenvalue of the matrixQ, we obtain[31]

V (x(t)) ≤ exp(−k3k2t)V (x(0)) +O(ν) (69)

‖z(t)‖ ≤√

k2k1

exp

(

− k32k2

t

)

‖z(0)‖+O(ν) (70)

So we proved that forN randomly selected points, Eq. (66)is verified and, moreover, the closed-loop is stable (forLsufficiently large) and converges to a set centered at zero andhaving radius proportional toO(1/L)).

In order to estimate the region of attraction for semiglobalstability, we use the uniformly boundedness results from[31]. Suppose that the setsΩc =

V (x(t)) ≤ c

, Ωǫ =

V (x(t)) ≤ ǫ

, Λ =

c < V (x(t)) ≤ ǫ

, with c > ǫ > 0,are compact. Supposing (68) is valid∀x ∈ Λ, and defining

k = minx∈Λ

−1/ǫ3Q‖z(x)‖2 +O(ν(x))

(71)

with k > 0, we have thanΩc andΩǫ are positively invariant,andx(t) enters the setΩǫ within the interval[0, (c − ǫ)/k].Now, defining

Bµ = x : ‖x‖ ≤ µ , Br = x : ‖x‖ ≤ r (72)

we have

µ = maxµ

Bµ ⊂ Ωǫ , r = maxr

Ωc ⊂ Br (73)

andX0 = x : ‖x‖ ≤ µ , X = x : ‖x‖ ≤ r (74)

Moreover, from the HJB equation we have that the optimalcost-to-goV time-derivative under the optimal state-feedbackv∗ = k(x) satisfies

V =∂V

∂x(f(x) +Bk(x) + ν) = −Π(x) = −zτQz +O(ν)

(75)The above equation together with (68) and the boundedness ofthe closed-loop solutions of the proposed controller establish

that V (t) = V (t) +O(ν), ˙V (t) = V (t) +O(ν) and therefore

x(t) = xopt(t)+O(ν). The rest of the proof can be establishedusing standard function approximation arguments (i.e., that fora O(1/L) term and anyǫ > 0 we have that there exist aLsuch that the magnitude of the termO(1/L) is smaller thanǫ for all L ≥ L) as well as the fact that for any functionf(x) its maximum amplitude can be estimated to be equal tothe maximum value|f(x[i])| along randomly selected pointsx[i], i = 1, . . . ,M, with accuracy that is proportional to thenumberM of random points.

REFERENCES

[1] R. Bellman. Dynamic Programming. Princeton University Press,Princeton, NJ, 1957.

[2] A. J. van der Schaft.l2 -gain analysis of nonlinear systems and nonlinearstate feedbackH∞ control. IEEE Trans. Automat. Contr., 37:770–784,1992.

[3] A. Isidori and W. Kang. H∞ control via measurement feedback forgeneral nonlinear systems.IEEE Trans. Automat. Contr., 40:466–472,1995.





13

[4] T. Basar and P. Bernhard.H∞-Optimal Control and Related MinimaxDesign Problems. Birkhauser, Boston, MA, 1995.

[5] W. M. Lu and J. C. Doyle.H∞ control of nonlinear systems: A convexcharacterization.IEEE Trans. Automat. Contr., 40:1668–1675, 1995.

[6] M. R. James and J. S. Baras. Partial observed differentialgames, infinite-dimensional hamilton-jacobi-isaac equations, and nonlinear H∞ control.SIAM J. Contr. Optimiz., 34:1342–1364, 1996.

[7] H. Chen and F. Allgower. A quasi-infinite horizon nonlinearmodel pre-dictive control scheme with guaranteed stability.Automatica, 14:1205–1217, 1998.

[8] D. Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. M. Scokaert. Con-strained model predictive control: Stability and optimality. Automatica,36:789–814, 2000.

[9] L. Magni, G. De Nicolao, L. Magnani, and R. Scattolini. A stabilizingmodel-based predictive control for nonlinear systems.Automatica,37:1351–1362, 2001.

[10] R. A. Freeman and P. V. Kokotovic. Inverse optimality in robuststabilization.SIAM J. Contr. Optim., 34:1365–1391, 1996.

[11] M. Krstic and Z.-H. Li. Inverse optimal design of input-to-statestabilizing nonlinear controllers.IEEE Trans. Automat. Contr., 43:336–350, 1998.

[12] D. Bertsekas and J. Tsitsiklis.Neuro-Dynamic Programming. AthenaScientific, 1996.

[13] R. Beard, G. Saridis, and J. Wen. Galerkin approximations of thegeneralized hamiltonjacobibellman equation.Automatica, 33:2159–2177, 1997.

[14] R. Beard, G. Saridis, and J. Wen. Approximate solutions to thetime-invariant hamiltonjacobibellman equation.Journal of OptimizationTheory and Application, 96:589–626, 1998.

[15] T. Parisini and R. Zoppoli. Neural approximations for infinite horizonoptimal control of nonlinear stochastic systems.IEEE Transactions onNeural Networks, 9:1388–1408, 1998.

[16] M. Abu-Khalaf and F. L. Lewis. Nearly optimal control laws fornonlinear systems with saturating actuators using a neural network HJBapproach.Automatica, 41:779–791, 2005.

[17] E. B. Kosmatopoulos. CLF-based control design with goodtransientperformance for known and unknown multi-input nonlinear systems.Proceedings of the European Control Conference, Budapest,Hungary,2008.

[18] E. B. Kosmatopoulos. Control of unknown nonlinear systems withefficient transient performance using concurrent exploitation and explo-ration. IEEE Trans. on Neural Networks, 21:1245–1261, 2010.

[19] E. B. Kosmatopoulos. CLF-based control design for unknown multi-input nonlinear systems with good transient performance.IEEE Trans.on Automatic Control, 11:1245–1261, 2010.

[20] A. Rantzer and M. Johansson. Piecewise linear quadratic optimalcontrol. IEEE Trans. on Automatic Control, 45:629–637, 2000.

[21] S. Prajna, A. Papachristodoulou, and F. Wu. Nonlinear control synthesisby sum of squares optimization: A Lyapunov-based approach.Proceed-ings of the Asian Control Conference, 2004.

[22] M. Johansson and A. Rantzer. Computation of piecewise quadraticlyapunov functions for hybrid systems.IEEE Trans. on AutomaticControl, 43:555–559, 1998.

[23] L. Rodrigues. Stability analysis of piecewise-affine systems usingcontrolled invariant sets.Systems & Control Letters, 53:157–169, 2004.

[24] K. M. Passino and S. Yurkovich.Fuzzy Control. Addison WesleyLongman, Menlo Park, CA, 1998.

[25] F. D. Torrisi and A. Bemporad. Hysdel- atool for generating computa-tional hybrid models.IEEE Trans.Control Systems Technology, 43:336–350, 2003.

[26] A. Papachristodoulou and S. Prajna. Robust stability analysis ofnonlinear hybrid systems.IEEE Trans. Automat. Contr., 54:1037–1043,2009.

[27] X.-M. Wu, A. M.-C. So, Z. Li, and Li S.-Y. R. Fast graph laplacian reg-ularized kernel learning via semidefinite-quadratic-linear programming.NIPS2009, 2009.

[28] C.-C. Chien, Y. Zhang, and P. A. Ioannou. Traffic densitycontrol forautomated highway systems.Automatica, 33:1273–1285, 1997.

[29] A. Hegyi, B. De Schutter, and J. Hellendoorn. Optimal coordination ofvariable speed limits to suppress shock waves.IEEE Trans. on IntelligentTransportation Systems, 6:102–112, 2005.

[30] E. B. Kosmatopoulos. Adaptive control design based on adaptiveoptimization principles.IEEE Trans. on Automatic Control, 53:2680–2685, 2008.

[31] H. K. Khalil. Nonlinear Systems. New York: Macmillan, 1992.

PLACEPHOTOHERE

Simone Baldi received the B.Sc., M.Sc. and Ph.D.degree from University of Florence, Italy, in 2005,2007, 2011 respectively. He covered a post-doc re-searcher position at the University of Cyprus, and heis currently a post-doc researcher at the InformationTechnologies Institute (I.T.I.), Centre for Researchand Technology Hellas (CE.R.T.H.). His researchinterests are in the area of control theory and includeadaptive control, switching supervisory control andapproximately optimal control, with applications inenergy efficient buildings and intelligent transporta-

tion systems.

PLACEPHOTOHERE

Iakovos Michailidis graduated from the Departmentof Electrical and Computer Engineering, AristotleUniversity of Thessaloniki in November 2010 andsince February 2011 is a PhD candidate at theDepartment of Electrical and Computer Engineering,Democritus University of Thrace. He has workedon selectivity of protective devices and short cir-cuit analysis (IEC60909) on energy distribution net-works. He is currently conducting research in thearea of model-based and model-free optimal controltechniques for large-scale systems, with applications

in efficient building control and traffic networks.

PLACEPHOTOHERE

Elias B. Kosmatopoulos received the Diploma,M.Sc. and Ph.D. degrees from the Technical Uni-versity of Crete, Greece, in 1990, 1992, and 1995,respectively. He is currently an Associate Professorwith the Department of Electrical and Computer En-gineering, Democritus University of Thrace, Xanthi,Greece. Previously, he was a faculty member of theDepartment of Production Engineering and Manage-ment, Technical University of Crete (TUC), Greece,a Research Assistant Professor with the Departmentof Electrical Engineering- Systems, University of

Southern California (USC) and a Postdoctoral Fellow with the Departmentof Electrical & Computer Engineering, University of Victoria, B.C., Canada.Dr. Kosmatopoulos’ research interests are in the areas of neural networks,adaptive optimization and control, energy efficient buildings and smart gridsand intelligent transportation systems. He is the author of over 40 journalpapers. He has been currently leading many research projectsfunded by theEuropean Union with a total budget of about 10 Million Euros.

PLACEPHOTOHERE

Antonis Papachristodoulou received anM.A./M.Eng. degree in Electrical and InformationSciences from the University of Cambridge,Cambridge, UK, in 2000 and the Ph.D. degreein Control and Dynamical Systems, with a minorin Aeronautics from the California Institute ofTechnology, Pasadena, in 2005. He then held aDavid Crighton Fellowship at the University ofCambridge and a postdoctoral research associateposition at the California Institute of Technologybefore joining the Department of Engineering

Science at the University of Oxford, Oxford, UK, in January 2006, wherehe is now an Associate Professor in Engineering Science and TutorialFellow at Worcester College. His research interests include scalable analysisof nonlinear systems using convex optimization based on sum ofsquaresprogramming, analysis and design of large-scale networked control systemswith communication constraints, and systems and synthetic biology.





14

PLACEPHOTOHERE

Petros A. Ioannou (S’80-M’83-SM’89-F’94) re-ceived the B.Sc. degree with First Class Honors fromUniversity College, London, England, in 1978 andthe M.S. and Ph.D. degrees from the University ofIllinois, Urbana, Illinois, in 1980 and 1982, respec-tively. In 1982, he joined the Department of Elec-trical Engineering-Systems, University of SouthernCalifornia, Los Angeles, California. He is currentlya Professor in the same Department and the Directorof the Center of Advanced Transportation Technolo-gies. He also holds courtesy appointments with the

Departments of Aerospace and Mechanical Engineering and Industrial andSystems Engineering. He is the Associate Director for Research for theUniversity Transportation Center METRANS at the University of SouthernCalifornia. He was visiting Professor at the University of Newcastle, Australiaand the Australian National University in Canberra during parts of Fall of1988, the Technical University of Crete in summer of 1992 and Fall of2001 and served as the Dean of the School of Pure and Applied Scienceat the University of Cyprus in 1995. In 2009 he was with the Department ofElectrical Engineering and Information Technologies of theCyprus Universityof Technology while on sabbatical leave from the Universityof Southern Cal-ifornia. Dr. Ioannou was the recipient of the Outstanding Transactions PaperAward by the IEEE Control System Society in 1984 and the recipient of a 1985Presidential Young Investigator Award for his research in Adaptive Control. In2009 he received the IEEE ITSS Outstanding ITS Application Award and the2009 IET Heaviside Medal for Achievement in Control by the Institution ofEngineering and Technology (former IEE). In 2012 he receivedthe IEEE ITSSOutstanding ITS Research Award. He has been an Associate Editor for theIEEE Transactions on Automatic Control, the International Journal of Control,Automatica and IEEE Transactions on Intelligent Transportation Systems. Heis a member of the Board of Governors of the IEEE Intelligent TransportationSociety. Dr. Ioannou is a Fellow of IEEE, Fellow of International Federationof Automatic Control (IFAC), Fellow of the Institution of Engineering andTechnology (IET), and the author/co-author of 8 books and over 200 researchpapers in the area of controls, vehicle dynamics, neural networks, nonlineardynamical systems and intelligent transportation systems.



Documents

Convex Design Control for Practical Nonlinear Systems