Geometry of adaptive control: optimization and geodesics

INTERNATIONAL JOURNAL OF ADAPTIVE CONTROL AND SIGNAL PROCESSINGInt. J. Adapt. Control Signal Process. 2004; 18:381–392 (DOI: 10.1002/acs.801)

Geometry of adaptive control: optimization and geodesics

Diego Col !oon1,y and Felipe M. Pait2,z,n

1Universidade de S *aao Paulo, Laborat !oorio de Automa-c *aao e Controle–PTC, Av. Prof. Gualberto trav 3–158, S *aao Paulo,

SP 05508–900, Brazil2Universidade de S *aao Paulo, Laborat !oorio de Automac *aao e Controle–PTC, Brazil

Also at Alphatec, Inc., Burlington, Massachusetts, USA

SUMMARY

Two incompatible topologies appear in the study of adaptive systems: the graph topology in controldesign, and the coefficient topology in system identification. Their incompatibility is manifest in thestabilization problem of adaptive control. We argue that this problem can be approached by changing thegeometry of the sets of control systems under consideration: estimating np parameters in an np-dimensionalmanifold whose points all correspond to stabilizable systems. One way to construct the manifold is usingthe properties of the algebraic Riccati equation. Parameter estimation can be approached as an optimalcontrol problem akin to the deterministic Kalman filter, leading to algorithms that can be used inconjunction with standard observers and controllers to construct stable adaptive systems. Copyright #2004 John Wiley & Sons, Ltd.

KEY WORDS: adaptive control; Riccati equation; system identification; topology and geometry of controlsystems

1. TOPOLOGIES IN ADAPTIVE CONTROL

Two topologies appear in the study of adaptive systems. Relevant for feedback control is thegraph topology, induced by both the gap and graph metrics; it is the coarsest topology on sets oflinear systems for which feedback stability is a robust property [1, 2]. System identification, onthe other hand, makes implicit use of the topology induced by the metric in which the distancebetween two systems is given by the Euclidean distance between the coefficients of their transferfunctions. Even on sets of linear systems with dimension not exceeding a given n; on which bothare defined, these topologies are not compatible: one is neither finer nor coarser than the other,

Contract/grant sponsor: FAPESP; contract/grant number 97/04668-1, 99/6591-8

Contract/grant sponsor: CNPq; contract/grant number 300932/97-9

Received 15 March 2003Revised 7 September 2003Accepted 17 October 2003Copyright # 2004 John Wiley & Sons, Ltd.

yE-mail: [email protected]

nCorrespondence to: F. Pait, Universidade de S*aao Paulo, Laborat !oorio de Automac*aao e Controle–PTC, Av. Prof.Gualberto trav 3–158, S*aao Paulo, SP 05508–900, Brazil.

zE-mail: [email protected]

that is, a set open in the graph topology may not be an open set in the coefficient topology, andvice versa.

Adaptive controllers are characterized by a double feedback loop: the controland the adaptation loops. The incompatibility between the topologies underlying thedesign of the loops manifests itself in the form of the stabilization problem. In fact,the parameter values for which the design model, upon which certainty-equivalencecontrol laws are designed, loses stabilizability, are exactly those for which the opera-tions of addition and multiplication of transfer functions are discontinuous under the graphtopology.

Myriad adaptive algorithms in the literature start by designing certainty-equiv-alence controllers, and jury-rig alternative feedback signals to be used when the certainty-equivalence control laws approach a singularity (see for instance Reference [3] and itsreferences). A cogent argument for this approach is the realization that certainty equivalenceimplies stabilizability, and when the former fails the latter can be achieved by other methods (seeReference [4], where an analysis that can be applied to the type of systems discussed here iscarried out). This is generally effected via logic-based hybrid control or time-varying feedback;but although switched controllers might be desirable for reasons of performance, there is noclear indication that they are necessary for stabilization. A second way to deal with thisincompatibility is to develop alternative parametrizations for sets of linear systems, in order toexclude the singular points}those corresponding to systems that are not stabilizable. Amongthe few references to this idea in the literature are References [5–7]. Unfortunately, the resultingparameter sets often do not have convexity properties needed for the use of conventionalestimation techniques.

The approach to system identification we advocate is to change the geometry of the tunedparameter set, and is based on the observation that the set of stabilizable systems canbe identified with the set of matrix pairs for which the algebraic Riccati equation has a posi-tive-definite solution. Rather than estimating np parameters in Rnp (or some subset thereof), wecan tune them in an np-dimensional manifold comprising a hypersurface in a space that includesthe terms of solution to the matrix Riccati equation, as well as the usual design modelparameters.

An application of these ideas to the control of simple, one-dimensional siso systems has beenpresented in a paper coauthored by Piccoli [8]. The approach taken therein was to add a penaltyterm to the goal function, thus driving the identification algorithm away from pointscorresponding to singularities in the certainty-equivalence control laws, and may be comparedto those of References [9–12]. Here, the idea of posing the identification problem in a non-Euclidean manifold is generalized to higher-dimensional siso processes using techniques thatdelve deeper into the geometry of the manifold, and do not impose the need for a prioriknowledge of system parameters in order to obtain stabilizability of the design model over thewhole parameter set.

In Section 2 we set up the framework of the adaptive control problem that we wish to solve.Relevant geometric properties of the Riemannian manifold whose points correspond tostabilizable design models are studied in Section 3. Posing parameter estimation on thismanifold as an optimal control problem akin to the Kalman filter permits the developmentof smoothing and filtering algorithms, presented in Section 4. Some facts from optimalcontrol gather informally in Appendix A. A previous version of this paper was presented inReference [13].

Copyright # 2004 John Wiley & Sons, Ltd. Int. J. Adapt. Control Signal Process. 2004; 18:381–392

D. COLON AND F. M. PAIT382

2. FRAMEWORK

We are concerned with designing an adaptive controller based on the siso design model,

’xxDðtÞ ¼ ðAþ dcÞxDðtÞ þ buDðtÞ

yDðtÞ ¼ cxDðtÞðSDÞ

Here xD 2 Rn; uD; yD 2 R; the matrix pair ðc;AÞ is observable and fixed with A stable, andd; b 2 Rn are design model parameters. The transfer function of SD is

cðsI � A� dcÞ�1b ¼cðsI � AÞ�1b

1� cðsI � AÞ�1dð1Þ

a fact that can be verified either by direct matrix manipulation or by rewriting SD as ’xxD ¼AxD þ buD þ dyD: Because of observability there is enough freedom to assign the poles of SD viasuitable choice of d; and the zeroes via choice of b; moreover the pole-zero cancellations thatcorrespond to eigenvalues of A in (1) are stable, therefore SD is adequate for designing adaptivecontrollers for processes about which there is a considerable amount of uncertainty, providedwe have reason to believe that they can be effectively controlled using a design based on an n-dimensional model [4]. The existence of values of b; d for which ðAþ dc; bÞ is not stabilizable,resulting in unstable cancellations in (1), is the origin of the stabilization problem we wish toavoid altogether. We shall defer making specific assumptions about the process itself; suffice tosay informally that we are concerned with classes of processes that could be controlled using adesign based upon SD; if we just knew the parameters b and d:

Let T ðpÞ be the invertible change of state variable matrix such that ðcT�1; TAT�1; TpÞ ¼ðpT;AT; cTÞ and construct the system

’xx ¼AIxþ bIuþ dIy

#xx ¼ ½T ðdÞ T ðbÞ�x

e ¼ ½dT bT�x� y

ðSI Þ

with

AI ¼AT 0

0 AT

" #; dI ¼

cT

0

" #and bI ¼

0

cT

" #:

The transfer function of ðSIÞ from yu

� �to ½dT bT�x is the same as the transfer function of SD from

yDuD

h ito yD; in fact if we were to carefully set SD’s input y to equal its output ½dT bT�x we would

obtain precisely SD’s transfer function. Because of the preceding, we are justified in calling SI anadaptive observer or identifier, appropriate for use in conjunction with design model SD: In thisapplication u and y will be set to equal the input and the output respectively of a controlledprocess. The 2n-vector-valued signal x may be called a regressor since it appears as a set ofcoefficients to the parameters in an affine error equation.

In view of the above, an appropriate regulator to go with SI could take the form:

uR ¼ fRðb; dÞ #xx ðSRÞ


GEOMETRY OF ADAPTIVE CONTROL 383

Together SI and SR form a parameterized controller, and fRðb; dÞ is chosen so that, were itconnected to SD as explained above and were uD to be set equal to feedback control signal uR;the resulting system would be internally stable.

To construct the remaining ingredients of a parameter adaptive control system, namely anexpression for parameterized feedback fRðb; dÞ and a tuner, usual steps involve borrowing alinear control design and estimation algorithm, using the latter to tune estimates ðd; bÞ on R2n;and finally combining both via some sort of certainty equivalence. This brings problems in that,unless restrictive hypotheses are made, the parameter space ends up including points for whichSD loses stabilizability. At such points the equations defining f are sure to hit a singularity.Rather than tackle the stabilizability issue with modifications on standard tuners or feedbackcontrol designs, we question the assumption that the Euclidean space is the correct set on whichto estimate parameters. The following section discusses what the geometry of an appropriate setmight look like.

3. GEOMETRY OF THE RICCATI MANIFOLD

Perhaps the most transparent, general purpose control design paradigm, which may be appliedto any stabilizable, detectable linear system of a known dimension, is linear-quadratic optimalcontrol. In fact, if b and d are such that SD meets those conditions}and the values ofparameters for which it does not are exactly those responsible for the loss of stabilizabilityproblem in adaptive control}then there exists a symmetric, positive definite solution P to thealgebraic Riccati equation:

ðAþ dcÞTP þ P ðAþ dcÞ � Pbr�1bTP þ Q ¼ 0 ð2Þ

Here r > 0 and Q > 0 are arbitrary design parameters. Their numerical values are of littleconsequence for the points we wish to make, but one could conceive of choosing them based onsome quadratic objective. Consider that the set of points satisfying (2) defines a 2n-dimensionalmanifold as a subset of Rnþnþn2 ; uniquely identified by the requirement that P ¼ PT > 0: That itis a differentiable manifold is an immediate consequence of the results of Reference [14]; we shalldenote it Ricc: Figure 1 depicts the two-dimensional Ricc as the graph of P as a function of band d: Let y be a parametrization of the manifold, in the sense that yðd; b; P Þ is smooth andtogether with (2) forms a smooth bijection. The domain of the parametrization is fy 2 R2n :ðAþ dðyÞc; bðyÞÞ stabilizableg; we shall refer to points of R2n outside this domain as thesingularity. We may take y ¼ d

b

� �; which is in keeping with the idea of indirect adaptive control,

and shall indeed do so in this paper, but not before remarking that other parametrizations maybe of interest for alternative adaptive control designs. The chief advantages of thisparametrization are that the expression for the error e is linear in y; and that the mappingfrom Ricc onto the set of values of y is a global chart, that is, a chart whose domain is all ofRicc: Notice also that (2) together with

f ¼ �r�1bTP ð3Þ

can be viewed as one among many possible choices of feedback controls fR in SI:The main goal of this section is to characterize Ricc as a Riemannian manifold by means of

its metric, which can be written as a matrix GðyÞ or alternatively as a second-rank tensor withelements gij: We shall consider a metric on Ricc induced by the ‘natural’ inner product on



ðd; b; P Þ-space, namely the Euclidean space Rnþnþn2 ; so that

~yyTG~yy ¼ ~dd

T~dd þ~bbT~bbþ trace~PP

T~PP

for vectors ~yy; ~dd; ~bb and ~PP in the tangent spaces corresponding to the variables y; d; b and P ;respectively. The reader who is comfortable with the definition of Ricc as a manifold endowedwith an atlas and a metric tensor might want to take for granted the computations that follow,which are not needed in the remaining sections of the paper, and fast-forward to the discussionof estimation algorithms in Section 4.

To proceed with the computations, let us rewrite the equations that define the manifold usingindex notation:

ðAji þ djciÞPjk þ PijðAjk þ djckÞ � Pijbjr�1blPlk þ Qik ¼ 0 ð4Þ

In this expression, we dropped the summation signs, a common act of laziness referred to as theEinstein summation convention: any repeated index is assumed to be summed over. So, the onlyfree indices in (4) are i and k: Any other indices are summed over, from 1 to n: To obtain partialderivatives of (4) we employ the rules

@xi@x‘

¼ di‘;@xij@x‘m

¼ di‘djm

where dij is the Kronecker delta, 1 when i ¼ j; 0 otherwise. We also adopt the convention thatthe derivative of a scalar with respect to a column (row) vector is a column (respectively row)vector. An index preceded by comma denotes differentiation. In this section all indices are keptin subscript because there is no need to distinguish between covariant and contravariantquantities.

Figure 1. The two-dimensional Ricc manifold.



For concreteness we now derive an expression in the co-ordinates y for the terms of themetric:

gmn ¼@di@ym

@di@yn

þ@bi@ym

@bi@yn

þ@Pij@ym

@Pij@yn

For the parametrization y ¼ db

� �gmn ¼ dmn þ

@Pij=@dm

@Pij=@bm

" #� ½@Pij=@dm @Pij=@bm� ð5Þ

Now compute the partials with respect to dm in (4)

djmciPjk þ ðAji þ djciÞ@Pjk@dm

þ@Pij@dm

ðAjk þ djckÞ

þ Pijdjmck �@Pij@dm

bjr�1blPlk � Pijbjr�1bl@Plk@dm

¼ 0

from which follows, using fi ¼ �Pilblr�1 from definition (3),

ðAji þ djci þ bjfiÞ@Pjk@dm

þ@Pij@dm

ðAjk þ djck þ bjfkÞ þ ciPmk þ Pimck ¼ 0 ð6Þ

Performing analogous calculations with respect to bm gives

ðAji þ djci þ bjfiÞ@Pjk@bm

þ@Pij@bm

ðAjk þ djck þ bjfkÞ þ fiPkm þ Pimfk ¼ 0 ð7Þ

Equations (6) and (7) above can be made a tad more explicit using, for instance, Kroneckerproducts. All that matters here is that they are non-singular linear equations, because alleigenvalues of Aþ dcþ bf are negative, so each has a unique solution. Thus the partials of Pwith respect to d and b are well determined, which in turn leaves expression (5) for the metricuniquely defined for each value of the parameter y:

Once a metric has been chosen it alone suffices to define Ricc as a 2n-dimensionalRiemannian manifold, independently of the original embedding in R2nþn2 which motivated themetric’s definition. This intrinsic point of view is the one taken in the sequel. The salient featureof Ricc is that points for which SD loses stabilizability are ‘at infinity’, that is, GðyÞ becomesunbounded so that any path on Ricc that tends towards the singularities of a certainty-equivalence feedback has infinite length. This happens because at the singularities the solutionto the Riccati equation tends to infinity, as illustrated in Figure 1, and so does @P=@y:

4. ESTIMATION ALGORITHMS

4.1. An optimal control problem

In order to develop algorithms for parameter estimation on Ricc; we consider the followingoptimal control problem with initial cost: given an interval ½s; t�; minimize

ðyðsÞ � y0ÞTSðyðsÞ � y0Þ þ

Z t

sð’yyTGðyÞ’yyþ ðxTy� yÞTQðxTy� yÞÞ dt ð8Þ



Here yðtÞ describes a curve on Ricc parametrized by t 2 ½s; t�; GðyÞ is the matrix expression ofthe metric studied in Section 3, yðtÞ 2 Rny and xðtÞ 2 R2n�ny are, respectively, a vector of data andthe regressor as explained in Section 2, and y0 is some 2n-vector. Matrices S and Q are positivedefinite design parameters of dimensions 2n� 2n and ny � ny ; respectively. We have consideredthe measurement y to be vector-valued since there is no extra difficulty in doing so; to considerthe single-output case simply set ny ¼ 1 so that y and Q are scalars.

If only the first parcel inside the integral were present, and initial and final conditions wereimposed on y; the problem would become one of minimizing a measure of the length of theparametrized curve}and in fact its solution would be the geodesic on Ricc connecting yðsÞ andyðtÞ (such a curve exists because Ricc is geodesically complete). The parcel weighting theidentification error is more familiar in the least-squares estimation literature, and the parceloutside the integral, which weights the deviation of initial condition yðsÞ from some a prioriguess y0; serves to regularize the problem.

Both to guide the search for a solution and to further motivate the formulation of theoptimization problem, it is useful to pose it as an equivalent filtering problem: minimize

J ðs; t; yÞ ¼ ðyðsÞ � y0ÞTSðyðsÞ � y0Þ þ

Z t

sðwTGðyÞwþ eTQeÞ dt ð9Þ

subject to

’yyðtÞ ¼ wðtÞ

yðtÞ ¼ xTðtÞyðtÞ � eðtÞð10Þ

This problem is a particular case of the deterministic Kalman filter (see References [15, 16]),with the complication that the weighting of the input depends on the state y: An interpretation isthat we search for y which best explains a linear relationship between data y and x; in the sensethat w; e; and yðsÞ are minimized according to functional (9). An amount of parameter drift thatwould be small (in terms of its effect on feedback design stability) for points far from thesingularity rapidly becomes unacceptable if it leads y towards values which correspond to non-stabilizable systems, thus we are less inclined to take into account data that points in such adirection. This provides a motivation, without recourse to geometry, for the choice of a y-dependent input weighting matrix.

In a kind of least-squares estimator often employed in the adaptive control literature, thematrix G is altogether absent, y being considered fixed. Such estimators have the dubiousadvantage of converging even in the absence of persistent excitation, however their ability totrack parameter changes such as those caused by model changes or process faults is poor, a factthat is often dealt with by a number of ad hoc modifications. Explicitly introducing a weightingon parameter drift, which is discussed for instance in Reference [17], might be preferable in itsown right, besides opening the possibility of introducing geometric considerations.

4.2. Smoothing algorithm

We now develop a recursive solution to the equivalent problems (8) and (9): first, we considersome fixed value for yðsÞ; and second, we further optimize with respect to all possibletrajectories of y: Following Appendix A, form the Hamiltonian

Hðt; y;w; lÞ ¼ wTGðyÞwþ eTQe� lw



whose minimum value is attained for

w ¼ 12G�1ðyÞlT ð11Þ

The differential equation for the row vector of Lagrange multipliers will involve derivatives ofthe matrix G; so we revert to index notation:

’lll ¼@H

@yl¼

@

@ylðwiGijwj þ eiQijej � liwiÞ ¼ wiGij;lwj þ 2eiQijxlj ð12Þ

Here Gij;l denotes @Gij=@yl: We represent the components of y and w with upper indices to

conform to a useful convention of tensor calculus. Together with the choice of some initialcondition yðsÞ; Equations (10)–(12) solve the smoothing problem, that is, they express analgorithm for estimation of the parameter y in the interval ½s; t�:

Existence and uniqueness of solutions to differential Equations (10)–(12) on the interval ½s; t�is not a question provided that G remains non-singular, its derivative bounded, and that ðx; yÞare bounded. Non-singularity follows from the definition of G and boundedness of Gij;l followsfrom differentiability, which implies boundedness in any compact subset, together with the factthat any trajectory for which G becomes unbounded must be on infinite length and thusincompatible with a finite J :

An alternative characterization of the smoothing algorithm can be obtained rewriting (11) as

Gkjwj þ wiGik ¼ lk

and then taking derivatives with respect to t and identifying w with ’yy:

Gkj;l’yyl ’yyj þ Gik;l

’yyl ’yyi þ Gkj.yyj þ Gik

.yyi ¼ ’yyiGij;k’yyj þ 2ðxQeÞk

Using symmetry of G results

2Gkm.yym � Gij;k

’yyi ’yyj þ Gkj;i’yyi ’yyj þ Gik;j

’yyi ’yyj ¼ 2ðxQeÞk

or finally

.yym þ Gmij’yyi ’yyj ¼ ðGkmÞ

�1ðxQeÞk ð13Þ

Here Gmij ¼

12G�1

km ðGkj;i þ Gik;j � Gij;kÞ are known as the Christoffel symbols and the right-handside is the expression of the contravariant gradient of e2=2 on Ricc: When e is zero, thesecond-order differential equation (13) is nothing but the expression of a geodesic on themanifold of metric G: The relationship of form (13) of the smoothing algorithm with the second-order tuner discussed in Reference [18] might be worth exploring. In particular, it would beinteresting to search for ways to introduce damping and to construct a stable algorithm.

4.3. Filtering algorithm

While the smoothing algorithm defines a recursive procedure for constructing parameterestimates, one thing that is still lacking is to optimize with respect to all trajectories. This is morein keeping with the spirit of Kalman filtering and has advantages, not least that using thesmoothing algorithm as derived in Section 4.2 requires solving the differential equation (13) onthe interval ½s; t� anew for each new value of y: To accomplish the minimization, notice thatonce we have decided to use the optimizing law given by (11) and (12), the final condition yðtÞbiunivocally identifies the initial value yðsÞ; therefore, in order to optimize with respect to initialconditions one can alternatively minimize the optimal accumulated cost V ðt; yÞ with respect to y:



The value #yy which minimizes the accumulated cost at time t must satisfy

@V@y

ðt; #yyÞ ¼ 0 ð14Þ

Along the trajectory which leads to #yy at time t; wðtÞ ¼ 12G�1ð#yyÞð@V =@yÞðt; #yyÞT ¼ 0; an

observation that simplifies the calculations that follow. Although (14) does not provide anexplicit formula for #yy; it serves to obtain a recursive expression as follows. Considering #yy as afunction of t and differentiating gives

d

dt@V@y

ðt; #yyÞ ¼@2V@t@y

ðt; #yyÞ þ ’#yy#yyT@2V

@yT@yðt; #yyÞ ¼ 0 ð15Þ

To solve this equation for #yy; first compute

@2V@t@y

ðt; #yyÞ ¼@2V@y@t

ðt; #yyÞ ¼@

@yðwTGðyÞwþ eQeÞ

��y¼#yy

¼ 2xQeðt; #yyÞ

Second define Cðt; #yyÞ ¼ @2V =@yT@yðt; #yyÞ and use (A5) from Appendix A to write

’CC ¼@2H

@yT@y�

@w@y

� �@2H

@wT@w@w@y

� �T

¼ wT @2GðyÞ

@yT@ywþ 2xQxT � 2

@w@y

GðyÞ@w

@yTð16Þ

Here indices were dropped for readability, wT@2GðyÞ@yT@yw and @w=@y being understood as2n� 2n matrices. From 2Gijwj ¼ li follows:

2Gik;swk þ 2Gij@wi

@ys¼

@li@ys

¼ Cis

Substituting into (16) gives

’CC ¼ 2xQxT � 12CG�1C

Inversion of matrix C can be avoided defining P ¼ 2C�1 so that ’PP ¼ �2C�1 ’CCC�1 ¼ 12P ’CCP;

and solving (15) for y results in the following.Filtering algorithm: Set parameter estimate #yy according to

’#yy#yyðtÞ ¼ �PxQeðt; #yyÞ

’PPðtÞ ¼ G�1ð#yyÞ �PxQxTPð17Þ

with PðsÞ ¼ S; initial constraint 2Gð#yyðsÞÞ ’#yy#yyðsÞ ¼ @V ðs; #yyðsÞÞ=@yT ¼ 2SðyðsÞ � y0Þ; and final

condition’#yy#yyðtÞ ¼ 0:

Unfortunately (17) still does not offer a complete practical solution to the filtering problem.The reason is that the differential equation for P is valid along each optimal trajectory, so oncewe perform an optimization with respect to the set of trajectories P needs to be recomputed.One way to deal with this problem is to use numerical methods to solve (17). The amount ofcomputation needed, however, would increase with time as the length of the interval underconsideration grows. Another way is to write down the full partial differential equation for Cderived from (A5), as spelled out in Appendix A. This equation may however be very hard tosolve. Yet another approach, currently being pursued, is to write down a covariant expressionfor the second derivative of the value function, that is to say, to reproduce the development inAppendix A in a manner intrinsic to the manifold on which variables are defined.



APPENDIX A: SOME FACTS FROM OPTIMAL CONTROL THEORY

Here some facts from optimal control gather informally. Sufficient smoothness is assumed sothat all relevant derivatives and their dogs exist and are continuous. All variables in thissection are local in scope, that is, their definition here has no impact on their use elsewhere in thepaper.

Consider the initial-cost problem of minimizing the functional

J ðs; t; x; uÞ ¼ pðxðsÞÞ þZ t

sqðt; x; uÞ dt

subject to the differential equation

’xxðtÞ ¼ f ðt; x; uÞ

The optimal accumulated cost (or Bellman value function) V ðt; xÞ must satisfy

V ðs; xÞ ¼ pðxðsÞÞ þZ s

sqðt; x; u

*Þ dt

for the optimal control u*ðtÞ; so that

’VVðt; xÞ ¼ qðt; x; u*Þ

Since V is a function of t and x only, along the optimal trajectory

@tV þ ð@xV Þf ðt; x; u*Þ ¼ qðt; x; u

*Þ ðA1Þ

the Hamilton–Jacobi–Bellman equation for the initial cost problem. Define the appropriateHamiltonian

Hðt; x; u; lÞ ¼ qðt; x; uÞ � lf ðt; x; uÞ

The optimal control is that which minimizes H with l ¼ @xV ðt; xÞ: Now take partial derivativeswith respect to x in (A1):

@2V@xj@t

þ@2V@xj@xi

fi þ@V@xi

@fi@xj

þ@V@xi

@fi@uk

@uk@xj

¼@q@xj

þ@q@uk

@uk@xj

Inverting the order of derivatives and identifying lj ¼ @V =@xj gives

@

@tlj þ

@

@xilj

� �fi þ li

@fi@xj

¼@q@xj

þ@q@uk

� li@fi@uk

� �@uk@xj

or

’llþ l@xf ðt; x; u*Þ ¼ @xqðt; x; u*

Þ ðA2Þ

To obtain the equation above we used the fact that the optimal u*must satisfy

@H

@u¼

@q@u

� l@f@u

¼ 0 ðA3Þ

which in fact together with (A2) is an expression of Pontryagin’s maximum principle for theproblem under consideration. One way of formally proving that (A1) is sufficient, and (A2) isnecessary, for a control to be optimal would be to reverse time, transforming the problem into afinal-cost problem, and applying the usual optimality principles.



We shall also have the occasion to use a recursion on Cðt; xÞ ¼ @2V =@xT@x that can beobtained by further computing derivatives with respect to x in (A2):

@2lj@xk@t

þ@2lj@xk@xi

fi þ@lj@xi

@fi@xk

þ@fi@un

@un@xk

� �

þ@li@xk

@fi@xj

þ li@2fi@xk@xj

þ@2fi@xj@un

@un@xk

� �¼

@2q@xk@xj

þ@2q

@xj@un

@un@xk

So

@Cjk

@tþ

@Cjk

@xifi þCji

@fi@xk

þ@fi@xj

Cik

¼@2q

@xj@xk� li

@2fi@xj@xk

þ@2q

@xj@un� li

@2fi@xj@un

�@lj@xi

@fi@un

� �@un@xk

But taking derivatives with respect to xj in (A3) gives

@2q@xj@um

þ@2q

@um@un

@um@xj

� li@2fi@xj@un

þ@2fi

@um@un

@um@xj

� ��

@li@xj

@fi@un

¼ 0

thus

@2q@xj@un

� li@2fi@xj@un

�@li@xj

@fi@un

¼ li@2fi

@um@un�

@2q@um@un

� �@um@xj

¼ �@2H

@um@un

@um@xj

ðA4Þ

Hence we can write

’CCjk þCji@fi@xk

þ@fi@xj

Cik ¼@2H

@xj@xk�

@um@xj

@2H

@um@un

@un@xk

ðA5Þ

which is the expression we wished to obtain. For instance, in the well-known linear-quadraticcase, where f ¼ Axþ Bu and q ¼ uTRuþ xTQx; from (A3) results u ¼ R�1BTlT=2; thus @u=@x ¼R�1BTC=2 and (A5) reduces to

’CC ¼ �ATC�CAþ 2Q� 12CBR�1BTC

which becomes the usual Kalman filtering Riccati differential equation when we substitute@l=@x ¼ C ¼ 2P�1:

Recall, however, that ’CCjk is just a shorthand notation for @Cjk=@t þ ð@Cjk=@xiÞfi; therefore(A5) can be interpreted as an ordinary differential equation along an optimal trajectory only.Along an arbitrary trajectory, (A5) has to be interpreted as a partial differential equationinvolving the derivatives of C with respect to t and x:

ACKNOWLEDGEMENTS

The second author is indebted to B. Piccoli, currently of iac-Roma, for collaboration during visits toSalerno and Trieste; to D. Liberzon of the University of Illinois and M. Krichman of Alphatech, Inc. foruseful discussions, and to W. Truppel of the University of California, Riverside, for inestimable help withtechniques employed in this paper.Work partially funded by fapesp}State of S*aao Paulo Research Council, grant 97/04668-1. The first

author held fapesp doctoral scholarship 99/05915-8. The second author received grant 300932/97-9 fromCNPq}Brazilian Research Council, and the generous support of Susanna V Stern.



REFERENCES

1. El-Sakkary AK. The gap metric: robustness of stabilization of feedback systems. IEEE Transactions on AutomaticControl 1985; 30(3):240–247.

2. Vidyasagar M. Control System Synthesis}A Factorization Approach. MIT Press: Cambridge, MA, 1985.3. Mareels I, Polderman JW. Adaptive Systems: An Introduction. Birkh.aauser: Boston, 1996.4. Pait FM, Morse AS. A cyclic switching strategy for parameter adaptive control. IEEE Transactions on Automatic

Control 1994; 39(6):1172–1183.5. Ghosh BK, Dayawansa WP. A hybrid parametrization of linear single input single output systems. Systems &

Control Letters, 1987; 8:231–239.6. Llanos Villareal ER. Geometria de Conjuntos de Sistemas Lineares. Master’s Thesis, Universidade de S*aao Paulo,

S*aao Paulo: Brazil, 1997 (in Portuguese).7. Pait FM, Morse AS. A smoothly parametrized family of stabilizable observable linear systems containing

realizations of all transfer functions of McMillan degree not exceeding n. IEEE Transactions on Automatic Control1991; 36(12):1475–1477.

8. Pait FM, Piccoli B. Geometry of adaptive control. In the European Control Conference, Porto, Portugal, September2001.

9. Kreisselmeier G. A robust indirect adaptive control approach. International Journal of Control, 1986; 43:161–175.10. Ossman KA, Kamen ED. Adaptive regulation of mimo linear discrete-time systems without requiring a persistent

excitation. IEEE Transactions on Automatic Control 1987; 32:397–404.11. Prandini M, Campi MC. A new recursive identification algorithm for singularity free adaptive control. Systems &

Control Letters 1998; 34(4):177–183.12. Prandini M, Campi MC, Bittanti S. A penalized identification criterion for securing controllability in adaptive

control. Journal of Mathematical Systems, Estimation, and Control 1998; 8:491–494.13. Pait FM. Geometry of adaptive control, part II: optimization and geodesics. Fifteenth International Symposium on

Mathematical Theory of Networks and Systems, University of Notre Dame, South Bend, IN, 2002.14. Delchamps DF. Analytic feedback control and the algebraic Riccati equation. IEEE Transactions on Automatic

Control 1984; 29(11):1031–1033.15. Sontag ED.Mathematical Control Theory: Deterministic Finite Dimensional Systems (2nd edn). Springer: New York,

1998.16. Willems JC. Deterministic Kalman filtering. Fifteenth International Symposium on Mathematical Theory of Networks

and Systems, University of Notre Dame, South Bend, IN, 2002.17. Ljung L. System Identification}Theory for the User. (2nd edn.) Prentice-Hall: Upper Saddle River, NJ, 1999.18. Pait FM. A tuner that accelerates parameters. Systems & Control Letters 1998; 35(1):65–68.



Documents

Geometry of adaptive control: optimization and geodesics