Algebraic Riccati Equation - IIT Kharagpurbibhas/ARE.pdfIntroductionAbout Riccati and his equation Non-symmetric Algebraic Riccati Equation (NARE) Continuous time ARE (CARE)Numerical

Introduction About Riccati and his equation Non-symmetric Algebraic Riccati Equation (NARE) Continuous time ARE (CARE) Numerical methods for solving ARE

Algebraic Riccati Equation

Bibhas AdhikariIIT Kharagpur, India

GIAN course on Singular Optimal Control, 2016


Outline

1 IntroductionThe Model and the ProblemReferences

2 About Riccati and his equation

3 Non-symmetric Algebraic Riccati Equation (NARE)Critical solutions

4 Continuous time ARE (CARE)Critical solutions

5 Numerical methods for solving ARE


References

1 Dario A. Bini, Bruno Iannazzo and Beatrice Meini,Numerical Solution of Algebraic Riccati Equations, SIAM,2012.

2 Sergio Bittanti, Alan J. Laub and Jan C.Willems (Eds.) TheRiccati Equation, Springer, 1991.

3 Peter Lancaster and Leiba Rodman, Algebraic RiccatiEquations, Oxford University Press, 1995.

4 James F. Bellon, Riccati Equations in Optimal ControlTheory, Masters Thesis, Georgia State University, 2008.

5 W. T. Reid, Riccati Differential Equations, in volume 86 ofMathematics in Science and Engineering, AcademicPress, New York, 1972.

6 Israel Gohberg, Peter Lancaster and Leiba Rodman,Invariant Subspaces of Matrices with Applications, SIAM,2006.


About Riccati and his equation

C + XA + DX − XBX = 0

Special case: A,B,C,D are real/complex numbers!!


Count Jacopo Riccati was born in Venice on May 28, 1676.His father died when he was only ten years old.In 1693, he enrolled at the University of Padua as astudent of law.Stefano was fond of Isaac Newton’s Philosophiae NaturalisPrincipia Mathematica, which he passed on to youngRiccati around 1695.After graduating on June 7, 1696, he married Elisabetta deiConti d’Onigo on October 15, 1696. She bore him 18children, of whom 9 survived childhood.


He did not follow any lecture courses in mathematics.Basically, the profound knowledge of the self-taught manwas acquired by reading.Riccati had far-reaching interests, ranging frommathematics to poetry, from physics to religion, aswitnessed by his works and his rich library.He also believed that the brain should be better exercisedin a variety of fields. As he wrote “Since adolescence, themind should be educated to treasure the most eminent ofsciences and the finest of arts. I do not want to claim thatevery topic should be probed in detail. Following one’s owntalent and inclination, one should select at least one topic,and study it in depth. In the others, one should follow theexample of the bee which sucks a drop of nectar from eachflower.."


He turned down many notable invitations, including themost appealing one of becoming president of St.Petersburg’s Academy (1725). He also refused the chair ofMathematics at the University of Padua and the invitationto the Court of Wien as Aulic Adviser.On April 2, 1754, he had a sudden bout of fever. A fortnightlater, on April 15, he passed away.


Riccati’s interest evolved around scalar equations of thetype

ẋ = ax2 + bx + c

with time varying or constant parameters.Besides a number of further equations of first order

ẋ = αtpxq + βtm

where m,p, and q are constants.he was particularly attracted by the equation

ẍ = αtm

which he called “misleading equation".A generalization of the equation into a matrix form (thematrix Riccati equation) plays a major role in many designproblems of modern engineering, especially filtering andcontrol.


NARE

C + XA + DX − XBX = 0

where A ∈ Cn×n,B ∈ Cn×m,C ∈ Cm×n,D ∈ Cm×m are known.

Find X ∈ Cm×n.

Construct:

H =[

A −B−C −D

]∈ C(m+n)×(m+n)


Observations

1 X satisfies the NARE if and only if

H[InX

]=

[InX

](A− BX )

⇒ HV ⊆ V

where

V =〈[

InX

]〉→ Graph space

2

λ ∈ Λ(A− BX )⇒ λ ∈ Λ(H)3

Λ(H) = Λ(A− BX ) ∪ Λ(−D + XB)


Finding a solution of the NARE is equivalent to finding the basisof an invariant subspace of H.

Observation:1 If

[YZ

]spans an n-dimensional subspace V of H such that

Y ∈ Cn×n is invertible, then X = ZY−1 is a solution of theNARE.

2 The solution X is related to the eigenvectors (Jordanchains) of A− BX .


The solution X is associated with a set of eigenvalues ofA− BX but it is NOT a one-one correspondence!!

A =[3 02 3

],B =

[−1 11 1

],C =

[2 02 0

],D =

[−2 1−1 0

]has solution

X =[1 11 1

],Y =

[0 11 0

].

Indeed,

H[InX

]=

[InX

]U, H

[InY

]=

[InY

]V

where

U =[2 11 2

],V =

[3 00 1

]have the same eigenvalues.


In most applications, the required solutions are associatedwith a subset of n eigenvalues of H corresponding to aunique n-dimensional invariant subspace.In other situations, where there exist different invariantsubspaces associated with the same set of eigenvalues,some additional properties are needed in order to selectthe subspace of interest.This choice is generally made relying on the spectralproperties of the matrix H.


Splitting properties

For any k -tuple of complex numbers, in particular theeigenvalues of a matrix.

The tuple ω = (λ1, . . . , λk ) of k complex numbers has a(k1, k2) splitting with respect to the imaginary axis orc-splitting if k = k1 + k2, ki ≥ 1, i = 1,2, and there exists apermutation π of {1,2, . . . , k} such that

λπ(i) ∈ C≤, i = 1 : k1λπ(i) ∈ C≥, i = k1 + 1, . . . , k .

C≤ := {z ∈ C : re(z) ≤ 0}, C≥ := {z ∈ C : re(z) ≥ 0}


Let ω1 = (λπ(1), . . . , λπ(k1)), ω2 = (λπ(k1+1), . . . , λπ(k)).

Proper splitting: A c-splitting is proper if ω1 OR ω2 does notintersect the imaginary axis.Strong splitting: A c-splitting is strong if both ω1 AND ω2 donot intersect the imaginary axis.Weak splitting: A c-splitting is weak if both ω1 AND ω2intersect the imaginary axis.

Stability regionA matrix A is called c-stable if all the eigenvalues of A lie in thestability region

C< = {z ∈ C : re(z) < 0}.


An invariant subspace of a matrix is called stable if it is spannedby Jordan chains associated with the stable eigenvalues,similarly, a subspace is called anti-stable if it is spanned byJordan chains associated with anti-stable eigenvalues.An invariant subspace of a matrix is called weakly stable if it isspanned by Jordan chains associated with stable and criticaleigenvalues, similarly, a subspace is called weakly antistable ifit is spanned by Jordan chains associated with antistable andcritical eigenvalues.

TheoremIf the eigenvalues of H have a proper (m,n) c-splitting and if Hhas an n-dimensional c-antistable graph invariant subspace,then the NARE has a unique solution X such thatΛ(A− BX ) ⊆ C>.


Critical solutions

Fréchet derivative

The Fréchet derivative of a matrix function F : Cn×n → Cn×n atX is a linear function L : Cn×n → Cn×n, E 7→ L(X ,E) such thatfor all E ∈ Cn×n

F(X + E)−F(X )− L(X ,E) = o(‖E‖)

1 The Fréchet derivative, if it exists, is unique.2 A function F is said to be Fréchet differentiable at X if

there exists its Fréchet derivative at X3 The notation L(X ,E) can be read as “the Frechet

derivative of F at X in the direction E", or “the Fréchetderivative of F at X applied to the matrix E".


Critical solutions

Consider the Riccati operator R : Cm×n → Cm×n defined by

R(X ) = C + XA + DX − XBX .

Then its Fréchet derivative is given by

L(X ,E) = EA + DE − EBX − XBE .

Consequently, L(X ,E) can be represented in terms of matrixvector products as

vec(L(X ,E)) = ∆X vec(E)

where∆X = (A− BX )T ⊗ Im + In ⊗ (D − XB).

The matrix ∆X coincides also with the Jacobian of R(X ) withrespect to the variables xij ordered columnwise.


Critical solutions

The eigenvalues of

∆X = (A− BX )T ⊗ Im + In ⊗ (D − XB)

are the sums of those of A− BX and D − XB (Homework!!).

A solution S of the NARE is called critical if the Jacobian ∆S ofthe Riccati operator R(X ) is singular at X = S.

Note that the matrix ∆X can be singular only if A− BX andXB − D have a common eigenvalue. Since the eigenvalues ofH are the union of the eigenvalues of A− BX and of XB − D,therefore the spectral properties of H play a crucial role for theexistence of critical solutions.


Critical solutions

Shifting techniques

Introduce a transform which convert H into a new matrix Ĥsuch that the graph c-(anti)stable invariant subspace is thesame for H and for Ĥ.This transformation will be particularly useful when wehave to compute a critical solution S. In fact, in this casethe new (transformed) algebraic Riccati equation is suchthat S is still a solution, but the Jacobian of the new Riccatioperator R̂(X ) is not singular at X = S.


Critical solutions

We describe the shift technique for NARE such that thecorresponding matrix H is singular.

Let the eigenvalues λi , i = 1, . . . ,m + n, of H have the following(m,n) c-splitting

re(λ1) ≤ . . . ≤ re(λm) ≤ 0 = λm+1 ≤ re(λm+2) ≤ . . . ≤ re(λm+n)

and thus the NARE has an (almost) c-antistabilizing solution S,such that A− BS has eigenvalues λm+1 = 0, λm+2, . . . , λm+n. Inparticular,

H[InS

]=

[InS

](A− BS).


Critical solutions

The goal is to construct a new matrix Ĥ having the same graphinvariant subspace of H, spanned by[

InS

]where the matrix A− BS is replaced by a matrix havingeigenvalues η, λm+2, . . . , λm+n, and η ∈ C>. In particular, theeigenvalue λm+1 = 0 of H is shifted to the eigenvalue η of Ĥ.

TheoremLet A be an n × n matrix with eigenvalues λ1, λ2, . . . , λn and letv be a nonzero vector such that Av = λ1v . For any nonzerovector x , the eigenvalues of A + vx∗ are λ1 + x∗v , λ2, . . . , λn.


Critical solutions

Construction of Ĥ1 Let v ∈ Cm+n be a nonzero vector such that Hv = 02 Let p ∈ Cm+n be such that p∗v = 1, and let η be a complex

number with positive real part.3 Define Ĥ = H+ ηvp∗

Then the eigenvalues of Ĥ are those of H except for theeigenvalue λm+1 = 0 of H which is replaced by η (Homework!!use the previous Theorem).

TheoremLet A be an n× n matrix and let v be an eigenvector of A, that isAv = λv for some λ. Let V be an n ×m matrix whose columnsspan an invariant subspace of A of dimension m including v , sothat AV = VP for a suitable P ∈ Cm×m. Then, for any nonzerox ∈ Cn, it holds that (A + vx∗)V = V (P + ṽ x̃∗) where ṽ is theunique solution of Vṽ = v and x̃ = x∗V . Moreover Pṽ = λṽ .


Critical solutions

Thus

Ĥ[InS

]=

[InS

]R

where R is a rank-one modification of A− BS, havingeigenvalues η, λm+2, . . . , λm+n.

We may partition p, v , and Ĥ according to the partitioning of Has

p =[p1p2

], v =

[v1v2

], Ĥ =

[Ã −B̃−C̃ −D̃

]and hence

Ã = A + ηv1p∗1 B̃ = B − ηv1p∗2C̃ = C − ηv2p∗1 D̃ = D − ηv2p∗2.


Critical solutions

Finally we have the new NARE

C̃ + XÃ + D̃X − XB̃X = 0

which has the same solution X as the original NARE, where theeigenvalue λm+1 = 0 of A− BX is shifted to the eigenvalue η ofÃ− B̃X .

Remark1 if re(λm+2) > 0, then X is the c-antistabilizing solution of

the new NARE.2 In the case where λm = λm+1 = 0, so that X is a critical

solution, if re(λm+2) > 0, then we may show that X is anoncritical solution of the new NARE


C + XA + A∗X − XBX = 0

where A,B,C ∈ Cn×n,B∗ = B,C∗ = C.Construct

H =[

A −B−C −A∗

]

Observations1 H is a Hamilton matrix: JH = −H∗J whereJ =

[0 In−In 0

].

2 The spectrum of H is symmetric with respect to theimaginary axis, i.e., the nonimaginary eigenvalues of Hcome in pairs (λ,−λ),


The eigenvalues of H can be ordered in such a way that

re(λ1) ≤ re(λ2) ≤ . . . ≤ re(λn) ≤ 0≤ re(λn+1) ≤ re(λn+2) ≤ . . . ≤ re(λ2n).

Thus the eigenvalues of H have an (n,n) c-splitting.1 If H has no eigenvalues on the imaginary axis, then the

splitting is strong and therefore there are unique, c-stable,and c-antistable invariant subspaces corresponding to then eigenvalues with negative real part and positive real part,respectively. (Homework!!)

2 If the splitting is weak, then there can be more than onec-stable n-dimensional invariant subspace. However, if allthe imaginary eigenvalues have even partial multiplicities,then there exists a unique, canonical weakly c-stableinvariant subspace. (Homework!!)


Consider the scalar equation

αx2 + βx + γ = 0

where α, β, γ ∈ R and α 6= 0. Then

H =[ β

2 α

−γ −β2

].

The eigenvalues of H are ±12√β2 − 4αγ.

1 If√β2 − 4αγ 6= 0 then there are two 1-dimensional

H-invariant subspaces of C2, namely

〈[1

−β±√β2−4αγ

2α

]〉2 If

√β2 − 4αγ = 0 then there exists only one such

subspace S =〈[

1− β2α

]〉which corresponds to the unique

solution x = −β/2α.


Recall the CARE

C + XA + A∗X − XBX = 0.

Note that X is Hermitian if and only if

[In X ∗

]J[InX

]= 0.

J -NeutralA subspace S is called J -Neutral if x∗J y = 0 for all x , y ∈ S.

Then it follows that if S =〈[

InX

]〉is an invariant subspace

associated with the solution X , then X is Hermitian if and only ifS is J -neutral subspace of H.


TheoremIf H has no purely imaginary eigenvalues and there exists asolution X such that A− BX is stable, then X is the uniquestabilizing solution, in particular X is Hermitian. If the purelyimaginary eigenvalues of H correspond to even-sized Jordanblocks in the Jordan normal form of H, and if there exists asolution X associated with the canonical c-stable invariantsubspace, then X is Hermitian.

Proof: The columns of[InX

]span the canonical c-stable

invariant subspace of H. Since the equation is symmetric, thematrix X ∗ is a solution of the CARE as well, and

H[

InX ∗

]=

[InX ∗

](A− BX ∗).


We show that the columns of[

InX ∗

]span the canonical c-stable

invariant subspace of H, and from the uniqueness of such asubspace we deduce that X = X ∗.

Since[−X In

] [InX

]= 0, the columns of

[−X ∗

In

]span the

space orthogonal to the space spanned by[InX

]. On the other

hand, since left and right invariant subspaces corresponding todisjoint sets of eigenvalues are orthogonal (Homework!!), theleft canonical c-antistable invariant subspace is orthogonal to

the span of[InX

].


Thus, since[InX

]and

[−X ∗

In

]span C2n, then we deduce that the

left canonical c-antistable invariant subspace of H is spanned

by[−X ∗

In

].

Moreover,[−X In

]H = (XB − A∗)

[−X In

]implies that the

matrix XB − A∗ collects the eigenvalues of H with positive realpart and the imaginary eigenvalues of H with half partialmultiplicity.Thus A− BX ∗ = −(XB − A∗)∗ has the same Jordan structureas A− BX , and by the uniqueness of the canonical invariantsubspace we have X = X ∗.


TheoremLet B � 0.

1 There exists a unique Hermitian solution X+ of the CAREsuch that the eigenvalues of A−BX+ have nonpositive realpart if and only if the pair (A,B) is c-stabilizable and thepartial multiplicities of the pure imaginary eigenvalues of H(if any) are all even.

2 There exists a unique Hermitian solution X− of the CAREsuch that the eigenvalues of A− BX− have nonnegativereal part if and only if the pair (−A,B) is c-stabilizable andthe partial multiplicities of the pure imaginary eigenvaluesof H (if any) are all even.


Critical solutions

The Fréchet derivative of the Riccati operator (corresponding toCARE) takes the form

L(X ,E) = EA + A∗E − EBX − XBE

which is expressed in matrix form

vec(L(X ,E)) = ∆X vec(E)

where∆X = (A− BX )T ⊗ In + In ⊗ (A∗ − XB).

As is the case of NARE, a solution S of the CARE is calledcritical if the Jacobian ∆S is singular at X = S.


Critical solutions

Observations1 The matrix H is Hamiltonian and its eigenvalues come in

pairs (λ,−λ). From the properties of the eigenvalues ofKronecker products, if H has no pure imaginaryeigenvalue, then any c-(anti)stabilizing solution of theCARE is not critical.

2 If H has some eigenvalues on the imaginary axis, that theirpartial multiplicities are even, and that there exists asolution S of the CARE associated with the canonicalweakly c-stable invariant subspace. From the Theoremproved earlier, the matrix S is the unique (almost)c-stabilizing Hermitian solution S. This solution is critical.In fact, S is such that A− BS has at least one purelyimaginary eigenvalue λ and the matrix(A− BS)∗ = A∗ − SB has the eigenvalue −λ. Thereforethe Jacobian ∆S is singular.


Critical solutions

Shifting the CARE

We describe the shift technique for the CARE with a singularHamiltonian H without changing the Hamiltonian structure ofthe new Ĥ.

Assumptions1 Assume that H has a zero eigenvalue with partial

multiplicity 2 and that there are no pure imaginaryeigenvalues.

2 Assume also that the CARE has an (almost) c-stabilizingsolution S, such that the eigenvalues of A− BS areλ1, . . . , λn−1,0. The solution S is critical, since zero is adouble eigenvalue of H.


Critical solutions

In order to maintain the Hamiltonian property we need to applya double shift by moving the zero eigenvalues to a pair ofeigenvalues η and −η, where η ∈ R, η 6= 0. Here and hereafterwe assume that η < 0.

Procedure

1 Denote by v =[v1v2

]a right eigenvector of H corresponding

to the zero eigenvalue.2 Then w∗ =

[v∗2 −v∗1

]is a left eigenvector of H

corresponding to the zero eigenvalue.3 Since λn = 0 is an eigenvalue of H with partial multiplicity

2, w∗v = 0. (Homework!!)


Critical solutions

Then it follows that the double shift which maps one zeroeigenvalue in η 6= 0 and the other zero eigenvalue in −η isgiven by

Ĥ = H+η[v1v2

] [p∗1 p

∗2]−η[q∗1q∗2

] [v∗2 −v∗1

]where the vectors p1,p2,q1,q2 are chosen in such a way that

p∗1v1 + p∗2v2 = 1

v∗2 q1 − v∗1 q2 = 1.

Observe1 Ĥv = ηv and w∗Ĥ = −ηw∗

2 Ĥ[InS

]=

[InS

]R̂ (Homework!!, use w∗v = 0)


Critical solutions

Finally, for the Hamiltonian property of Ĥ we must have

J T([

v1v2

] [p∗1 p

∗2]−[q∗1q∗2

] [v∗2 −v∗1

])J

= −([

v1v2

] [p∗1 p

∗2]−[q∗1q∗2

] [v∗2 −v∗1

])This yields (after some algebraic manipulation, Homework!!)

p1 = v2 + θv1,p2 = θv2 − v1,q1 = θv2,q2 = −θv1, θ = ‖v‖22.

In this way, Ĥ is still Hamiltonian, and the matrix coefficients aregiven by

Â := A + η(v1v∗2 + θv1v∗1 − θv2v∗2 )

B̂ := B − η(θv1v∗2 + θv2v∗1 − v1v∗1 )Ĉ := C − η(θv2v∗1 + θv1v∗2 + v2v∗2 ).


Critical solutions

If v∗2 v1 6= 0, a simpler formula can be obtained as follows

p1 = θv2,q1 = θv1,p2 = q2 = 0, θ = 1/(v∗2 v1)

and in that case

Â := A, B̂ := B − ηv1v∗1 , Ĉ := C − ηv2v∗2

form Ĥ a Hamiltonian matrix.

The eigenvalues of Ĥ are λi = −λn+i , i = 1, . . . ,n − 1, andλn = −λn+1 = η. Therefore, if η < 0, the almost c-stabilizingsolution S of the original CARE is now the c-stabilizing solutionof the new CARE. Moreover, while the solution S is critical forthe original CARE, the same solution S is not critical for thenew CARE.


Critical solutions

Example

Consider the CARE

A =[0 00 −1

],B =

[1 00 0

],C =

[0 11 2

]and X+ =

[0 11 1/2

]is the unique Hermitian solution. Moreover,

it is a critical solution since A− BX+ =[0 −10 −1

]. The

eigenvalues of the associated matrix

H =

0 0 −1 00 −1 0 00 −1 0 0−1 −2 0 1

are −1,0,1, where 0 has partial multiplicity 2. In particular,Hv = 0


Critical solutions

where

v =[v1v2

], v1 =

[10

], v2 =

[01

].

Applying the double shift technique with η = −2 we obtain

Â =[−1 −20 0

], B̂ =

[−1 11 0

], Ĉ =

[0 22 4

].

The eigenvalues of the matrix Ĥ are −2,−1,1, and 2, and thematrix X+ is the unique c-stabilizing solution of the CARE withmatrix coefficients Â, B̂, Ĉ moreover X+ is noncritical.


Invariant subspace method

The most straightforward way to find an invariant subspace isthrough eigenvectors, but the procedure may lead tounexpected numerical problems since it may happen that theinvariant subspace to be computed is well conditioned, whilesome single eigenvector is not. A more numerically soundprocedure is based on the Schur decomposition.

AssumptionAssume that the CARE has a unique c-stabilizing solution X+,in particular, the eigenvalues of the matrix H of have a strong(n,n) c-splitting since the matrix H is Hamiltonian and theCARE has a c-stabilizing solution.


Thus

H[UV

]=

[UV

]Γ

where Γ is an n × n c-stable matrix, then

X+ = UV−1.

Therefore, in order to compute the c-stabilizing solution X+ it issufficient to compute a basis for the c-stable invariant subspaceof H, which is unique by the splitting of the eigenvalues of H.


This task can be performed efficiently by using the semiorderedreal Schur decomposition of H. More specifically, the matrix His factored as H = QRQT , where Q and R are partitioned intofour n × n blocks

Q =[Q11 Q12Q21 Q22

], R =

[R11 R120 R22

],

andQ is orthogonalR11 and R22 are block upper triangular matrices withdiagonal blocks of size at most 2, moreover, the matrix R11is c-stable, that is, R11 collects the eigenvalues of H withnegative real part.

The first n columns of Q span the c-stable invariant subspaceof H so that

X+ = Q21Q−111is the stabilizing solution


The semiordered real Schur decomposition can be computedby the MATLAB functions schur and ordschur according to thefollowing two steps:

1 [U,T] = schur(H,‘real’) computes a real Schurdecomposition of H by means of the QR algorithm

2 [Q,R] = ordschur(U,T,select)swaps the diagonalblocks by means of orthogonal transformations in such away that the eigenvalues with indices selected by thelogical vector select are the eigenvalues of the leadingn × n submatrix of R.

The same approach can be applied in more general contexts,say, in the case of a NARE. In this generalized version, one hasto identify which eigenvalues of the associated matrix Hcorrespond to the sought solution. Then, by a sequence ofunitary transformations, one must put the desired eigenvaluesin the upper left block of an ordered Schur form of H.


Standard Newton’s method

Newton’s method is the customary numerical tool for solvingscalar nonlinear equations. Given an equation f (x) = 0, wheref is continuously differentiable in a neighborhood of a solutionα ∈ C, Newton’s method generates a sequence {xk} defined bythe recurrence

xk+1 = xk − f (xk )/f ′(xk )

for a suitable initial guess x0 and whose limit is α.The method can be used also for solving equations of the kindF (X ) = 0, where F : V → V is a differentiable operator in aBanach space (we are interested only in the case in which V isCm×n). The sequence is defined by

Xk+1 = Xk − (F ′Xk )−1F (Xk ),X0 ∈ V ,

where F ′(X ) is the Fréchet derivative of F at the point X .

IntroductionThe Model and the ProblemReferences

About Riccati and his equationNon-symmetric Algebraic Riccati Equation (NARE)Critical solutions

Continuous time ARE (CARE)Critical solutions

Numerical methods for solving ARE

Documents

Algebraic Riccati Equation - IIT Kharagpurbibhas/ARE.pdfIntroductionAbout Riccati and his equation Non-symmetric Algebraic Riccati Equation (NARE) Continuous time ARE (CARE)Numerical