~IAM RFVIF:\I,:

Vol 20. N('1 4. QctOtw-, IQ7R

© Soculy 'or Industrial a"d APrlt~d Mllh~mah'-"003b·I"~f7 8/20"'·003 ISO100/0

NINETEEN DUBIOUS WAYS TO COMPUfETHE EXPONENTIAL OF A MATRIX*

CLEVE MOLERt AND CHARLES VAN LOANt

Ab~tr.ct. In principle. th~ exponential of a malri~ could be: computed in many ways. Methods involving

appro~imation theory. differential equations. the matri~ eijZenvalues. and the matri~ characteristic poly­

nomial have oeen proposed. In practice:. consideration of computational stability ~nd efficiency indicatesthat some of the methods arc prc:rerable to olhers. but Ihal none are completely sii'isfactory.

1. Introduction. Mathematical models of many physical. biological.. andeconomic processes involve systems of linear. constant coefficient ordinary dmerentialequations

i(t)=Ax(t).

Hcre A is a given. fixed, real or complex n-by-n matrix. A solution vector x(t) issought which satisfies an initial condition

x(O)=xo.

In control theory, A is known as the state companion matrix and x(t) is the systemresponse.

In principle, the solution is given by x(t)=e'Axo where e'A can be formallydefined by the convergent power series

t2 A 2

e 'A = I + fA +-- + ...2!

The effective computation of this matrix function is the main topic of this survey.We will primarily be concerned with matrices whose order n is 'Iess than a few

hundred, so that all the clements can be stored in thc main memory of a contemporarycomputer. Our discussion will be less germane to the type of large, sparse matriceswhich occur in the method of lines for partial differential equations.

Dozens of methods for computing erA can be obtained from more or less classicalresults in analysis. approximation theory, and matrix theory .. Some of the methodshave becn proposed as specific algorithms. while others are based on less constructivecharacterizations. Our bibliography concentrates on recent papers with strongalgorithmic content. although we have included a fair number of references whichpossess historical or theoretical interest.

In this survey we try to describe all the methods that appear to be practical.classify them into five broad categories. and assess their relative effectiveness. Actu­ally. each of the "methods" when completely implemented might lead to manydifferent computer programs which differ in various details. Moreover, these detailsmight have more influence on the actual performance than our gross ~ssessmeQt~indicates. Thus. our comments may not directly apply to particular subroutines.

In assessing the effectiveness of various algorithms we will be conccrned with thefollowing attributes. listed in decreasing order of importance: generality, reliability,

• Received oy the editors July 2:1.1976. and in rc:vis~d form March 19. 1977.t Department of Malhematics. University of N~w M~~ico. Albuquerque. New M~~ico 87106. This

work was partially supported oy NSF Grant MCS76-03052.t Department of Computer Science. Cornell University. hhaca. New York 14850. This work was

rartially support~d by NSF Grant MCS76-086116.

801

802 CLEVE MOLER AND CHARLES VAN LOAN

stability, accuracy, efficiency, storage requirements, ease of use, and simplicity. Wewould consider an algorithm completely satisfactory if it could be used as the basis fora general purpose subroutine which meets the standards of quality software' nowavailable for linear algebraic equations, matrix eigenvalues, and initial value problemsfor nonlinear ordinary differential equations. By these standards, none of thealgorithms we know of are completely satisfactory, although some are much betterthan others ..

Generality means thjwthe method is applicable to wide classes of matrices. Forexample, a method which works only on matrices with distinct eigenvalues will not behighly regarded ..

When defining terms like reli/bility, stability and accuracy, it is important todistinguish between the inherent s~nsitivity of the underlying problem and the errorproperties of a particular algorithm for solving that problem. Trying to find the inverseof a nearly singular matrix, for example, is an inherently sensitive problem. Suchproblems-are said to be poorly posed or badly conditioned. No algorithm working withfinite precision arithmetic can be expected to obtain a computed inverse that is notcontaminated by large errors.

An algorithm is said to be reliable if it gives some warning whenever it introducesexcessive errors. For example, Gaussian elimination without some f<;>rmof pivoting isan unreliable algorithm for inverting a matrix. Roundoff errors can be magnified bylarge multipliers to the point where they can make the computed result complt:telyerroneous, but thue is no indication of the difficulty.

An algorithm is stable if it does not introduce any more sensitivity to perturbationthan is inherent in the underlying problem. A stable algorithm produces an answerwhich is exact for a problem close to the given one. A method can be stable and stillnot produce accurate results if small changes in the data cause large changes in theanswer. A method can be unstable and still be reliable if the instability can bedetected. For example, Gaussian elimination with either partial or complete pivotingmust be regarded as a mildly unstable algorithm because there is a possibility that thematrix elements will grow during the elimination and the resulting roundoff errors willnot be small when compared with the original data. In practice, however, such growthis rare and can be detected.

The accuracy of an algorithm refers primarily to the error introduced by truncat­ing infinite series or terminating iterations. It is one component, but not the onlycomponent, of the accuracy of the computed answer. Often, using more computertime will increase accuracy provided the method is stable. For example, the accuracyof an iterative method for solving a system of equations can be controlled by changingthe number of iterations.

Efficiency is measured by the amount of computer time required to solve aparticular problem. There are several probiems to distinguish. For example, compul­ing only e'" is different from computing e'''' for several values of I. Methods which usesome decomposition of A (independent of I) might be more efficient for the secondproblem. Other methods may be more efficient for computing e,A Xu for one or severalvalues of I. We are primarily concerned with the order of magnitude of the workinvolved. In matrix eigenvalue computation, for example, a method which requiredO(n·) time would be considered grossly inefficient because the usual methods requireonly O(n \

In estimating the time required by matrix computations it is traditional to esti­mate the number of multiplications and then employ some factor to account for theother operations. We suggest making this slightly more precise by ddining a basic

THE EXPONENTIAL OF A MA TRtX 803

Roating point operation. or "flop". to be the time required for a particular computersystem to execute the FORTRAN statement

A(l. J)= A(l. J)+ T*A(l. K).

This involves one floating point multiplication, one floating point addition, a fewsubscript and index calculations. and a few storage references. We can then say. forexample. that Gaussian elimination requires nJ /3 flops to solve an n-by-n linearsystem Ax = b.

The eigenvalues of A playa fundamental role in the study of e'A. even thoughthey may not be involved in a specific algorithm. For example. if all the eigenvalues liein the open left half plane. then e'A. -+ 0 as t -+ 00. This property is often called"stability" but we will reserve the use of this term for describing numerical propertiesof algorithms.

Several particular classes of matrices lead to special algorithms. If A is symmetric,then methods based on eigenvalue decompositions are particularly effective. If theoriginal problem involves a single. nth order differential equation which has beenrewritten as a system of first order equations in the standard way. then A is a

companion matrix and other special algorithms are appropriate.The inherent difficulty of finding effective algorithms for the matrix exponential is

based in part on the following dilemma. Attempts to exploit the special properties ofthe differential equation lead naturally to the eigenvalues Aj and eigenvectors Vj of Aand to the representation

••

x(t) = L aj eJ."vj.i-I

I J ••• '" -.

804 CLEVE MOLER AND CHARLES VAN LOAN

IIIiIIJ~,

but not negligible. Then, if the divided difference'" ",re -e

A-po.

is computed in the most obvious way, a result with a large relative error is produced.When multiplied by a, the final computed answel' may be very inaccurate. Of course,for this example, the formula for the off-diagonal element can be written in other wayswhich are more stable. However, when the same type of difficulty occurs in nontrian­gular problems, or in problems that are larger than 2-by-2, its detection and cure is byno means easy.

The example also illustrates another property of etA which must be faced by anysuccessful algorithm. As I increases, the elements of e0- may grow before they decay.If A and po. are both negative and a is fairly large, the graph in Fig. 1 is typical.

s/m

( FIG. 1. Th~"hump".

s

Several algorithms make direct or indirect use of the identity

The difficulty occurs when sl m is under the hump but s is beyond it, for then

lIe'''II« Ik'A/'"II'".

Unfortunately, the roundoff errors in the mth power of a matrix, say B'", are usuallysmall relative to IIBII'" rather than liB '"II. Consequently, any algorithm which tries topass over the hump by repeated multiplications is in difficulty.

Finally, the example illustrates the special nature of symmetric matrices. A issymmetric if and only if a = 0, and then the difficulties with multiple eigenvalues andthe hump both disappear. We will find later that multiple eigenvalue and humpproblems do not exist when A is a normal matrix.

It is convenient to review some conventions and definitions at this time. Unless

otherwise stated, all matrices are n-by-n. If A = (aii) we have the notions of transpose,AT = (ail)' and conjugate transpose, A· = (a,,), The following types of matrices have

i>j,i ;ti j.

THE EXPONENTIAL OF A MATRIX

an Important role 10 play:

A symmetric-A T = A.A Hermitian-A* =A.A normal-A*A =AA*.o orthogonal- 0 TO = I,o unitary-O*O = I,

T triangular-ti; = O.D diagonal- di; = O.

R05

Because of the convenience of unitary invariance. we shall work exclusively withthe 2-norm:

/!All= max IIAxli.hi-I

However, all our results apply with minor modification when other norms are used.The condition of an invertible matrix A is denoted by cond (A) where

cond (A)=IIAIIIIA-'II.

Should A be singular. we adopt the convention that it has infinite condition. Thecommutator of two matrices Band C is [B. C) = BC - CB.

Two matrix decompositions are of importance. The Schur decomposition statesthat for any matrix A. there exists a unitary 0 and a triangular T. such that

O*AO=T.

If T = (ti;). then the eigenvalues of A are tll ....• t"".The Jordan canonical form decomposition states that there exists an invertible P

such that

where 1 is a direct sum, 1 =110';' ·(;91.,of Jordan blocks

Ai 1 0 0

o Ai 1 0

1j= (mi-by-m,).

o 0 0 ... A

The A. are eigenvalues of A. If any of the :)are greater than I, A is said to bedefective. This means that A does not have a full set of n linearly independent

eigenvectors. A is derogatory if there is more than one Jor.gan block associated with agiven eigenvalue .. -

2. The sensitivity of the-problem. It is important to know how sensitive a quantityis he fore its computation is attcmpted. For the problem under consideration we areinterested in the relative perturbation

1Ie,IA.EI- r'AIi

cfJ(t) = lie 'All

In the following three theorems we summarize some upper bounds for cfJ(t) which arederived in Van Loan [32].

-,

806 CLEVE MOLER AND CHARLES VAN LOAI'

>=-

THEOREM 1. If a (A) = max {Re (A )IA an eigenvalue of A} and JL(A) = max {JLIJL

an eigenvalue of (A" + A )/2}, Ihen

cP(/)~ tllEIl exp [~A)-a(A)+IIEII)t (t ~ 0).

The scalar JL(A) is the "log norm" of A (associ~ted with the 2-norm) and has many

interesting properties 135]-(42). In par.ticular, JL(A)~ a(A).THEOREM 2. If A = PJP- is the Jordan decomposition of A and m is the dimen­

sion of the largest Jordan block in J, then

cP(t)~ tllEIIMJ(tf e"",I'M/EI' (t ~O),

where

MJ(t)=mcond(P) max t'lj!.0:0,=0",-1

THEOREM 3. If A = Q(D + N)Q* is Ihe Schur decomposilion of A with D diagonal

and N striclly upper Iriangular (nij = 0, i ~ j), Ihen

cP(r)~ rllEllMs(d e ""sl'lIEl, (I ~ 0),

where

,,-1Ms(t) = L (IINlitt I k!.1-0

As a corollary to any of these theorems one can show that if A is normal, then

This shows that the perturbation bounds on cP(r) for normal matrices are as small ascan be expected. Furthermore, when A is normal, Ik'AIi = Ik'A/"'II'" for all positiveintegers m implying that the "hump" phenomenon does not exist. These observationslead us to conclude that the eA problem is "well conditioned" when A is normal.

It is rather more difficult to characterize those A for which e'A is very sensitive tochanges in A. The bound in Theorem 2 suggests that this might be the case when Ahas a poorly conditioned eigensystem as measured by cond (P). This is related to alarge Ms(t) in Theorem 3 or a positive JL(A) - a (A) in Theorem 1. It is unclear whatthe precise connection is between these situations and the hump phenomena wedescribed in the Introduction.

Some progress can be made in understanding the sensitivity of e'A by defining the"matrix exponential condition number" v(A, t):

A discussion of v(A, t) can be found in (32). One can show that there exists aperturbation E such that

This indicates that if v(A, I) is large, small changes in A can induce relatively largechanges in e'A. It is easy to verify that

v(A, t)~/IIAII,

THE EXPONENTIAL OF A MATRIX 807

with equality if and only if A is norma:. When A is not normal. v(A. t) can grow like ahigh degree polynomial in t.

3. Series methods. The common theme of what we call series methods is the

direct application to matrices of standard approximation techniques for the scalarfunction e'. In these methods. neither the;order of the matrix nor its eigenvalues playadirect role in the actual computations.

METHOD 1. TAYLOR SERIES. The definition Je"=I+A+A2/2!+'" .

is. of course. the basis for an algorithm. If we momentarily igndTe efficiency. we cansimply sum the series until adding another term does not alter the numbers stored inthe computer. That is. if

•T.(A)= L Ai/j!

i-O

and fl r T. (A)) is the matrix of floating point numbers obtained by computing T" (A) infloating point arithmetic. then we find K so that fl [TK(A)] = fl [TK+1(A)]. We thentake TdA) as our approximation to e".

Such an algorithm is known to be unsatisfactory even in the scalar case [4] andour main reason for mentioning it is to set a clear lower bound on possible per­formance. To illustrate the most serious shortcoming. we implemented this algorithmon the IBM 370 using "short" arithmetic. which corresponds to a relative accuracy of16-~== 0.95 . 10-". We input

= [-49 24JA -64 31

and obtained the output

e" = [-22.25880-61.49931 -1.432766J.-3.474280

A total of K = 59 terms were required to obtain convergence. There are several waysof obtaining the correct e" for this example. The simplest is to be told how theexample was constructed in the first place. We have

A = [~

and so

which. to 6 decimal places is.

" [-0.735759e ""

-1.471518 0.551819J.1.103638

The computed approximation even has the wrong sign in two components.Of course. this example was constructed to make the method look bad. But it is

important to understand the source of the error. By looking at intermediate results inthe calculation we find that the two matrices A 16/ 16! and A 17/17! have elements

-,

808 CLEVE MOLER AND CHARLES VAN LOAN

between lOb and 107 in magnitude but of opposite signs. Because we are using arelative accuracy of only 10-5, the elements of these intermediate results haveabsolute errors larger than the final result. So, we have an extreme example of"catastrophic cancellation" in floating point arithmetic. It should be emphasized thatthe difficulty is not the truncation of the series, but the truncation of the arithmetic. Ifwe had used "long" arithmetic which does not require significantly more time butwhich involves 16 digits of accuracy, then we would ha'{e obtained a result accurate toabout nine decimal places.

Concern over where to truncate the series is important if efficiency is beingconsidered. The example above required 59 terms giving Method 1 low marks in thisconnection. Among several papers concerning the truncation error of Taylor series.the paper by Liou [52] is frequently cited. If fJ is some prescribed error tolerance, Liousuggests choosing K large enough so that

_ A <: IIA 11K +\ 1 <:IITK(A) e II=CK+})!)C-IIAIIIlK+2))=fJ.

Moreover, when elA is desired for several different values of I, say I = I, ... , m, hesuggests an error checking procedure which involves choosing L from the sameinequality with A replaced by mA and then comparing [TK(A)J"'xo with TdmA)xo.In related papers Everling [50] has sharpened the truncation error bound imple­mented by Liou, and Bickhart [46] has considered relative instead of absolute error.Unfortunately, all these approaches ignore the effects of roundoff error and so mustfail in actual computation with certain matrices.

METHOD 2. PADE APPROXIMATION. The (p, q) Pade approximation to eA IS

defined by

where

Npq(A)= f (p+q-j)!p!1:'0 (p +q)!j!(p _ j)!A'

and

-~:-D (A) = t (p +q - j)!q! (-A)i.pq 1-0 (p +q)!j!(q - j)!

Nonsingularity of Dpq(A) is assur~ if p and q are large enough or if the eigenvalues ofA are negative. Zakian [76] and Wragg and Davies [75] consider the advantages ofvarious representations of these rational approximations (e.g. panial fraction,continued fraction) as well as the choice of p and q to obtain prescribed accuracy.

Again, roundoff error makes Pade approximations unreliable. For large q,. -AI'" . A/~

D<I<I(A) approaches the senes for e - whereas N'Iq (A) tends to the senes for e -.

Hence, cancellation error can prevent the accurate determination of these matrices.Similar comments apply 10 general (p, q) approximants. In addition to the cancellationproblem, the denominator matrix Dpq(A) may be very poorly conditioned with respectto inversion. This is particularly true when A has widely spread eigenvalues. To seethis again consider the (q, q) Pade approximams. It is not hard to show that for largeenough q, we have

cond [D<I<I(A)] = cond (e-A/2)~ e(a,-a.1/2

where a I ~ ••• ~ a~ are the real parts of the eigenvalues of A.

THE EXPONENTIAL OF A MATRIX 809

When the diagonal Pade approximants Rqq(A) were computed for the sameexample used with the Taylor series and with the same single precision arithmetic. itwas found that the most accurate was good to only three decimal places. This occurredwith q = 10 and cond [Dqq(A») was greater than 10·. All other values of q gave lessaccurate results.

Pade approximants can be used if IIAII is not too large'. In this case. there areseveral reasons why the diagonal approximants (p = q) are preferred over the offdiagonal approximants (p ¢ q). Suppose p < q. About qn3 flops are required to evalu­ate Rpq (A). an approximation which has order p +q. However. the same amount ofwork is needed to compute Rqq (A) and this approximation has order 2q > p + q. Asimilar argument can be applied to the superdiagonal approximants (p > q).

There are other reasons for favoring the diagonal Pade approximants. If all theeigenvalues of A are in the left half plane. then the computed approximants with p > q

tend to have larger rounding errors due to cancellation while the computed approxi­mants with p < q tend to have larger rounding errors due to badly conditioneddenominator matrices Dpq(A).

There are certain applications where the determination of p and q is based on thebehavior of

lim Rpq(IA).,-co

If all the eigenvalues of A are in the open left half plane. then e,A ..• 0 as I'" 00 and thesame is true for Rpq (IA) when q > p. On the other hand. the Pade approximants withq < p. including q = O. which is the Taylor series. are unbounded for large I. Thediagonal approximants are bounded as I'" 00.

METHOD 3. SCALING AND SQUARING. The roundoff error difficulties and the

computing costs of the Taylor and Pade approximants increases as IIiAIi increases. oras the spread of the eigenvalues of A increases. Both of these difficulties can becontrolled by exploiting a fundamental property unique to the exponential function:

The idea is to choose m to be a power of two for which eA/ ••• can be reliably andefficiently computed. and then to form the matrix (eAl •••) ••• by repeated squaring. Onecommonly used criterion for choosing m is to make it the smallest power of two forwhich IIAli/m ~ 1. With this restriction. eAl ••• can he satisfactorily computed by eitherTaylor or Pade approximants. When properly implemented. the resulting algorithm isone of the most effective we know.

This approach has heen suggested by many authors and we will not try toattribute it to anyone of them. Amon/! those who have provided some error analysisor suggested some refinements are Ward [72]. Kammler [97]. Kallstrom [116].Scraton [67]. and Shah [56].[57] .

.. 14/"" .

If the exponential of the scaled matrix e . is to be approximated by Rqq(AI2').

then we have two parameters. q and j. to chot>se. In Appendix 1 we show that ifIIAII~2'-'then I

where

810 CLEVE MOLER AND CHARLES VAN LOAN

IJ(Rqq(A/2')f -e"l1lie "II

This "inverse error analysis" can be used to determine q and j in a number of ways.

For example, if £ is any error tolerance, we can choose among the many lq, j) pairs forwhich the above inequality implies

IIEII<:

IIAII=£'

Since [Rqq(A/i)]21 requires about (q + j +!)n3 flops. to evaluate, it is sensible tochoose the pair for which q + j is minimum. The table below specifies these "opti­mum" pairs for various values of £ and 11A1i. By way of comparison, we have includedthe corresponding optimum (k, j) pairs associated with the approximant [T,,(A/2i)f.These pairs were determined from Corollary 1 in Appendix 1, and from the fact thatabout (k + j -1)n3 flops are required to evaluate [T,,(A/Zi)]2'.

TABLE 1

Optimum scaling and sq_ring parameten with diagonal Padi and Taylor seriesapprozimation.

~

10-)

10-"10-910-1210- "

10-2

(1,0)(1,0)(2,0)(3,0)(3,0)(1,0)

(2. I)(3, I)(4, I)(5, I)

10-1

(1,0)(2,0)(3,0)(4,0)(4,0)(3.0)

(4.0)(4,2)(4,4)(5,4)

100

(2. 1)(3. 1)(4,1)(5. 1)(6. 1)(5.1)

(7,1)(6,3)(8,3)(7,5)

10'

(2,5)(3.5)(4,5)(5,5)(6.5)(4.5)

(6,5)(8.5)(7,7)(9,7)

102

(2,8)(3.8)(4.8)(5.8)(6,8)(4,8)

(5.9)(7.9)(9.9)(10, 10)

10)

(2,11 )(3,11)(4,11)(5, II)(6,11)(5, J I)

(7. 1J)(6,13)(8,13)(8,14)

To read the table, for a given £ and IIAII the top ordered pair gives the optimum (q, j)

associated with [Rqq (AO! »)~' while the bottom ordered pair specifies the most efficientchoice of (k, j) associated with (Tit (A/2' )(.

On the basis of the table we find that Pade approximants are generally more

efficient than Taylor approxlmanty. When IIA 1/ is small. the Pade approximant requiresabout one half as much work for the same accuracy. As IIAII grows, this advantagedecreases because of the larger amount of scaling needed.

Relative error bounds can be derived from the above results. Noting from

Appendix 1 that AE = EA. we have

"e"(eE - 1)11

lie "II

;a liEIIelEI~ EIIAII eolAI.

A similar bound can be derived for the Taylor approximants.

THE EXPONENTIAL OF A MATRIX 811

The analysis and our table does not take roundoff error into account. althoughthis is the method's weakest point. In general, the computed square of a matrix R canbe severely affected by arithmetic cancellation since the rounding errors are smallwhen compared to /lR/l2 but not necessarily small when compared to /lR2/i. Suchcancellation can only happen when cond (R) is large because R -) R 2 = R implies

/lR/l2

cond (R)~/lR211'

The matrices which are repeatedly squared in this method can be badly conditioned.However. this does not necessarily imply that severe cancellation actually takes place.Moreover. it is possible that cancellation occurs only in problems which involve a largehump. We regard it as an open question to analyze the roundoff error of the repeatedsquaring of eAt •••and to relate the analysis to a realistic assessment of the sensitivityof e"'.

In his implementation of scaling and squaring Ward [72] is aware of the possi­bility of cancellation. He compute.s an a posteriori bound for the error, including theeffects of both truncation and roundoff. This is certainly preferable to no errorestimate at all. but it is not completely satisfactory. A large error estimate could be theresult of any of three distinct difficulties:

(i) The error estimate is a severe overestimate of the true error. which is

actually small. The algorithm is stable but the estimate is too pessimistic .. (ii) The true error is large because of cancellation in going over the hump. but

the problem is not sensitive. The algorithm is unstable and another algorithmmight produce a more accurate answer.

(iii) The underlying problem is inherently sensitive. No algorithm can beexpected to produce a more accurate result.

Unfortunately, it is currently very difficult to distinguish among these three situations.

METHOD 4. CHEBYSHEV RATIONAL APPROtMATION. Let cqq(x) be the ratio oftwo polynomials each of degree q and considet maXO::i'<COjcqq(x)-e-'I. For variousvalues of q. Cody, Meinardus, and Varga [62] have determined the coefficients of theparticular Cqq which minimizes this maximum. Their result~can be directly translatedinto bounds for /lcqq(A)-e"'/i when A is Hermitian with eigenvalues on the negativereal axis. The authors are interested in such matrices because of an application topartial differential equations. Their approach is particularly effective for the' sparsematrices which occur in such applications.

For non-Hermitian (non-normal) A. it is hard to determine how well. c••••(A)approximates eA.. If A has an eigenvalue A off the negative real axis, it is possible forcQQ(A) to be a poor approximation to e'. This would imply that cqq(A) is a poorapproximation to e'" since

These remarks prompt us to emphasize an important facet about approximationof the matrix exponential. namely, there is more to approximating e'" than justapproximating el at the eigenvalues of A. It is easy to illustrate this with Padeapproximation. Suppose

[0 6 ° 0]

A=0060.000 6o 0 ° °

812 CLEVE MOLER AND CHARLES VAN LOAN

Since all of the eigenvalues of A are zero, R 11(z) is a perfect approximation to e: atthe eigenvalues. However,

whereas

and thus,

[I 618 54]

o 1

6 18R))(A)= 0 0

1 6'o 0

o 1

r 6

18 36]

eA = 0 1

6 18o 0

1 6o 0

o 1

IIeA_Rll(A~I= 18.

These discrepancies arise from the fact that A is not normal. The example illustratesthat non-normality exerts a subtle influence upon the methods of this section eventhough the eigensystem, per se, is not explicitly involved in any of the algorithms.

4. Ordinary differential equation methods. Since e,A and e''''xo are solutions toordinary differential equations, it is natural to consider methods based on numericalintegration. Very sophisticated and powerful methods for the numerical solution ofgeneral nonlinear differential equations have been developed in recent years. Allworthwhile codes have automatic step size control and some of them automaticallyvary the order of approximation as well. Methods based on single step formulas,multistep formulas, and implicit multistep formulas each have certain advantages.When used to compute e'A all these methods are easy to use and they require verylittle additional programming or other thought. The primary disadvantage is a rela­tively high cost in computer time.

The o.d.e. programs are designed to solve a single system

i = f(x, t), x(O) = Xo,

J:II!

and to obtain the solution at many values of I. With f(x, I) = Ax the kth column of e,A

can be obtained by setting Xo to the kth column of the identity matrix. All the methodsinvolve a sequence of values 0 = 10, II, .•. ,Ii = t with either fixed or variable step sizehi = 1,+I -I,. They all produce vectors Xi which approximate x (I;).

METHOD 5. GENERAL PURPOSE O.D.E. SOLVER. Most computer center librariescontain programs for solving initial value problems in ordinary differential equations.Very few libraries contain programs that compute e,A. Until the latter programs aremore readily available, undoubtedly the easiest and, from the programmer's point ofview, the quickest way to compute a matrix exponential is to call upon a generalpurpose o.d.e. solver. This is obviously an expensive luxury since the o.d.e. routinedoes not take advantage of the linear, constant coefficient nature of our specialproblem.

We have run a very small experiment in which we have used three recentlydeveloped o.d.e. solvers to compute the exponentials of about a dozen matrices andhave measured the amount of work required. The programs are:

(1) RKF45. Written by Shampine and Watts (108], this program uses theFehlberg formulas of the Runge-Kutta type. Six function evaluations are required pt:r

-,

THE EXPONENTIAL OF A MATRIX 813

slep. The resulting formula is fifth order with automatic step size control. (See also[4J.)

(2) DE/STEP. Written by Shampine and Gordon [107J, this program uses vari­able order, variable step Adams predictor-corrector formulas. Two function evalua­tions are required per step.

(3) IMPSUB. Written by Starner (109), this program is a modification of Gear'sDIFSUB (106) and is based on implicit backward differentiation formulas intendedfor stiff differential equations. Starner's modifications add the ability to solve"infinitely stiff" problems in which the derivatives of some of the variables may bemissing. Two function evaluations are usually required per step but three or four mayoccasionally be used.

For RKF45 the output points are primarily determined by the step size selectionin the program. For the other two routines, the output is produced at user specifiedpoints by interpolation. For an n-by-n matrix A, the cost of one function evaluation isa matrix-vector multiplication or n 2 flops. The number of evaluations is determined bythe length of the integration interval and the accuracy requested.

The relative performance of the three programs depends fairly strongly on theparticular matrix. RKF45 often requires the most function evaluations, especiallywhen high accuracy is sought, because its order is fixed. But it may well require theleast actual computer time at modest accuracies because of its low overhead.DE/STEP indicates when it thinks a problem is stiff. If it doesn't give this indication, itusually requires the fewest function evaluations. If it does, IMPSUB may requirefewer. (.

The following table gives the results for one 9articular matrix which we arbitrarilydeclare to be a "typical" nonstiff problem. The ;natrix is of order 3, with eigenvalues,.\= 3, 3, 6; the matrix is defective. We used three different local error tolerances andintegrated over [0, 1J. The average number of function evaluations for the threestarting vectors is given in the table. These can be regardeaCas typical coefficients of n 2

for the single vector problem or of n J for the full matrix exponential; IBM 370 longarithmetic was used.

TABLE 2

Work as a function of subrou/in~ and local ~"or /ol"anc~.

IO-~10-'10-12

RKF45

2178323268

DE/STEP

118160211

IMPSUB

1732021510

Although people concerned with the competition between various o.d.e. solversmight be interested in the details of this table. we caution that it is the result of onlyone experiment. Our main reason for presenting it is to support our contention thatthe use of any s~ch routine must be regarded as very inefficient. The scaling andsquaring method of § 3 and some of the matrix decomposition methods of § 6 requireon the order of 10 to 20 n 3 flops and they obtain higher accuracies than those obtainedwith 200 n 3 or more flops for the o.d.e. solvers.

814 CLEVF'MOLER AND CHARLES VAN LOAN

This excessive cost is due t.o the fJct that the programs are not taking advantag~of the linear, constant coeffiCient ~[Ure of the differential equation. They must

repeatedly call for the multiplication of various vectors by the matrix A because, as faras they know, the matrix may have changed since the last multiplication.

We now consider the various methods which result from specializing generalo.d.e. methods to handle our specific problem.

METHOD 6. SINGLE STEP O.D.E. METHODS. Two of the classical techniques for thesolution of differential equations are the fourth order Taylor and Runge-Kutlamethods with fixed step size. For our particular equation they become

h4

xl+ I == (I +hA + ... +4!A 4) Xj == T4( hA )xl

and

Xj+! ==xj+~k, +!k2+!k3+~k4'

where k, = hAx;. k2 = hA(xj + ~kd, k3 == hA(Xj +~k2)' and k4 = hA(xj + k3). A littlemanipulation reveals that in this case, the tWO methods would produce identicalresults were it not for roundoff error. As long as the step size is fixed, the matrix

T.(hA) need be computed JUSt once and then Xi+1 can be obtained from Xj with JUStone matrix-vector multiplication. The standard Runge-Kutta method would require 4such multiplications per step.

Let us consider x (t) for one particular value of I, say 1== 1. If h == 1/ m, then

x (1) == x (mh) = x," == [T4(hA )J'"xo.

Consequently, there is a close connection between this method and Method 3 whichinvolved scaling and squaring (54), (60). The scaled matrix is hA and its exponential isapproximated by T.(hA). However, even if m is a power of 2, [T4(hA )}'" is usually notobtained by repeated squaring. The methods have roughly the same roundoff errorproperties and so there seem to be no important advantages for Runge-KUlla withfixed slep size.

Let us now consider the possibility of varying the step size. A simple algorithmmight be based on a variable step Taylor method. In such a method, tWO ap­proximations to X,., would be computed and their difference used to choose the stepsize. Specifically, let £ be some prescribed local relative error tolerance and define XI ••

and X , •• I by

Xf., = T~(h,A)x,.

x,·•• = T4(h,A»)".

One way of determining hf is to require

Notice that we are using a 5th order formula 10 compute the approximalion, and a 4thorder formula to control step size.

At first glanc~. this method appears 10 be considerably less etficienllhan one with

fixed slep SIZ~ because Ih~ malnces T.(h,A) and T~(h,A) cannol be precomputed.Each step requires 5 n: flops. However. in thos~ problems which involve large"humps" as described in § I. a smaller step is needed 31 the beginning of thecomputation than at the end. If the step size changes by a factor of more than 5, thevariable step method will require less work .

.,

-'~.". -;:,

:j

THE EXPONENTIAL OF A MATRIX ./ 815

The method does provide some insight into the costs of more sophisticatedintegrators. Since

we see that the required step size is given approximately by

hi = [II~ :IIJ1/~

The work required to integrate over some fixed interval is proportional to the inverseof the average step size. So. if we decrease the tolerance £ from. say 10-6 to 10-9• thenthe work is increased by a factor of (1 03)1/5 which is about 4. This is typical of any 5thorder error estimate-asking for 3 more figures roughly quadruples the work.

METHOD 7. MULTISTEP O.D.E. SOLVER. As far as we know. the possibility ofspecializing multistep methods. such as those based on the Adams formulas. to linear.constant coefficient problems has not been explored in detail. Such a method wouldnot be equivalent to scaling and squaring because the approximate solution at a giventime is defined in terms of approximate solutions at several previous times. The actualalgorithm would depend upon how the starting vectors are obtained. and how the stepsize and order are determined. It is conceivable that such an algorithm might beeffective. particularly for problems which involve a single vector, output at manyvalues of t. large n, and a hump.

The problems associated with roundoff error have not been of as much concern todesigners of differential equation solvers as they have been to designers of matrixalgebra algorithms since the accuracy requested of o.d.e. solvers is typically less thanfull machine precision. We do not know what effect rounding errors would have in aproblem with a large hump.

5. Polynomial methods. Let the characteristic polynomial of A be"-I

c(z)=det(zl-A)=z"- ~ CkZk.k-U

From the Cayley-Hamilton theorem c(A) = 0 and hence

It follows that any power of A can be expressed in terms of !,A ..... A "-I:" -I

A • = I {3k;A'.,-0

This implies that e,A is a polynomial in A with analytic coefficients in t:

" -I== ~ a,(t)A'.

,-0

816 CLEVE MOLER AND CHARLES VAN LOA)'.;

(k ~ 2).

The methods of this section involve this kind of exploitation of the characteristic

polynomial.METHOD 8. CAYLEY-HAMILTON. Once the characleristic polynomial is known.

the coefficients f3,'1 which define the analytic functions aj(t) = ~ f3./'/ k! can begenerated as follows:

(k < n )

(Ie = n)

(k >n,j=O)

(k >n,j>O).

One difficulty is that these recursive formulas for the f3.j are very prone to roundofferror. This can be seen in the I-by-I case. If A=(a) then f3.o=a· and au(t)=L (att/ k! is simply the Taylor series for eO'. Thus. our criticisms of Method 1 apply.In fact, if at = -6, no partial sum of the series for eO' will have any significant digitswhen IBM 370 short arithmetic is used.

Another difficulty is the requirement that the characteristic polynomial must beknown. If A I, •.. ,A" are the eigenvalues of A, then c(z) could be computed fromc (z) = n; (z - Ai). Although the eigenvalues could be stably computed, it is unclearwhether the resulting Cj would be acceptable. Other methods for computing c(z) arediscussed in Wilkinson 114]. It lurns out that methods based upon repeated powers of>A and methods based upon formulas for the Ci in terms of various symmetric functionsare unstable in the presence of rpundoff error and expensive to implement. Tech­

niques based upon similarity tra;1sformations break down when A is nearly deroga­tory. We shall have more to say About these difficulties in connection with Methods 12and 13.. '

In Method 8 we attempted to expand erA. in terms of the matrices I. A ..... A "-I.

If {Ao•...• A"-d is some other set of matrices which span the same subspace. thenthere exist analytic functions {3, (t) such that

"-Ie"" = L f3j(t)A,.

,-0

The convenience of this formula depends upon how easily the Ai and f3j(t) can begenerated. If the eigenvalues A), ... ,An of A are known. we have the following threemethods.

METHOD 9. LAGRANGE INTERPOLATION.

METHOD 10. NEWTON INTERPOLATION.

ft ,-1e' ..•.=e·"JT ~ IAI.···.A,] n (A-A.!).

,- -: • - J

The divided differences IA I•...• A,] depend on t and are defined recursively by

IA I. A 2] = (e'" - e"')/ (A. - A 2).

IAI.··· .Ad-IA2.··· . A•.• .]IA •..... At..) = ----------­

A.-A •.• I

THE EXPONENTIAL OF A MATRIX 817

We refer to MacDuffee [9J for a discussion of these formulae in the confluenteigenvalue case.

METHOD 11. v ANDERMONDE. There are other methods for computing thematrices

.which were required in Method 9. One of these involves the Vandermonde matrix

If Pik is the (j, k) entry of V-I, then

"" k-IAi = i.. PikA ,k-I

and

"e'A = L eA"Ai'

I-I

When A has repeated eigenvalues. the appropriate confluent Vandermonde matrix is

involved. Closed expressions for the Pik are available and Vidysager [92] has proposedtheir use.

Methods 9. la, and 11 suffer on several accounts. They are O(n·) algorithmsmaking them prohibitively expensive except for small n. If the spanning matricesAo •.... A~_I are saved. then storage is n 3 which is an order of magnitude greaterthan the amount of storage required by any "nonpolynomial" method. Furthermore.even though the formulas which define Methods 9. 10. and 11 have special form in theconfluent case, we do not have a satisfactory situation. The "gray" area of nearconfluence poses difficult problems which are best discussed in the next section ondecomposition techniques.

The next two methods of this section do not require the eigenvalues of A and thus

appear to be free of the problems associated with confluence. However. equallyformidable difficulties attend these algorithms.

METHOD 12. INVERSE LAPLACE TRANSFORMS. If ~[e'AJ is the Laplace transformof the matrix exponential. then

~[e'AJ = (sl- A rl.The entries of this matrix are rational functions of s. In fact,

where c(s)= det (sl-A)= s" - L::~CkSk and for k = 1.' . '. n:

(Ao = I).

Hlo CLEVE MOLER AND CHARLES VAN LOAI\:

010...00

: l~OC=I· CU

c~l

I C2 ••.

C"_I

These recursions were: derived by Leverrier and Faddee:va r 3] and can bl: USl:J 10evaluate e,A:

" -Ie'A = ~ ~-I[s"-~-lle(s)IA~.

~ aU

The inverse transforms 5"-I[S"-~-1 le(s)J can be expressed as a power series in 1. Liour 102J suggests evaluating these series using various recursions involving the: e~. Wesuppress the details of this procedure because of its similarity to Method S. There arcother ways Laplace transforms can be used to evaluate e,A [78], (~O]. (1)8], (89J.193J.

By and large. these techniques have the same drawbacks as Methods 8-11. The:y areO(n 4) for general matrices and may be seriously effected by roundoff error.

METHOD 13. COMPANION MATRIX. We now discuss techniques which involve the

computation of eC where C is a companion matrix:

Companion matrices have some interesting properties which various authors havetried to exploit:

(i) C is sparse. '(ii) The characteristic polynomial of C is e(z) = z" - L::~e.z~.

(iii) If V is the Vandermonde matrix of eigenvalues of C (see Method 11). thenV-ICV is in Jordan form. (Confluent Vandermonde matrices are involved inthe multiple eigenvalue case.)

(iv) If A is not derogatory. then it is similar to a companion matrix; otherwise it issimilar to a direct sum of companion matrices.

Because C is sparse. small powers of C cost considerably less than the usual n·1

flops. Consequently, one could implement Method 3 (scaling and squaring) with areduced amount of work.

Since the characteristic polynomial of C is known, one can apply Method 8 orvarious other techniques which involve recursions with the c~. However. this is notgenerally advisable in view of the catastrophic cancellation that can occur.

As we mentioned during our discussion of Method 11. the closed expression forV-I is extremely sensitive. Because V-I is so poorly conditioned. exploitation ofproperty (iii) will generally yield a poor estimate of eA,

If A = YCy-l. then from the series definition of the matrix exponential it is easyto verify that

Hence. property (iv) leads us to an algorithm for computing the exponential of ageneral matrix. Although the reduction of A to companion form is a ralional process,the algorithm for accomplishing this an: extremely unstable and should be avoided(14J.

We mention that if the original differential equation is actually a single nth orderequation written as a system of first order equations, then the matrix is already incompanion form. Consequently. the unstable reduction is not necessary. This is theonly situation in which companion matrix methods should be considered.

THE EXPONENTIAL OF A MATRIX 819

We conclude this section with an interesting aside on computing c H whereH = (h,,) is lower Hesscnherg (h" = O. i >; + I). Notice that companion matrices arelower Hessenberg. Our interest in computing eH stems from the fact that any real

matrix A is orthogonally similar to a lower Hessenberg matrix. Hence. if

A = OHOT•

then

eA = Or HOT.

Unlike the reduction to companion form. this factorization can be stably computedusing the EISPACK routine ORTHES [113J.

Now. let fk denote the kth column of e H. It is easy to verify that

Hfk = ~ hilJ;;2" _I

(k ~ 2).

by equating the kth columns in the matrix identity H eH = eHH. If none of thesuperdiagonal entries hk-U are zero. then once f" is known. the other fk followimmediately from

fk-I =_1_[ Hfk -£ hidJhk-I.k ,-k

Similar recursive procedures have been suggested in connection with computing c C

/1041. Since f" equals x(l) where x(t) solves Hx =).:. x(O)=(O.··· .0. I)T. it could befound using one of the o.d.e. methods in the previous section.

There arc ways to recover in the above algorithm should any of the hk-U bezero. However. numerically the problem is when we have a small. but non-negligiblehk-U. In this case rounding errors involving a factor of l/hk-u will occur precludingthe possihility of an accurate computation of rHo

In summiJry. methods for computing rA which involve the reduction of A tocompanion or Hessenberg form are not attractive. However. there arc other matrixfactorizations which can be more satisfactorily exploited in the course of evaluating cAand these will be discussed in the next section.

6. Matrix decomposition methods. The methods which are likel\' to he mostctlicient for prohlems involving large matrices and repeated e\'aluati~n of r'A arethm.c which are hased on factorizations or decompositions of the m;ttrix A. If Ahappens to be symmetric. then all these methods reduce to a simple very effectivealgorithm.

All the matrix decompositions are based on similarity transformations of the form

A=SBS-'.

As we have mentioned. the power series definition of r,A implies

C'A = S C'"S-I.

The idea is to find an S for which C,II is easy to compute. The difficulty is that S may beclose to Singular which means that cnnd (S) is large.

METHOD ) 4. EIGENVECTORS. The naive approach is to take S to be the matrixwhose columns are eigenvectors of A. that is. S = V where

V = r vII· .. Iv" 1

-

820

and

CLEVE MOLEK. ANO CHARLES VAt-.: LOA/\:

j=l.···.II.

Th~s~ II equations can be writt~n

A V = YD.

where D = diag (A J. ••• , A"). The exponential of D is trivial to compute assuming Wehave a satisfactory method for computing the exponential of a scalar:

e'V = diag (eA", •••• eA-').

S· V' . I h ," V 'V V-ImCe IS nonsmgu ar We ave e = t' .

In terms of the dilfen:mial equation i = Ax, the same eigenvector approach takesthe following form. The initial condition is a combination of the eigenvcctors.

"x{O) = L CliV"

I-I

and the solution x{r) is given by

"x(r)= ~ u,e4,'v,.

I"""U

Of course, the coefficients aj are obtained by solving a set of linear equationsVa = x(O).

The difficulty with this approach is not confluent eigenvalues per se. For example.the method works very well when A is the identity matrix, which has an eigenvalue ofthe highest possible multiplicity. It also works well for any other symmetric matrixbecaUSe the eigenvectors can be chosen orthogonal. If rc:liable subroutines such asTRED2 and TQL2 in EISP ACK [113] are used. then the computed V, will beorthogonal to the full accuracy of the computer and the resulting algorithm for etA hasall the attributes we desire~xcept that it is limited to symmetric matrices.

The theoretical difficulty occurs when A does not have a complete set of linearlyindependent eigenvectors and is thus defective. In this case there is no invertibkmatrix of eigenvectors V and the algorithm tm:aks down. An example of a ddectiv~matrix is

A defective matrix has confluent eigenvalues but a matrix which has confluent eigen­values need not be defective.

In practice. ditticulties occur when A is "nearly" defective. One way to makc thisprecise is to use the condition number. cond (V) = IIVilli V- 'II. of the matrix of eigen­vectors. If A is nearly (exactly) dekctive. then cond (V) is large (infinite). Any errorsin A. including roundoff errors in its computation and roundoff errors from theeigenvalue computation. may be magnified in the final result by cond (V).

Consequently. when cond (V) is large. the computed etA will most likely be inac­curate. For example, if

-\

THE EXPONENTIAL OF A MATRIX

then

D = diag (I + Eo 1 - d.;'Ind

cond (V) = 0(;).If £ = 10-" and IBM 370 short floating point arithmetic is used to compute theexponential from the formula eA = VeDV-I• we obtain

[2.71H307() 2.7500001­2.71 R254

Since the exact exponential to six decimals is

2.7182H2]2.71R255 .

we see that the computed exponential has errors of order 10' times the machineprecision as conjectured.

Onc might feel that for this example rA might be particularly sensitive toperturbations in A. However. when we apply Theorem .3 in § 2 to this example. wefind

II fA·E1 Allr - r '<: 411EII 2"£1'IIrAl1 - r.

independent of £. Certainly. rA is not (~verly sensitive to changes in A and so Method)4 must he regarded as unstahle.

Before we procecd to the next mcthod it is interesting to notc the connectionbetwecn the use of eigenvectors and Method 9. Lagrange interpolation. When theeigenvalues are distinct thc cigcO\'cctor approach can he cxpresscd

("'" = \. ctiag (rA,')\ .-, = ~ ("A,'t·,\·,T.,- ,

where )",T is the jth row of V-'. The Lagr;'lnge formula is

("U, = ~ ("A,'A •.•- I

where

A, = 11 (A - A I /)• -, (A, - A ) .•• "'t .•

Becausc these two expressions hold for all t. the individu;)J terms in the sum must bethe same and so

S2:? CLEV!'. MOLER AND CHARLES VAN LOA1'.

This indicates that the AI are, in fact, rank one matrices ohtained from th.: .:ig.:n­vectors. Thus, the O(n"') work involved in the computation of the A, is totallyun necessary.

METHOD 15. TRtANGULAR SYSTEMS OF EIGENVECTORS. An improvement inboth the efficiency and the reliability of the conventional eigenvector approach can heobtained when the eigenvectors are computed by the OR algorithm (14 J. Assumetemporarily that although A is not symmetric, all its eigenvalues happen to be real.The idea is to use EISPACK subroutines ORTHES and HQR2 to compute theeigenvalues and eigenvectors (] 13]. These subroutines produce an orthogonal matrixo and a triangular matrix T so that

Since 0-I = 0T, this is a similarity transformation and the desired eigenvalues OCl:uron the diagonal of T. HQR2 next attempts to find the eigenvectors of T. This results ina matrix R and a diagonal matrix D, which is simply the diagonal part of T, so that

TR = RD.

Finally, the eigenvectors of A are obtained by a simple mauix multiplication V = OR.The key observation is that R is upper triangular. In other words, the

ORTHES/HQR2 path in EISPACK computes the matrix of eigenvectors by firstcomputing its "OR" factorization. HQR2 can be easily modified to remove the finalmultiplication of 0 and R. The availability of these two matrices has two advantages.First', the time required to find V-lor to solve systems involving V is reduced.However, since this is a small fraction of the total time required, the improvement inefficiency is not very significant. A more important advantage is that cond (V) =cond (R) (in the 2-norm) and that the estimation of cond (R) can be done reliably andefficiently.

The effect of admilting complex eigenvalues is that R is not quite triangular. buthas 2-by-2 blocks on its diagonal for each complex conjugate pair. Such a matrix iscalled quasi-triangular and we avoid complex arithmetic with minor inconvenienl:e.

In summary, we suspect the following algorithm to be reliable:(1) Given A, use ORTHES and a modified HQR2 to find orthogonal 0, diagonal

D. and quasi-triangular R so that

AOR = ORD.

(2.) Given Xu, compute Yu hy solving

Ry" = 0 Tot".

Also estimate cond (R) and h.:nce the accuracy of yo.

(3) If cond (R) is too large, indicate that this algorithm cannot solve the problemand eXII.

(4) Given I, compute x (I) by

X(I) = Ve'P}'o.

(If we want to compute the full exponential. then in Step 2 we solve R Y = 0 T for Yand then use e,A = V <,'Dy in Step 4.) It IS important to note that the first three stepsare independent of I, and that the fourth stcp. which requires relatively littk work, canbe repeated for many valul:s of I.

We know there:: are exampks where the exit is takt:n in Step 3 even though theunderlying problt'm is not poorly conditioned implying that the algorithm is unstable.

THE EXPONENTIAL OF A MATRIX

Nevertheless. the algorithm is reliable insofar as cond (R) enables us to assess theerrors in the computed solution when that solution is found. It would be interesting tocode this algorithm and compare it with Ward's scaling and squaring program forMethod 3. In addition to comparing timings, the crucial question would be how oftenthe exit in Step 3 is taken and how often Ward's program returns an unacceptablylarge error hound.

METHOD 16. JORDAN CANONICAL FORM. In principle. the problem posed bydefective eigensystems can be solved by resorting to the Jordan canonical form (JCF).If

is the JCF of A. then

e,A = p[ctl'Ej1" ·Ej1etl']p-I.

The exponentials of the Jordan hlocks Ji can be given in closed form. For example. if

The difficulty is that the JCF cannot be computed using floating point arithmetic.A single rounding error may cause some multiple eigenvalue to become distinct orvice versa altering the entire structure of J and P. A related fact is that there is no apriori bound on cond (P). For further discussion of the difficulties of computing theJCF. see the papers by Golub and Wilkinson r 11OJ and Kagstrom and Ruhe rIll J.

METHOD 17. SCHUR. The Schur decomposition

A = OTO T

with orthogonal 0 and triangular T exists if A has real eigenvalues. If A has complexeigenvalues. then it is necessary 10 allow 2-hy-2 hlocks on the diagonal of T or tomake 0 and T complex (and replace 0 T with 0*), The Schur decomposition can becomputed reliably and quite efficiently hy ORTHES and a short-ended version ofHQR2. The required modifications arc discussed in the EISP ACK guide r 113].

Once the Schur decomposition is available.

e,A = 0 e ,T0T.

The only delicate part is the computation of r,T whcre T is a triangul;1r or quasi­triangular matrix. Note that the eigenvcctors of A are not required.

Computing functions or triangular matnces is the subject of a recent paper byParlett r 112J. If T is upper triangular with diagonal clements A \ •...• Aft. then it isclear that e,T is upper triangular with diagonal clements e~I' •...• eA.'. Parlett showshow to compute the off-diagonal elements of e,T recursively from divided differencesof the eA". The example in § 1 illustrates the 2-by-2 case.

CLEVE MOLER Ar-.;D CHARLES VAN LOAr-.;

Again. the diHiculty is magnification of roundoff error caused by nearly contluenteigenvalues A,. As a step towards handling this problem, Parktt describes a general­ization of his algorithm applicable to block upper triangular matrices. The diagonalblocks an: determined by clusters of nearby eigenvalues. The cont1uence problems donot disappear, but they are confined to the diagonal blocks where special techniquescan he applied.

METHOD 18. BLOCK DtAGqNAL. All methods which involve decompositions ofthe form

A = SBS-'

involve two conflicting objectives:(I) Make B close tn diagonal so that e,H is easy to compu":.(:2) Mal..: Swell condition..:LI sn that errors arc not magnilieLi.

The Jordan canonical form places all the emphasis on the first objective. whik theSchur decomposition places most of the emphasis on the second. (We would regardthe decomposition with S.= I and B = A as placing even more emphasis on the secondobjective.)

The block diagonal method is a compromise between these two extremes. Theidea is to use a nonorthogonal, but well conditioned, S to produce a B which istriangubr and block diagonal as iIIistrated in Fig. 2.

B=

FIG. 2. T"un/(ular hlock Jia/(ollul form.

Each block in B involves a cluster of nearly confluent eigenvalues. The numher ineach cluster (the size of each block) is to be maLie as small as possihk while maintain­ing some prescribed upper bound for cond (S). such as cond (S)< IOU. The choice ofthe bound 100 implies roughly that at most 2 significant decimal figures will be lostbecause of rounding errors when e ,..•.is obtained Irom f,H via e,A = S e'~ S-I. A largerbound would mean the loss of more figures while a smaller bound would mean morecomputer time-both for the factorization itself and for the evaluation of e'~.

In practice, we would expect almost all the blocks to be I-by-l 'or 2-by-2 and theresulting computation of e'H to be very fast. The hound on cond (S) will mean that it isoccasionally necessary to have larger blocks in B, but it will insure against excessiveloss of accuracy from confluent eigenvalues.

THE EXPONENTIAL OF A MATRIX

G. W. Stewart has pointed out that the grouping of the ei/!envalues into clustersand the resulting olock structure of B is not merely for increased speed. There can oc;10 important improvement in accuracy. Stewart suggests expressing each block B, inthe form

Bi = y;l +E,

where y, is the •.•verage value of the eigenvalues in the jth cluster. If the grouping hasneen done properly. the matrices Ei should then be nearly nilpotent in the sense thatE,' will rapidly •.•pproach zero •.•s k increases. Since E, is triangular. this will certainlybe true if the di •.•gonal part of E, is small. that is. if all the eigenvalues in the cluster areclose together. But it will also be true in another important case. If

E, = [V;

where f is the 'computer rounding unit. then

~]c•.•n be regarded as negligible. The ±J; perturbations are typical when a double.defective eigenvalue is computed with. say. HQR2.

The fact that E, is ne •.•rly nilpotent means that e,R, can be found rapidly andaccurately from

e ''', = ('",'e 'F., :

computing e'F., by a few terms of the Taylor series.Sever •.•' researchers. including Parlett. Ruhe .•.•nd Stewart. •.•re currently develo­

ping computer progr •.•ms n<1sed on some of these ide •.•s. The most dillicult detail is theproper choice of the eigenv •.•lue clustering. It is also important for program etliciencyto avoid complex arithmetic as much as possible. When fully developed. these pro­grams will be fairly long and complicated but they may come close to meeting ourother criteria for satisfactory methods.

Most of the computational cost lies in obtaining the oasic Schur decomposition.Although this cost varies somewhat from m •.•trix to matrix oecause of the iterativenature of the OR algorithm. a good <1\'er<1gefigure is 15 n' nops. including the furtherreduction to hlock diagonal form. Ag •.•in we emph<1si7.e th •.•t the reduction is in­dependent of I. Once the decomposition is oht<1ined. the calculation of r,A requiresanoul 2" , nops for each t. If we require only x(t) = r,AXn for various t. the equation51" = XII should he solved once at a cost of n-'j3 flops. and then each x(r) can beootained with n: flops.

These arc rough estimates. There will be differences between programs based onthe Sehur decomposition and those which work with the hlock diagonal form. hut thetimings <'1nuld he similar because Parlett's algorithm for the exponenti<ll is very fast.

7. Splittin~ methods. A most •.•g/.!ravating. yet interesting. property of the matrixexponential IS that the familiar additive law fails unless we have commutivity:

Nevertheless. the exponentials of Band C are related to that of B + C. for examrle.

CLI:Vf MOLEH ,\ND CHA/{L!,S VA'" l.OAI'

by the Trotter product formula /3U):

eBTr = lim (eBlm eOm)",.",-co

METHOD 19. SPLITTING. Our colleagues M. Gunzburger and D. Gottleib sug­gested that the Trotter result be used to approximate eA by splitting A into B + C andthen using the approximation

This approach to computing eA is of potential interest when the exponentials of BandC can be accurately and efficiently computed. For example. if B = (A + A T)(2 andC = (A -A T)f'2 then eH and eC can be effectively computed by the methods of * 5.For this choice we show in Appendix 2 that

(7.1 )

where JL (A) is the log norm of A as defined in § 2. In the following algorithm. thisinequality is used to determine the parameter m.

(a) Set B = (A +A T)(2 and C = (A -A T)/2. Compute the factorization B =o diag (JL,)O T (OrO = 1) using TRED~ and TQL:? 1113]. Variations ofthese programs can be used to compute the factorization C = UDUT whereU'rU = I and D is the direct sum of zero matrices and real 2-by-2 blocks ofthe form

corresponding to eigenvalues ±ia.(b) Determine In = 2' such that the upper bound in (7.1) is less than some

prescribed tolerance. Recall that JL (A) is the most positive eigenvalue of Band that this quantity is known as a result of step (a).

(c) Compute X = 0 diag (e ••/m)O T and )' = V eDlm V T. In the latter compu­tation, one uses the fact that

exp [ -a~m (J 1III] = [ cos (01 m )o -sin (01 m ) sin (al m)]cos (01 m )

.. ~, A(d) Compute the approxlmallon, (X}')- , to e by repeated squaring.If we assume 5 n) flops for each of the eigenvalue decompositions in (a), then the

overall process outlined above requires about (13 + j) n J flops. It is difficult to assessthe relative efficiency of thiS splitting method because it depends strongly on thescalars IliA T, A III and JL (A) and these 4uantilies have not arisen in connection with anyof our previous eighteen methods. On the baSIS of truncation error bounds, however,it would seem that this technique would be much less efficient than Method 3 (scalingand squaring) unless JL(A) weft: negative and iliA T, AlII much kss than IIAII.

Accuracy depends on the rounding errors which arise in (d) as a result of therepeated squaring. The remarks about repeated squaring in Method 3 apply also here:there may be severe cancellation but whether or not this only occurs in sensitive e'"

problems is unknown.

(7.2)

THE EXPONENTIAL OF A MATRIX

For a general splitting A = 8 + C. we can detcrmine 1M from the inequality

II A_( Rim Clm)'"II$lIrB. CHI nlJn"IICIIe e e - 2m e .

which we estahlish in Appendix 2.To illustrate. suppose A has companion form

If

B=[~ I"c;']

T T Tand C = enc where c = (co.' ... Cn-I) and en = (0. O.... ,0,1). then

eRI ••• = nt [Br~k-fl m k!

and

Notice that the computation of these scaled exponentials require only D(n2) Rops.Since 11811= 1.IICII = l!cll. and III B. Cl1l2 21\c1l. (7.2) becomes

The parameter m can he determined from this inequality.

8. Conclusions, A section called "conclusions" mu<;t dcal with the obvious ques­tion: Which method i<;best'> Answering that qucstion is very risky. We don't knowenough ahout the sen<;itivity of the original prohlem. or ahout the detailed per­formance of careful implementations of v,nious m'ethods to make any firmconclusions. Furthermore. hy the time this paper appcars in the open literature. anygiven conclusion might well have to he modified.

We have considered five general c1as<;es of methods. What we have called poly­nomial methods are not really in the competition for "hest". Some of them require thecharacteristic polynomial and so are appropriate only for certain special problems andothers have the same stahility difficulties as matrix decomposition methods but aremuch less efficient. The approaches wc have outlined under splitting methods arelargely speculative and untried and prohably only of interest in special settings. Thisleaves,three classes in the running.

The only generally competitivc series method is Method 3. scaling and squaring.Ward's program implementing this method is certainly among the best currentlyavailable. The program may fail. but at least it tells you when it does. We don't knowyet whether or not such failures usually result from thc inherent sensitivity of the

CLEVE MOLl:1< AND CItAI<LES VAN LOAN

problc:m or from thl: instability of th~ algorithm. The method basicalh' compul':~ ('''for a single: matrix A. To compute e,A for p arbitr::ry values of I require~ about p timl:sas much work. The amount of work is O(n'\ eve:n for the vector problc:m t""xu. Thecoetticlent of n:' incr~ases as IIAII increases.

Specializations of o.d.e. methods for the eA problem have not yet been im­plemented. The best method would appear to involve a variable: ordc:r. variabh: stl:pditkn:nce schc:me. We suspect it would be: stable: and rdiablc: but I:xpensivc:. Its oe:stshowing on efficiency would be for the vector problem t',A XU with many values of t

since the amount of work is only O(n :!). It would also work quite well for vector

problt:ms involving a large sparse A since no "nonsparsc" approximation to theexponential would be explicitly required.

The best programs using matrix decomposition mcthods arc just now b.:ingwrille:n. They start with the: Schur de:composition and include some: sort of .:ige:nvalul:cluste:ring. The:rl: are: variants which involve further reduction to a block form. In allcases the initial decomposition costs O(n:l) steps and is independent of I and IIAII.After that, the work involved in using the decomposition to compute t',AXU fordifferent I and Xo is only a small multiple of n:!.

Thus, we see perhaps three or four candidates for "best" method. The choice willdepend upon the details of implementation and upon [h'e particular problem beingsolved.

Appendix 1. Inverse error analysis of Pade mlltrix approximation.LEMMA 1. If IIH" < 1. Ihm log (1 + H) exiSb and

<: IIHIIIllog (/ + H~I = .,--..'

Proof If IIHII< 1 then log (/ + H) = ::::= I (-I )~+I(Hk Ik) and so

IIlog(l+H)II~ ~ IIHII~ ~IIHII ~ IIHII~ = IIHII .k ~ I k 1- II I -IIH II

LEMMA 2. If IIAII ~ ~ and p > O. (hen IID,-.(A fill ~ (q +p )/p.Proof From the definition of D....,(A) in * 3. D",,(A) = 1+ F where

". (P+l{-j)!cl! (-A)'F=~ I ,,-,-,-.

I' I lp .,. l{ ). (1/ - I), I,

Using the fact that

we find

~ [II ]' I If ciIIFII~~ -IIAII -~-lIAllk-I):5-

I - I 1'''' cf I' II .,.II P .,.ci

and so IIDpq(Aflli = 11(1'" Ff III~ II( I -IIFII) ~ (q'" p)/p.LEMMA 3.If jlAIl~S. 4 ~ p. alld p ~ 1. (ilt'" R....,(A) = e"+F where

THF EXPONENTIAL OF A MATRIX 829

Proof. From the remainder theorem for Pade approximants [7 I].

Rpq(A)=eA_,(-I)q A,,+q+IDt>q(Ar' IIell-U)Au"(I-utdll.p + q)! 0

and so e-A Rt>q(A) = I + H where

H=(-I),,+IA,,+q+IDt>q(A)-1 II e-UAll"(I-utd/i.p + q)! n

By taking norms. using Lemma 2. and noting that (p + q)/ p e'~ ~ 4 we ohtain

IIHII~ 1 IIAlIp+q+IP+q r e'~uP(l-u)qdu(p+q)! p n

I I~4I1AIIP"Q"1 p.q.

(p+q)!(p+q+ I)!

With the assumption IIA II~ ~ it is possible to show that for all admissable p and q.IIHII ~ ~and so from Lemma I.

1110 (I+H)II~ IIHII ;a8I1AIIP+Q+1 p!q! .g l-IIHII (p+q)!(p+q+l)!

Setting F = Jog (I + H). we see that e -AR",,(A) = I + H = e F. The lemma now followshecause A and F commute implying Rt>q(A) = eA eF = eA+F.

LEMMA 4. IfllAII~~ then Rpq(A)=eA+F where

p!q!

IIFII ~ 8I1AII,,+Q+1(p +q)!(p +q + I)!

Proof. The case p :;;q. p :;; I is covered hy Lemma I. 1f p +q = O. then F = - A andthe above inequality holds. Finally. consider the case q >p. q ~ 1. From Lemma 3.Rq,,(-A) = e -A+F where F satisfies the ahove hound. The lemma now follows becauseII-FII=IIFII and R",,(A)= [R",,(-AW' = re-A+Frl = eA-F.

) "" A+ETHEOREM A. I. IfilAIi/2' ~ 2. then [Rpq(A/2')j" = e where

Pmof. From Lemma 4. R,.,{AI2') = r"'·F where

IIFII~R[II;'lIr+"·1 p!q'

The theorem follows by noting that if E = 2'F. then

COROLLARY I. If IIAII/2' ~~. then [Tt(A/2')f = eA+E where

bJu CLEVE MOLEH AND CHAHLES VAN LOA!\:

II I I [ ,]" MECOROLLARY 2. If A 112' ~~, rhell R~(AI2) - = e, whert'

IEls8("Allf'l. (q!)~ «!)~'I-J (q!)~IIAII- 2' (20 )!(20 + 1)! - 2

Appendix 2. Accuracy of splitting techniques. In this appendix we: dt:rivc: the:inequalities (7.1) and (7.2). We assume throughout that A is an lI-bY-II matrix anJthat

A=B+C.

It is convcnient to define the: matrices

Snl = eA/"',

and

T. - 8/". Clft!III -e e ,

where m is a positive integer. Our goal is to bound liS::: - T:::II. To this end we shallhave to exploit thc following properties of the log norm ~ (A) defined in § 2:

(i) lie 'All ~ e ••(A)I (r ~ 0)

(ii) ~(A)~IIAII

(iii) ~(B + C)~ ~(B)+IICII.

These and other results concerning log norms are discussed in references (35 j-[ 42 j.LEMMA 1.If8~max{~(A).~(B)+~(C)} then

liS::: - T:::II~ m et1lm-ll/mIlS •••- 7:"11.

Proof. Following Reed and Simon [11] we havem-I

S::: - T::: = L S~(Sm - Tm)T:-1-k.k -0

Using log norm property (i) it is easy to show that both IIS...II and IIT •••II are boundedaboY\: by e9/m and thus

,,,-1liS:::- T:::II~ ~ IISmIlAIIS...- T".IIIITmll"'-H

k -u•••-1

~IISm-Tmll L eHklmeH, •••-I-kl/mk-U '

from which thc lemma immediately follows.In Lemmas 2 and 3 we shall make usc of the notation

,'-',F(r) I,_,u =F(rd-F(ro),

whcre F(r) is a matrix whosc entries are functions of r.LEMMA 2.

I

T. -S = J. 'BI"'[ (I-'IA/'" ~C] ,CI •••d••• m t e • e r.o m

THE EXPONENTIAL OF A MATRIX

P f W h T S ,nl ••• fl-tlA/ ••• tCI.., I'~I d hroo. cave •••- •••=c (' C t~nan t us

T ••• -S..,= r {;retRI"'eCl-tlA/"'erCl"'J}dt.

The lemma follows since

d [ ,If I ••• II-tlAI ••• tCI"'J tl/I"'[ CI-tlA/", 1 C] tCI",- C C C =e (' .- c .~ m

LEMMA 3. If X alld Y arc malriccs Ilzcn

Proof. We have fcx. YJ = e'xy ell-nx I::~and thus

frx. YJ= In' {~fc,xYCII-')XJ} dl.

Since d/dl(e'xy eO-tlxl = e'x[X. Yl eCl-nx we getI

IIfcx. Y11I~ f IIe,xllllrx. YJIlIIe(l-nxUdt(I

I

~lIrx. Ylil f e"lX)l e"IX)(I-tI dt(I

from which the lemma immediately follows.THEOREM A.2. If 0;;; max {p.(A). p.(B)+ J.L (C)}. then

liS:;: - T:;:II~ 2~ eHIlfB. CJII.

831

Prodf. If 0 ~ I ~ 1 then an application of Lemma 3 with X == (1-/)A/m andy == C/ m yields

II! C (!-rIAl •••• C/m JII~ e "fA )(1-1)/"'11[(1 - t)A/ m. C/ m JII

~cAO-')I •••(1-/)IIfB. CIII.,"

By cour1ing this lnequ;Jlity with Lemma ~ wc can hound liT ... - S•••II:I

liT ••• - S",II~ f IIc'RI •••llllleCl-tIA/ •••. C/m JIIlie tC/"'1I dtn

The theorem follows by comhining this result with Lemma 1.COROLLARY 1. If B = (A + A *)/2 and C = (A - A *)/2 then

1liS:;: - T:;:II ~ - c"rA~I!A *. A III.4m

CLEVE MOLEH AND CHARLES VAl" l.UAI"

Proof Since I..L(A)=J.L(B) and J.L(C)=O, we can sel 0=J.L(A). Th..- carollan I~

~slablish~d by nOling Ihal IB, C] = MA", A).COROLLARY 2.

liS: - T:II~~e ••(H)""CUllrB, C)II;;:~e"BIH'illirB. Cll!._III _ff/

Proof. max {J.L(A). J.L(B)+J.L(C)}~J.L(B)+IICII;;:IIBII+IICII.

Acknowledgments. W~ have greatly profited from the comm~nts and sugg~slionsof so many people that it is impossible to mention them all. How~ver, we are

particularly obliged to B. N. Parlett and G. W. Stewart for their v~ry p~rcepliwremarks and to G. H. Golub for encouraging us to write this paper. We are obliged toProfessor Paul F~derbush of the University of Michigan for helping us with theanalysis in App~ndjx 2. Finally, we would like to thank the rder~~s for theirnum~rous ami mosl hc.:lpful comments.

REFERENCES

Background.f 1I R. BF.LLMAN, Introduclion 10 Matrix Analysis, McGraw-Hilt, New Yorl, 1969.

[21 C. DAVIS, Explicil funcliunul culculus, J. Line:ar Algebra Al'pl., 6 (I (73), PI'. 193-199.131 V. N. FADDEEVA, Cumpllluliollal MC'lhods of LinC'ur AIIIC'bra, Dover Publica lion,. Nc:w Yorl, 1959.

14) G. E. FORSYTHE, M. A. MALCOLM AND C. B. MOLER, CompulC'r MC'llwdJ fur Math.."IlIIIW!Computations. Prentice-Halt. Englewood Clifts. NJ. 1977.

15J J. S. FRAME. Matrix func/iulls "nd Upp/'L'U/iuIU, part 1/: FunCliollJ uf matrias. IEEE Spcctrum. I(April, 19(4). PI'. I02-JOIi.

16J ---, Matrix funclionJ Wid upplicU/iotls, purt IV: Matrix funclions und cUIIJ/il/l ••", mutrlCC's. Ihid .. I(June, 19(4), 1'1'.123-131.

[7) ---, Mutrix limc/iut!> I/Ild applica/iolls, purt V: Si",ilarilY r••duL·/iutls hy rUliullU! ur (t"IIU~()tIU!mUlrias. Ihid .• I (July. 1964), PI'. IU3-1 10.

18) F. R. GANTMACHER, Th<' Tht'ory of Mamas. Vols. I and 11, Chelsca Publishing Cu .. Nc:w Ynrk.1959.

(91 C. c. MAcDuFFEE. The Th<,ury of MatriaJ. Chd,ca. Nc:w York, 1956.

(lOl L. MIRSKY, An InlroduCllon 10 Lmear AIK~bm, Oxford Universily Prc:,s, London. 1955.II II M. REED AND R. SIMON, Funclwnul Analysis. Acadcmic Pres" New York, 197:!.

1121 R. F. RINEHART, Ti,t' t'I/IIivul<'IIa uf dt'fillillum oj a matrix fUllc/iut., Amcr. Malh. Munthly, 6:!(1955), PI'. 31)5~14.

(131 P. c. ROSENIJI.OOM. Huutlds uti funoiullS of "'.,IrI •.••I. IbiJ., 74 (11)67). PI'. 1J:!U-IJ:!h.

(141 J. H. WILKINSON. The AIx ••hrUlc EI~""VQlu •• Pruhlt''''. Oxford Univcr,ity Prc:", Oxlnrd. 11J05.

Properries and represeflfalions.

(151 T. M. ApOSTOL. Sume apliol furlllulas for tht' mUlrlx apunC'tl/ial. Amc=r. Math. Murllhl,. 70 (!lJolJ).pp. :!!!4-29:!

1161 R. W. ATHf.H.TON AND A. E. Dr: GANCE. all th<, •.valualwII of rht' durval/vt' of th" mll/rLl

expono"iu! fUtlrtloll. IEEE T/am. AUlOmailC Control. AC ..2U (1975). PI'. 7U7-711l';.117) A. URADSH,\W. TIlt' "'~"'I>"UCl"''' uf ,allll'!.· .1,1/" 'ptt'm' •••.t/h olll/III<,n/ t'11:t'tll',liut'I. Intcrnal. J.

Syslc:m, So .. :; (l1J74). 1'1'. oU7-hIJ.

IIHI R. HELLMAN. POlurhu/",n T ••chllu.Jut's /II l'.1u/ht'tnul/n. Phy>in. allJ Enl:/IIa"n~. Huh. Rlnchart.and WlnslOn. New York. 11J04.

f 191 J. c. CAVENDISH. On Iht' norm of Q mUlrlx a1"III •.nt"' !.Ihis Review. 17 (1975). 1'1'.174-175

f:!u) C. G. CULLEN. R.."wrks Oil C/lml'lIl/11~ ,,"'. IEEE Trans AUlOmall<: Conlrol. AC .. lo (1971). PI'.1J4-95.

I:! I) F. FER, Rt'sulul/on tie ['C'qual/utl mUlrlull" JVI.1/ = pV pur prudui/ infini d·apon.."I/t'II •.,. Acad. Roy.Belg. Bull. CI. S':I.. 44 (195tiJ,l'p XII,I-X:!I)

(22) E. P. FULMER. Cumputal/un of /ht' matrIX apoMtI/ial, Amer. Math. Monlhly, H2 (1975), PI'.156-159.

THE EXPONENTIAL OF 1\ MATRIX

1~3113. KAr.~TROM. Bnw,d.' and pnlmhal/on hnunds for Ihl' malr;x rxponrnl;a/. BIT. 17 (1977).1'1'.39-57.

124] T. KIITO. Pnlmhal;nn Thrnry for Linl'nr Oprralors. Ch~l'. 9. Sprinl!er-Verlaj:. New Yorl... Il)ftft.1251 R. n. KIRCHNER. An l'x,,/ic;1 form II/a fnr l' A'. Amer. Math. Monthly. 74 (19£>7). PI'. 12011-/204.

12£>1H. O. KREISS. Uhl'r Malr;:l'n d,l' hrschrankll' Ha/h1(n1pprn rrUII1(l'. Math. Scand .. 7 (J959). 1'1'.71-RO.

127] D. L. POWERS. On Ihc I"/i:cn'lructurc nf II.I' malrir cx"nnrnl;a/. Internal. J: Sv~tcms 5cl.. 7 (' 97(,).1'1'.72.1-725.

12R I E. J. PIIT7F.R. A l'o;din1( Ihr )ordnn C{//JOn;ca/ form ;n Ihl' d;.'clI.u;nn nf /inl'ar .<v.'lrnr' ""11. Clln.'I{//1l("()cfl,C/ml.<, Amer. M:llh. Monlhl\'. 73 (19ftft). 1'1'. 2-7.

129} N. M. RwJ'. "'forc ex,."cil fo",,,,/a, fnr Ihr rtponrnl;a/ mntrix. Queen's Malhcm~t'cal Rel'nnls no ..1970--21. Queen'~ Unive~ity. KinJ:!'ton. Ontario. 1970.

(30) H. F. TROrrER. ProdUCI of srm;1(roll",' of opt'ralors. Proc. Amer. Malh. Soc .. 10 (1959). PI'. 545-551.(31) C. F. V AN LOAN. A sllldv of thr malrix cIponrnl;a/. Numerical AnalY!'is Report no. 7. Department of

M~lhemati~. Unive~ily of Manchesler. Manche~ter. Enj:land. 1975.

r 32) ---. Till' .,rn.';I;V;/I' of Ihr mntrix rxponrnl;a/. SI AM J. Numer. Anal.. 14 (1977). I'fI. 971-9X I.

[331 G. H. WEISS AND A. A. MARAnUntN. Thr Bnkrr-Hml.<dorff fnrmu/a and a prnhlrm ;n cn''<la/,.hv<lr<, J. M;'Ith. ;'Ind Phvs .. 3 (1902). flfI. 771-777.

1341 A. D. ZIEnUR. On drlcrm,","" Ihr .<1",Clllrr af A h\· ann/\·z;nli: r A'. this Review. 12 (1970). flfI.9X- I02.

Lo~ norms and .Habilit)'.1351 W. A. COPPEL. Slah;/iI\' nnd Asymplol;c Brl,av;nr of D;/frrrnl;a/ Equal;ons. D. C. Heath. Boston.

19(,5.

[301 G. D."HLOUIST. Slah;"I}' and I'"or hound.' ;n Iht' n"m('rica/ ;nl('1(rat;on nf nrd;nary d;/frrt'nl;a/

t'qual;on.<. Tran~actions of thc Ro\'alln~,itute of Technoloj:}'. no. ] 30. Siockhoim. Sweden. 1959.1371 C A. DFSOF.R liNn H. HIINF.DA. Till' mramrr of a malrix 0.' a 100/10 nna/v:(' cnmplllrr a/1(orilhm.,

fnr C/rruil ann/v.';.'. IEEE Trans. Circui, Thcory. 0- 19 (1972). pfl. 4XO-JXti.13R] E. DEUTSCH. On mntr/\ norm.' and /o/1ar;lhm;r norm.'. Numer. M;'Ith .. 24 (1975). 1'1'. 49-5 I.

1391 c. V. PliO. L01(ar;lhm;c drr;valll'C'.' of a squarc malr;x. J. Linear All!ebra Appl .. ti (1973), PI'.159-IM.

f4nl ---. A furtht'r rrmnrk on Ihr /ol!nrilhm;c drr;val;vrJ of a JquarC' malrix. It-id .• 7 (1973). PI'.275-2n.

14 II T. STROM. M;n;m;zntinn nf nnrmJ antl/n/1arill,m;c nnrrru "\' t/;a1("nn/ ,,;m;/anl;i'''. Computinj:. 10(1972). PI'. 1-9.

I~:I---,0" /o/1nr;lhm;r dcr;I'(/I/I'rJ. SIAM J. Numer. An:!!.. I ~ (1975). flfI. 741-75.1.

Survey articles.143) M. HEll LEY. Slud\' of mrthndJ of cnm,.Ullnc Iranslllnn mamas. Proc. IEEE. 120 (1973). PI'·

9n5-1) I 2.

1441 c n. Mnl r.R, f);(firlll"r, ,,, ro""""",,, Ilrr "'f'<rnrtlllni of " IIInln',. flrocel'<linl:' of ,he SecondUSA-hpan COml''''l'I Cnnkrt·ncl·. 1\.1'.1 ('.S .. Monlvak. N.I. 11)7~. flfI. 71)-~2.

T""I('alcd Tn)'lor serif'S.1451 L. Y. BI\HIIR ANn A. K. SINIII\. Alnlrll rtf'<,nrnl/{// ",."",ach 10 .!"nam;c rr.'pon.'r. Computers and

Structures. 5 11975), 1'1',151)-1(,'

I~(,IT. 1\. RWt.;IIRT. Mnlrll rr"onrnllal A"",""n","on "\. "',"I'nlr.l "nl\'cr'cr,r,. Proc.IEEE. 5(, (10£,1').

1'1'.372-.173.

1471 G. J. BIERMAN. Pn,.'rr 'rTlr< rnllll,,"on "r "on<lllon an.! r()L'aTlanrt' mnlncrs. IEEE Trans. AUlo­malic Conlrol. AC-17 (107:), 1'1' :~S_:11

r -1Rl o. 1\. ("A I J\ 111\ N. """""""'11/ .,1/",,,.,, of /,,,,.,,, ("\lrtr, \ '''1,/, tndrh' tt".,,,,,tr,1 ,,,,,r r"" ctnnl.t. Prnl".

IFIT ... ss (1''',7). PI'. ~fllh-~1J17

IJ91 K. c. DALY. £1."/IIOlln/1 11,1''nnlTl' rr",rnrnllnl. Ekclron. Lel1 . ~ (1<172). fl.. 1"0.r '0) \1,'. EVF R1.1NG. On ,hr rl'a/llnllo" IIr r"" "\. I"''''r, 'rTlCI. Prnc IF.F.:E. 5<, (I I)f> 7). I' 413.

1511 D. A. GlIlI .. Thr ".111110" of Irnr(ll "'n\la,,' rorffi"rrll ord,nan' d;(Tt'rrnlln/ rqual;"ns ••.;11. APL.

('omrUI. Me1hod' Mech~nll-' ~nu Enell:. , (1972). 1'1' 1~t)_I"('.1521 M. L. '-'OLT. A nOl'cl mrlh".! of rr'o/lInllnc Irm'<lrn' rr'pon'r. PIOC. IEEE. 54 (19M:'). flp. 20-23.

15.1 I J. B. MIINKIN AND J. C. HIING. n., """,don rrro" ,n co",,,Ulnllnn of Iran.,;I;on malrirr.<. Repnnts ofthe .1010' AUlom~IIC ('onlrol ('onfcrl'nc..:. 1'1' ftfl-ft~. Unl\"Cr'"" of Colorado. Boulder. CO. 19£>9.

834 C1.EVI: MOLEK ANI> CHARU:S VAN L() ..\:-'

(S-I I E. J. MASTASCUSA. A rdtllwn br'/k·all Lwu'> 1II",hod tllld fUUT/h mda RUIIXr'-l'Ull" mr'/;IUJ ,,,,rualual/ull of lrtlmlt'nl rrspuII>r. Proc. IEEE. 57 (I ~6~). pp. 1103-1\0-1.

/55) J. B. PLANT. On Ih•• ('U11lpUlaIWIIof IramlOIl mtllflCr" fur I/mr muaT/arll S)·SIe'nlJ.IbuJ .. 50 ( I ~oti). pp1397-139~.

[56] M. M. SHAH. 011 Ihr e'ualualion of 1'''''. Cambridge Repon CUED/B-Conlrol TRS. Cambnd)!e.England. 1971.

157J ---. Allal~>is of ruulldoff alld IrunCal/Oll e'rron m Iht' ('umplllalWII of lrurallUHI malfl'·r'>. (' ••mhno):"Repon CUED/B-Conlrol TR I:!. Cambnd~e. En~l ••no. I Y71.

(58] C. J. STANDISH. Truncale'd Ta~lor >ait's apprUJLl11laliun to Iht' slalC: ITansil/on malfl.( uf u cmll/nuou>paramt't'" Markou cham. J. Linear Algc:bra Appl.. I:! (I Y75). pp. 17Y-I~3.

[59] D. E. WHITNEY. Propogalr'd t'"or /wunds for numuical SolUlion of ITanSirnl rt'JpollSt'. Proc. IEEE. 54(1966), pp. 1084-10115.

160) ---, More' slmilaril/e's bt'/wan Rungr-Kulla and maITix rxpune'fIlitll me'lhod, fur nulUUllfiXtransi<,ntrr'sponse'. Ibid., 57 (ILJ69). pp. 2053-2054.

Rational approximation.1611 J. L. BLUE AND H. K. GUMMEL, Ralionul apprOXi11laliofll to Ihe' matrix nptJlle'nl/al fur »'>1<'111>of ",I!

difJl'frntial rqualiuns. J. Com[Julallonal Phys .. 5 (1 Y70), pp. 70-113.(62] W. J. CODY. G. MEINARDUS AND R. S. VAKGA. Chr'byshr'v raliollal approximal/on to np (-x) In

(0, +00] and applicalions 10 hr'al conduclion problt'ms, J. Approximallon Theory. 2 (IY6LJ). pp.50-65 .

/631 W. FAIK AND Y. LUKE, Padi approximalions to Ihe' oprrator t'xponr'fllial, Numer. Malh .. 14 (I LJ7U).

pp. 37LJ-3112.

1641 S. P. NORSETT, C-pol}'numials for ralional approximalion to Ihe' apunt'ntial fUIIClion. Ibid .• :!5

(1975), pp. 39-56.1651 E. B. SAFF. On Ihe' Jt'gru of bt'SI ralional appruximaliolllo Iht' t'xpont'ntial funcliull. J. ApproXimation

Thc:ory, 9 (1973), [J[J. LJ7-101.(66) E. B. SAFF AND R. S. V AKGA. On Iht' uro> and polr's of Padi appruximanls 10 eXp (:). Numcr.

Malh .. 25 (1975). pp. 1-14.167\ R. E. SCRATON. Commt'nl On ralional apprOXlmanlS to Iht' maITix apunr'ntial. Elcclron. Lei\., 7

(1971). pp. 260-261.[6111 J. L SIEMIENIUCH, ProP<'TII<'>of arlain raliOlltlI approximalions to e' -'. BIT, 16 (I LJ76). pp. 17::?-191.

16LJI G. SIEMlcNIUCH AND 1. GLADWELL. On ,ime' disCTt'lizalions for 111Ir'arlime' de'P<'lIde'1IlpamaldifJt'Tr'rllial e'qualiom, Numc:ncal Analysis R"pon no. 5. Dep.utment of Mathematics. Unl\'erslly 01Mancheslc:r. Manchester. En):land. 1'17-1.

(70J D. M. TRUJlLLU. The' Jlft'Cl nume'flcal m/e'j!,rtlIWfI of Ime'ar matrix diffc'ft'ntial e'q14tlIIVIISU>lfll( Ptldiapprox,malions. Internal. J. lor /'oiumer. Methods Engrg .. 9 (1975), pp. 259-270.

(71 J R. S. VARGA, Ofl h,ght'f ordt'f >labIt' ,mp!tw me'thuds for solvinll parabo!tc pamal difJc'fe'ntialt'quations. J. Math. Phys .• 40 (1961), pp. :!20-231.

(72) R. C. WARD. Numt'f/cal compulaliun of firt' ",alrix t'Xpunt'nlial wilh accuracy t'slimtll ••. SIAM J.Numcr. Anal.. 14 (ILJ77). pp. (,\lO~IO.

(73) A. WKI\GG AND C. DAVIEs. EOcJlulllWfI "f IIr~ malfix e'~pun~fIlilll. EleL·lron. Leu .. 9 (IY73). pp.525-526.

(74] ---. Cumpulaliun uf Iht' r.ptJllrnlitll of tl ",tl,r/J I: The'ore'llcal coruide'raliuns. J. Insl. Math. Appl..II (ILJ7:I). pp. 30'1-37 ••

[75 J ---, Cumpulalllln of lire' optJlle'nlwl vf a IIIUtru IJ' Pracllcal conSltJaalwns. Iblo .. 15 (1975). pp273-271)

(76J V. ZAKIAN. RallOnal apprOJlmUII/I /U lire' ma'fI. optJflm14tll. Electron. Leu .. b (197U). pp. 1\1-1-1\15

(77 J V. ZA"IAN AND R. E. SCH.ATC>". (Um",rfll, Oil ra14uflul appruxlmalwflS lu Ihe' matri. e'xpullt'fllwl.Ibid .. 7 (1971). pp. 2011--262

Polynomial methuds.(711] G. J. BIEMMAN. Fmlle' St'fle'S>ululWns for Ilrr Iftlll,IIWn malT/x and covaTlanCt' of a l,m ••-mVtlTlWIl

SYSlt'm, IEEE Tr ••ns. AutomatIC' Connol. A('· Ib (I Y71). pp. 173-175.

(79J J. A. BOEHM AND J A. 1II U H.""A"". An al,.:u",lrm for j(e'IIc'ftl14ngcon>/Ilut'nt malT/ct'>. IEEE Trans.CirCUli Theory. CT -1/\ ( I ~7 I ). rf> 17/\- 17'1

180) C. F. CHEN AND R. R. PARKEH.. C,.,orrul,:a'wn of He'auuid~'s t'xpansiufl le'ch1/lqur to trtlnSIlWnmafrix <'l>Olutl14on.IEEE Tr ••ns Eouc.. E-'I (I Ybb). pp. 2ULJ-212.

(81) W. C. DAVIDON. Expont'nllal FunL'lwn uf tl 2-br-2 Mtltru. Hewleu·Packard HP65 Library Pro­gram.

THE EXPONENTIAL VI- A MATRIX

11<21S. DEARDS. 0" thl' l'valua"",/ of np itA). Matrix Tensor Quart.. D (1973). PI' 141-14:./1(11 S. GANI\PATHY ANI> R. S. RAO. Tra"<lf'''' rl'SfIO"SI' l'valual'o" from thl' .flatl' ,ra".<lt,on matrn. Proe.

IEEE. 57 (1909). PI".. 147-349.

11<4] I. C. GOKNAR. 0" thl' l'valuat,on of con.t'ltUl'nt malricl'S. Internal. J. Systems Sci .. 5 (1974). PI".2 \J-2 Iii.

IR51 I. I. KOLODNER. On I'Xp (tA) with A satitfllin$( a pnlynomia/. J. Math. Anal. Appl .. 52 (1975). PI".514-524.

IXol Y. L. Kuo ANI) M. L. UOt •. Comml"nt.t "" "A nOI't"/ mnhod of I'valua"n$( I'A' in closl'd form". IEEE

Trans. AIIIOm:1lic Control. AC-I 6 (1971). p. 521.

11<71E. J. MI\STI\SCtJSA. A mNh"d of ca/c,,/atin$( I'A' I>a.tf'd on thl' CaY/I'II-Hami/ton throrl"m. Proc.IEEE. 57 (1%9). PI". 1321<-1.129.

[RHI K. R. RAO AND N. AHMF.D. Hf'OI,i.tit/1" npansion of transilion matricl's. Il>id .• 56 (J 90!!). PI".11114-11116.

11<91---. Eva/uation of transilion malricl's. IEEE Trans. Automatic Control. AC-14 (1969). PI".779-780.

190) B. Roy. A. K. MANDAL. D. Roy CHOUDHURY. A. K. CHOUDHURY. On Ihl' I'va/ualion of Ihl'st(l/I' transilion matnx. Proc. IEEE. 57 (1969). PI". 234-235.

1911 M. r-;. S. SWAMY. 0" a formu/a for l'valuatintr. I'A' whl'n Ihl' l'iKl'nvalul'.t a" not nl'cl'ssari/y di.ttincr.Matrix Tensor Quart .. 2.l (1972). PI". 07-72.

/921 M. VmYSAGER. A nOl'1'1mrthod of I'l'alUalrn£ rA' in clo.trd form. IEEE Trans. Automatic Control.

AC-15 (1970). PI". (.00-601.

(93) V. ZAKIAN. SolUllon of homo$(l'Nl'ouS ordinary linl'ar diffl'tl'nlial I'quatiO'1S by "uml'rical i"t'I'TJio" of

Laplaa tra"sfomlS. Electron. Letl.. 7 (1971 ). PI". 546-548.

Companion matrix methods.194) A. K. CHOUDHURY. ET AL .. 0" Ihf' I'valualio" of I'A'. Proc. IEEE. 56 (1968). PI". 1110-11 J 1.

1951 L. FALCtDlENO AND A. LUVINSON. A numl'rical approach 10 compuli"1C Ihl' companion malrixI'xpo"I'Ntial. CSELT RaT'porti teehnici. No.4. 1975. PI". 69-71.

(96) C. J. HARRtS. Et'lJ/ualion of matrix pol\'nomials in thl' stall' companion malrix of 1i"l'ar timl' invariant.tllstl'm.t. Internal. J. Systems Sci .. 4 (1973). PI". 301-307.

197J D. W. KAMMLF.R. Numl'rical I'valualion of I'XP (At) wI.,." A i.t a companion matrix. unT'ublishedmanuscript. University 01 Southern IIhnois. Carbondale:. IL. 1976.

198) I. KAUFMAN. Evaluation of an a"a/llt,cni fU"Clion of a companion malrix wilh disli"cl l'i£l'"oo/ul'.t., Proc. IEEE. 57 (1969). T'p. 1180- J I II I.

(99) ---. A "otl' 0" thl' .•Evalualio" of an analvrical fUnClio" of a companion matrix wilh di.t/inctl'itr.rNvalul's". Il>id .. 57 (19(,9). 1"1".201'3-201'4.

11001 I. KAUFMAN AND P. H. ROE. 0" sn'rmt dl"scril>l'd 1>11a companion malrix. IEEE Tran~. AutomaticControl. AC-15 (1970). T'p. tl92-09:1.

1101] I. KAUFMAN. H. MANN AND). VLACH. A fast proudurl' fm thl" analysi.t of linl'ar timr invariantnl"M'orh.IEEE Tran~. Circuit Theory. CT-18 (1971). PI". 739-741.

1102) M. L. UOU. El'alunrion of thf' tran,itia" matrix. Proc. IEEE. 55 (19(,7). pp. 221<-229.1103) A. K. MANDAL. F.T AL .. Numrrical compulalion ml'lhod for Ihr ('l·alualion of thl' transition malrix.

Ihid .. 11(, (1969), PT'. 500-~0:.111141 W. E. TIIOM"ON. ",.a/unt,on of ,ra"trrn' rrtf'""tr. It.ic1 ... ~J (1 '1(,(.). 1'. 1.~R4.

Ordinary differential equations.II(5) B. L. EHLE AND J. D. LA ••••.SON. Gr"Nal,ud R,mtr.r-Kmta proassl'S for stit! initial valul' problf'ms. J.

Insl. Math. Appl.. 1(, (1975). PI". 11-21.

IlOtl) C. W GEAR. Numl'ncal /nll/al Valur ProMl'ms ", Ord",a T\' Diff('ll'ntia( Equal/ons. Prentice-Hall.Enl!lewood Clifts. NJ. 197J

IJ 071 L. F. SHAMPINE AND M. K. GORDON. Compllt('l SolUllon of Ord,naT\! Difful'nlial Equations- TItI'I"itial Valul' PraMI'm. W H. Freem~n and Co .. San Fr:1ncisco. 1975.

[lOR) L. F. SHAMPINE ANn H. A. W 1\TT"'. Proc"cal snlulln" nf md,naT\! dif!('Il'ntial I'quatio".t I>r RII"III'­/(UllO mrthod,. SandIa Lah Reporl SAND 70-05R5 AlhuQuerQue. NM. 1976.

11091 J. STARNER. NumrrrcaI50Im,0" of ,mpl,m diff('ll'nl/al-altr.cbra,c I'qual/ons. Ph.D. Thesis. Universityof New MeXICO. AlbuQucrQue. NM. 1970

Matrix decomposilion methods.11101 G. H. GOLUII AND) H W,I.K'NSON. f/I-cnnd'II0M,! 6/<1l'nsv-'tl'ms and thl' computation of Ihl'

Jordan canon/cal form. thl~ Rcvle"-. I I' ( 1Q7h). PI'>.57R-fo 19.

CLEVE MOLEK AND CIi,\/{LcS VAN LUAt-;

II J I) B. KAGSTRO~1 AND A. HUH!:. An al~oriliJm for num ••f1cul COl1lpUIUIIUnof Ii•••)OlJ"fI 1I0ll"ul'0"1I ,,/

u compl ••~ malrl~. R.:rort UMINF 51.74. Departm.:nl of Informauon Prot''''',"!:. UnlVc"'I~ ,11

Umea. Um.: ••. Swed.:n. IY74. To appcar in Tram. Malh. Soflwave.

f 1121 B. N. PARLETI. A r••curr ••na among IIII?t'/ ••mm/S of funcliolls of ",angular ma"ias. Linear AII/ebraAppl.. 14 (1976), pp. J 17-/21.

1113/ B. T. SMITH. J. M. HOYLE. J. J. DONGARRA. B. S. GARBOW. Y. I"EIIE. v. C. KLEMA. C. B.MOLEK. Mu"u E/~""sysl ••m Rouli""s: EISPACK Guiu ••. 2nd cd .. L,:clure N,He, In Cumpu'.:r

Science b. Sprtn!:er- Verlag. New Yur~. 1971>.

integrals involvillg eA'.

(114) E. S. ARMSTRONG AND A. K. CAGLAYAN. An algorilhm for Ih•• w••ighlinK mUlric ••s in liJr

s"mplrd-dala oplimallintar regulalDr problem. NASA Technical NOle. TN D-1I312. 1976./115) J. JOHNSON AND C. L PHILLIPS. An algorithm for the compulalion of Ih" inlt/(ral of Ih •• SlUIr

"ansI/ion m""ix. IEEE Trans. AUlOmalic ConICal. AC-Ib (1971). pp. 204-205.

(1161 C. KALLSTROM. CompulIlIlI ap (A) alld tXP (As)ds. RefX)n 7309, Divi.iun of AUlom:llic Conl/ol.

Lund Inslllule of Technology. Lund. Sw.:d.:n. IY73 .. [ I J 7) A. H. LEVIS. Sumr COIllPUIUlionulus~CIS of tht mutrix apull<,nlial. IEEE Trans. Automalic Conlrol.

AC-14 (lY6Y). pp. 41o-tll.

.(118) C. F. V AN LOAN. Compuling inttfCrals inoolvinfC tht ma"ix txponmliul. Cornell Compul.:r Scienc"

Repon TR 76-270. 1976.

Selected applications.II 19) F. H. BRANIN. Cumpultr m ••thods of network analysis. Proc. IEEE. 55 (1967). pp. 17117-180 I.

(1201 R. BROCKETT. Finite Dirrutnsional Linear Systems. John Wiley. New York. 1970.(121) K. F. HANSEN. B. V. KOEN AND W. W. LllTLE. Stubll' numtrical solulions of the reacror kinl'''cs

tquations. Nuclear xi. Engra: .• 22 (1965). pp. 51-59.

(122] J. A. W. DA NOBREGA. A nl'w solution oflhl' point kinttics ~qualions. Ibid .. 46 (1971). pr. 366-375.