ARMA model parameter estimation based on the equivalent MA

Digital Signal Processing 16 (2006) 670–681

www.elsevier.com/locate/dsp

ARMA model parameter estimation based on the equivalentMA approach

Aydin Kizilkaya a,∗, Ahmet H. Kayran b

a Electrical and Electronics Engineering Department, Pamukkale University, 20040 Kınıklı, Denizli, Turkeyb Electronics and Communication Engineering Department, Istanbul Technical University, 34469 Maslak, Istanbul, Turkey

Available online 28 September 2006

Abstract

The paper investigates the relation between the parameters of an autoregressive moving average (ARMA) model and its equiv-alent moving average (EMA) model. On the basis of this relation, a new method is proposed for determining the ARMA modelparameters from the coefficients of a finite-order EMA model. This method is a three-step approach: in the first step, a simplerecursion relating the EMA model parameters and the cepstral coefficients of an ARMA process is derived to estimate the EMAmodel parameters; in the second step, the AR parameters are estimated by solving the linear equation set composed of EMA para-meters; then, the MA parameters are obtained via simple computations using the estimated EMA and AR parameters. Simulationsincluding both low- and high-order ARMA processes are given to demonstrate the performance of the new method. The end resultsare compared with the existing method in the literature over some performance criteria. It is observed from the simulations that ournew algorithm produces the satisfactory and acceptable results.© 2006 Elsevier Inc. All rights reserved.

Keywords: ARMA model parameter estimation; Equivalent MA model; MA-cepstrum recursion; Equivalent model approach; Spectral estimation

1. Introduction

The applications of ARMA models have found great attention in a wide range of control and signal processingareas such as time series analysis, signal modeling, spectral estimation, system identification, etc. From the viewpointof parametric modeling, a variety of methods for estimating the parameters of an ARMA model have been developedin the realm of statistical and engineering literature [1–11].

Compared with the pure AR or MA models, in the parametric modeling of random signals, ARMA models providethe most effective linear model of stationary random fields since they are capable of modeling the unknown processwith minimum number of parameters using both poles and zeros rather than only poles or zeros [1–3,12,13]. Therefore,ARMA models are preferable over its AR or MA counterparts.

In spite of the general superiority of ARMA models, the parameter estimation procedure for an ARMA model ismuch more difficult than the pure AR models. This is due to the intrinsic nonlinearity of estimating the MA partsof an ARMA model; it requires nonlinear optimization. However, estimation of the AR parameters of an ARMA

* Corresponding author. Fax: +90 258 212 5538.E-mail address: [email protected] (A. Kizilkaya).

1051-2004/$ – see front matter © 2006 Elsevier Inc. All rights reserved.doi:10.1016/j.dsp.2006.08.010

A. Kizilkaya, A.H. Kayran / Digital Signal Processing 16 (2006) 670–681 671

model is performed by the so-called modified Yule–Walker equations requiring the solution of a simple set of linearequations [3,4,13]. It follows from that the estimation facility of AR model parameters has motivated a number ofresearchers to construct a relationship between AR and ARMA models. On the basis of this relation, the approach ofequivalence between an infinite-order AR model and a finite-order ARMA model is used in [1–3,9]. In this approach,a sufficiently high-order AR model is used to determine the ARMA model parameters. Recently, in [14], this approachis also employed to estimate the parameters of a two-dimensional ARMA model. Another application of this approachis to determine the unknown input of an ARMA model, by inverse filtering the actual signal data with a high-orderAR model. In [7], the input signal required for determining the ARMA model parameters are obtained by this way.

In this paper, a new method for the estimation of the parameters of an ARMA model by utilizing the relationshipbetween an infinite-order MA model and a finite-order ARMA model is presented. The relation between AR andARMA models inspired us to develop this method. As alluded to in the last paragraph, there is a relationship ARMA(p, q) = AR (∞) for a stationary and reversible ARMA model of order (p,q). The same relation is also available forthe MA and ARMA models, i.e., ARMA (p, q) = MA (∞). As a result, it is possible to approach an ARMA (p,q)model by a sufficiently high-order MA model called as equivalent MA (EMA) model here. The proposed methodis a three-step approach: in the first step, a simple recursion relating the EMA model parameters and the cepstralcoefficients of an ARMA process is derived to estimate the EMA model parameters; then the EMA model parametersobtained by the developed MA-cepstrum recursion technique are used in the suggested linear procedure so as to getthe AR and MA parameters of the ARMA model.

In order to illustrate the performance of the new method, simulations including both low- and high-order ARMAprocesses are simultaneously realized over the method in [3]. The end results generated by the method proposed andthe method in [3] are then compared with respect to some performance criteria mentioned as (i) Cramer–Rao lowerbounds of the estimated parameters; (ii) the norms of difference vectors acquired by subtracting the true and estimatedAR and MA parameters; (iii) Itakura–Saito (I–S) distance measure between the true and estimated power spectra;(iv) the magnitude plots of true and estimated power spectra.

The rest of the paper is organized as follows. Section 2 deals with the model and the assumptions used in derivationof the new method. In Section 3, the method for estimating the parameters of an ARMA model is presented. InSection 4, some ARMA models existing in the literature are utilized to evaluate the performance of the proposedmethod; the results obtained simultaneously by the method in [3] are compared according to some performancecriteria. Finally, in Section 5, we give some concluding remarks and suggestions.

2. Model and assumptions used in the proposed method

A stationary ARMA process of order (p,q) is modeled as the output of a linear time-invariant (LTI) digital filterdriven by a white noise signal. Without loss of generality, the transfer function of this filter is given by

H(z) = X(z)

W(z)= B(z)

A(z)=

∑q

l=0 blz−l∑p

k=0 akz−k(1)

with a0 = 1 and b0 = 1. The expression defined in (1) is also called the transfer function of a causal stationary ARMAmodel of order (p,q), and {ak ; 1 � k � p} and {bl ; 1 � l � q} are the respective AR and MA parameters of theARMA model. The nth sample of the process corresponding to this model satisfies the following difference equation:

x(n) = −p∑

k=1

akx(n − k) +q∑

l=0

blw(n − l), (2)

where {w(n)} is the input sequence consisting of the samples of a zero-mean white Gaussian noise (WGN) processwith variance σ 2

w; {x(n)} is the output sequence whose samples are generated by the expression in (2). This outputsequence provides our knowledge basis over the range 0 � n � N − 1. The power spectral density of x(n) is given by

Sx(ejω) = σ 2

w

∣∣∣∣B(ejω)

A(ejω)

∣∣∣∣2

= σ 2w

∣∣∣∣∑q

l=0 ble−jωl∑p

k=0 ake−jωk

∣∣∣∣2

. (3)

We make the following assumptions for the proposed method:

672 A. Kizilkaya, A.H. Kayran / Digital Signal Processing 16 (2006) 670–681

Fig. 1. Block diagram of the proposed ARMA model parameter estimation algorithm.

(A1) The ARMA model defined in (1) is causal and stable.(A2) a0 = 1 and b0 = 1.(A3) The ARMA model orders p and q are known.(A4) The unobserved excitation {w(n)} is taken to be a zero-mean WGN spatial series with variance σ 2

w .

3. Proposed method

Block diagram of the proposed ARMA model parameter estimator is illustrated in Fig. 1. Fundamentally, differentfrom the methods in [1–3,9], the method suggested for the estimation of the parameters of an ARMA model definedas in (1) uses the approach of equivalence between an infinite-order MA model and a finite-order ARMA model. Forphysically realizable, a sufficiently high-order MA model is employed and here it is called as equivalent MA (EMA)model. In the proposed method, the EMA model coefficients are used to estimate the AR and MA parameters of anARMA model of order (p,q). In order to obtain the EMA model parameters, a simple recursion relating the EMAmodel parameters and cepstral coefficients of a given N -length data record of an ARMA process is derived. Thecepstral coefficients are estimated by the well-known periodogram techniques. So using the estimated EMA modelcoefficients in the developed linear procedure, first the AR parameters are obtained and then the MA parameters areestimated by employing the estimated AR and EMA parameters. Each of the estimation steps shown in Fig. 1 will beelucidated in the following sections.

3.1. Estimation of the EMA model parameters

The formulae to be derived for the estimation of the parameters of a (p,q)-order ARMA model are based upon therelation ARMA (p, q) = MA (∞). On the basis of this relation, we can write

H(z) = B(z)

A(z)=

∑q

l=0 blz−l∑p

k=0 akz−k=

∞∑m=0

dmz−m, (4)

where {dm} are the parameters of EMA model. In practice, it is possible to approach an ARMA (p,q) model by asufficiently high-order MA model. Thus, for any L value, the relationship defined in (4) can be given as∑q

l=0 blz−l∑p

k=0 akz−k∼=

L∑m=0

dmz−m (5)

with L stands for the order of the EMA model.To solve the EMA model parameter estimation problem, we use the spectral factorization theory [15, pp. 61–63]

defined by

Sx(z) = σ 2wH(z)H ∗(1/z∗) (6)


with (·)∗ denotes the complex-conjugate. The expression given in (6) is the same with (3) for z = ejω. From the as-sumption of (A1), H(z) and H ∗(1/z∗) correspond to the minimum-phase and maximum-phase systems, respectively.Consequently, the natural logarithm of Sx(z), lnSx(z), will be analytic and can be expanded in a Laurent series as[15, p. 62]

lnSx(z) =∞∑

i=−∞ciz

−i , (7)

where {ci} are the cepstral coefficients of a given N -length data set. Because of the assumption of (A1), lnH(z) willbe also analytic function on the unit circle |z| = 1. So using (7), lnH(z) can be represented as

lnH(z) = ln

(L∑

m=0

dmz−m

)=

∞∑i=1

ciz−i . (8)

For the sake of simplicity, we make a change of variable z−1 = u in (8). After making this change, taking the derivativeof both sides of (8) with respect to u produces∑L

m=1 mdmum−1∑Lm=0 dmum

=∞∑i=1

iciui−1. (9)

After multiplying both sides of (9) by the denominator polynomial, we obtain

L∑m=1

mdmum−1 =∞∑i=1

L∑m=0

icidmum+i−1. (10)

It is certain that d0 = 1 in accordance with the assumption of (A2). After equating the coefficients of equal powersof u in (10) and using the information of d0 = 1, we finally arrive at the following recursive equation for the EMAmodel parameters {dm}:

dm = cm + 1

m

m−1∑i=1

icidm−i , d0 = 1, 1 � m � L. (11)

Note that the recursion derived in (11) corresponds to a special case of the equation introduced in [11]. It is appearedfrom (11) that an estimate of the cepstrum of the process is required for the estimation of the EMA model parameters.An estimate of the cepstrum can be obtained with ease by the following way [11]:

ci = F−1{ln Sx(ejω)

}, (12)

where F−1{·} implies the inverse Fourier transform, and Sx(ejω) is a nonparametric power spectrum estimate of

x(n). In simulation examples, we used the periodogram with rectangular window and the modified periodogram withHanning window for the nonparametric power spectrum estimation of N -length observation field {x(n)}.

3.2. Estimation of the AR parameters of the ARMA model

The relation between the parameters of EMA and ARMA models is established based on the assumption of (A2).Bearing this in mind, in the proposed method, this relation is constructed by taking account of the division processbetween the AR and MA parts of the ARMA model in (1). For ARMA (2, 2) model, this division process is describedas follows:

B(z)

A(z)= 1 + b1z

−1 + b2z−2

1 + a1z−1 + a2z−2

= 1︸︷︷︸d0

+ (b1 − a1)︸︷︷︸d1

z−1 + (b2 − a2 − a1(b1 − a1))︸︷︷︸d2

z−2 + ((a21 − a2)(b1 − a1) − a1(b2 − a2))︸︷︷︸

d3

z−3 + · · · .

(13)


The relation between the parameters of EMA and ARMA models is appeared at the quotient part of (13). By induction,the connection among the parameters of {dm}, {ak}, and {bl} for the respective EMA (L) and ARMA (p,q) models isgiven by

p∑j=1

Dj aj = −D0 + B. (14)

Using (5) and (14), the coefficient vectors in (14) are composed in the forms of

D0 =

⎡⎢⎢⎢⎣

d1d2...

dL

⎤⎥⎥⎥⎦ , Dj =

⎡⎢⎢⎢⎢⎢⎣

0j−11d1...

dL−j

⎤⎥⎥⎥⎥⎥⎦ , B =

⎡⎢⎢⎢⎣

b1...

bq

0L−q

⎤⎥⎥⎥⎦ , (15)

where 0i denotes the i-length zero vector.Note that the EMA model order L cannot be selected below the value of L = p + q . If the EMA parameters are

determined exactly, only L = p + q parameters of the EMA model will be sufficient so as to characterize the ARMA(p,q) model completely. Since L is usually chosen larger than the number of unknowns, p +q , the matching betweenthe respective vectors

∑p

j=1 Dj aj and B − D0 in (14) can be performed by minimizing the difference of these vectorswith respect to the AR and MA parameters. Supposing that the AR and MA parameters to be estimated are real, thisminimization operation is defined by

∂ε

∂ak

= 0, 1 � k � p, (16)

∂ε

∂bl

= 0, 1 � l � q, (17)

where

ε =∥∥∥∥∥

p∑j=1

Dj aj + D0 − B

∥∥∥∥∥2

. (18)

Using (18), the minimization processes in (16) and (17) generate p and q linear equation sets as much as unknownAR and MA parameters, respectively. Thus, the resulting system of linear equations achieved by (16) is written in thefollowing matrix form:⎡

⎢⎢⎣C11 C12 · · · C1p

C21 C22 · · · C2p

......

. . ....

Cp1 Cp2 · · · Cpp

⎤⎥⎥⎦

︸︷︷︸C

⎡⎢⎢⎣

a1a2...

ap

⎤⎥⎥⎦

︸︷︷︸a

=

⎡⎢⎢⎣

f1f2...

fp

⎤⎥⎥⎦

︸︷︷︸f

, (19)

where

Cij =L∑

k=q+1

Di (k)Dj (k), (20)

fi = −L∑

k=q+1

D0(k)Di (k), (21)

with 1 � (i, j) � p. The column vector a in (19) involves the desired AR parameters. Using expressions (20) and (21)in (19), the calculation of

a = C−1f (22)

gives the estimation of these parameters.


3.3. Estimation of the MA parameters of the ARMA model

Essentially, the minimization defined in (17) corresponds to obtain the MA parameters by employing the relationgiven in (14):

bl = dl +p∑

j=1

aj Dj (l), 1 � l � q. (23)

It follows from that substituting the EMA and AR parameters estimated by (11) and (22) into (23), we can simplyestimate the MA parameters of the ARMA model.

The expressions like Di (j) in (20), (21), and (23) denote the j th components of the vectors Di defined in (15).It is interesting to note that the formulae given in (14), (15), (19)–(21), and (23) are similar in structure to the

algorithm presented in [3]. This is an expected result since the algorithm in [3] and our method reveal through thesame approach, the equivalent model approach. However, different from the method in [3], we use the equivalent MA(EMA) parameters rather than equivalent AR (EAR) parameters.

3.3.1. Summary of the proposed methodTaking account of the assumptions introduced in Section 2, the method proposed for the estimation of the parame-

ters of a causal stationary ARMA (p,q) model consists of the following steps:

Step 1. Estimate the EMA model parameters {dm, 1 � m � L} by employing the recursive equation derived in (11),and then use them for determining the p + 1 vectors {Dj , 0 � j � p} in (15).

Step 2. Estimate the AR parameters of the ARMA model solving the system of linear equations (19) composed of thevalues calculated exploiting the expressions in (20) and (21), by (22).

Step 3. Use the results calculated in Steps 1 and 2 for estimating the MA parameters of the ARMA model by (23).

3.3.2. Remarks on the proposed methodRemark 1. If the process under consideration is already MA, then it is to be L = q , from (14), it will be B = D0, i.e.,bm = dm, 1 � m � q . In this case, the parameters of a pure MA model are obtained rearranging (11) as follows:

bm = cm + 1

m

m−1∑i=1

icibm−i , b0 = 1, 1 � m � q. (24)

It is noteworthy that this recursive equation corresponds to a special case of the equation derived in [11], and itenables us to estimate the MA parameters of random signals as well.

Remark 2. As in [3], if the parameters of the EMA model of order L are estimated exactly, then L = p+q coefficientsare enough to estimate the parameters of the ARMA (p,q) model, exactly. This opinion is the simple consequence of(14) and (15) for L = p + q .

4. Simulation results

This section moves forward with computer simulations. In order to confirm the theoretical developments and toevaluate the performance of the proposed approach, some simulation experiments are made. Two ARMA modelprototypes used in the simulations are taken from the previously published papers of [3,6]. In all of the simulations,given that x(n) = 0 for n < 0, the samples of the N -length data sets of signal x(n) defined by (2) are generatedsynthetically by using these ARMA models. The noise sequence driving these models are composed of the samples ofzero-mean WGN process with variance σ 2

w = 1, i.e., WGN (0, 1). The N -length data sets {x(n)} are simultaneouslyapplied to the method [3] and our proposed method. The results obtained by these methods are then compared fromthe perspective of parameter and spectral estimates.


Similar in [3], the estimation procedure has been provided over 100 independent runs of the methods with datalength of N = 1024. The final estimates calculated by averaging the values estimated by 100 independent runs of thealgorithms are characterized by the mean, and the standard deviations for each estimated parameter are also computed.The equivalent model order (EMO) L required for the estimation procedure of both methods was chosen empiricallyin simulated examples. Hence, EMOs given by L = {4,6,10,20,30} were used. An estimate of the driving noisevariance σ 2

w was obtained by the way introduced in [11].To gain more insight into the performance of the algorithms, simulation results acquired by the algorithms have

been assessed over the following performance criteria.

(i) Cramer–Rao lower bound on the standard deviation of the each estimated ARMA model parameter: To evaluatethe consistency of the parameter estimators, computation of the standard deviation of each parameter is important.Standard deviations are not small of Cramer–Rao lower bound (CRLB) that sets a lower bound on the standarddeviation/variance of each parameter estimated by any unbiased estimator. Consequently, the consistency of anyparameter estimation method increases, as the standard deviations of the estimated parameters close to the CRLB.The CRLB on the standard deviation of each estimated ARMA model parameter was computed using the formulagiven by Kızılkaya and Kayran [16].

(ii) The norms of difference matrices obtained by taking the difference between original and estimated AR and MAparameters: Four widely used norms [17] called as L1, L2, L∞, and Frobenius were used here. The desired normvalues must be small in amount. It means that the estimated parameters converge closer to the true values, as thecomputed norms get smaller.

(iii) Itakura–Saito (I–S) distance measure: I–S distance measure [18] was calculated for evaluating the similaritybetween original and estimated spectra. For the parameter estimation accuracy, this measure must be small inamount. Note that the power spectra for the considered ARMA models were obtained by normalizing them withinput noise variance σ 2

w in (3). The samples of the normalized power spectra were calculated in [0,π] frequencyintervals with π /511 spacing. In other words, we used 512 frequency points for plotting the estimated and truepower spectra of models.

(iv) Magnitude plots of the power spectra: The magnitude plots of true and estimated power spectra for each consid-ered ARMA model are displayed for visual evaluation.

Note that the last three criteria were recently used in [14] for evaluating the performance of algorithms.

4.1. Example 1. (2, 2)-order broadband ARMA process

This example is associated with the broadband ARMA process generated by the ARMA model whose transferfunction is previously mentioned in [3]:

H(z) = B(z)

A(z)= 1 − 0.294z−1 + 0.302z−2

1 − 0.450z−1 + 0.550z−2.

The parameter estimates characterized by the mean and the standard deviations for EMOs L = {4,10,20,30} aretabulated in Table 1. The corresponding performance criteria related with these estimates are given in Table 2.

It is observed from the results in Tables 1 and 2 that the proposed method gives better estimates than the Martinelliet al.’s method [3]. The magnitude plots depicted in Fig. 2 verify this assertion as well. For all of EMOs, the minimumnorms and I–S distance measures are provided with the proposed method. Note that in the meaning of parameterand power spectral estimation accuracy, the proposed method yields the best estimates for the minimum EMO order,L = 4. Table 1, Table 2, and Fig. 2a have been supported this idea: the estimated parameters and the correspondingpower spectra converged to the true ones with higher-accuracy. The standard deviations of parameters generated bythe proposed method become close to the theoretical CRLB, as the EMOs increase.

For this example, in the proposed method, we used the periodogram (with rectangular window) technique [19,pp. 393–395] in estimating the cepstral coefficients of broadband ARMA process.


Table 1Statistics of parameter estimates for Example 1

EMOs Methods Parameters a1 a2 b1 b2 σ 2w

True −0.450000 0.550000 −0.294000 0.302000 1.000000(CRLB)0.5 0.089489 0.083187 0.101322 0.097348 –

L = 4 min 1 Mean −0.455326 0.605638 −0.299564 0.353778 1.001403Std. dev. 0.101848 0.168377 0.115307 0.182785 0.047528

2 Mean −0.442257 0.565460 −0.288019 0.322966 1.019082Std. dev. 0.144213 0.212552 0.160905 0.228131 0.070769

L = 10 min 1 Mean −0.392498 0.508561 −0.238134 0.271743 0.993278Std. dev. 0.110349 0.114922 0.118313 0.128148 0.047894

2 Mean −0.407866 0.502751 −0.253628 0.264582 1.003902Std. dev. 0.121310 0.118470 0.133537 0.126728 0.047477

L = 20 min 1 Mean −0.341574 0.459053 −0.187341 0.230176 0.983678Std. dev. 0.123726 0.109125 0.132134 0.120757 0.047836

2 Mean −0.370453 0.464923 −0.216215 0.231877 1.005026Std. dev. 0.125363 0.109981 0.132381 0.115804 0.047614

L = 30 min 1 Mean −0.301296 0.418542 −0.146997 0.196397 0.974179Std. dev. 0.129652 0.108319 0.141427 0.118835 0.047151

2 Mean −0.337523 0.440758 −0.183285 0.212827 1.005962Std. dev. 0.116811 0.111010 0.124603 0.115161 0.047657

Std. dev: standard deviation, Method 1: method of [3], Method 2: proposed method.

Table 2Norm values and I–S distance measures for Example 1

EMOs Methods Performance criteria

L1-norm L2-norm L∞-norm Frobenius-norm dIS

AR MA AR MA AR MA AR MA

L = 4 min 1 0.060964 0.057341 0.055892 0.052076 0.055638 0.051778 0.055892 0.052076 0.0005992 0.023203 0.026947 0.017291 0.021802 0.015460 0.020966 0.017291 0.021802 0.000054

L = 10 min 1 0.098941 0.086123 0.070878 0.063533 0.057502 0.055866 0.070878 0.063533 0.0006512 0.089383 0.077790 0.063306 0.055045 0.047249 0.040372 0.063306 0.055045 0.000563

L = 20 min 1 0.199373 0.178483 0.141519 0.128588 0.108426 0.106659 0.141519 0.128588 0.0023432 0.164624 0.147907 0.116473 0.104727 0.085077 0.077785 0.116473 0.104727 0.001670

L = 30 min 1 0.280162 0.252606 0.198480 0.181002 0.148704 0.147003 0.198480 0.181002 0.0043202 0.221719 0.199888 0.156796 0.142160 0.112477 0.110715 0.156796 0.142160 0.002844

dIS: I–S distance measure, Method 1: method of [3], Method 2: proposed method.

4.2. Example 2. (4, 2)-order narrowband ARMA process

This example is related to the high-order narrowband ARMA process generated by the ARMA model which hasthe transfer function of

H(z) = B(z)

A(z)= 1 + 0.556z−1 + 0.810z−2

1 − 2.595z−1 + 3.339z−2 − 2.200z−3 + 0.731z−4.

This model is considered previously in [6]. Different from the Example 1, we used the modified periodogram (withHanning window) technique [19, pp. 408–412] in estimating the cepstral coefficients of the narrowband ARMAprocess, for the proposed method. Moreover, in this example, the estimation procedure has been realized by theEMOs, L = {6, 10, 20, 30}.

The parameter estimates characterized by the mean and the standard deviations are presented in Table 3, and theperformance criteria calculated using the values in Table 3 are given in Table 4. From these tables and spectra in


Fig. 2. Magnitude plots of power spectra for the broadband ARMA (2, 2) model, with EMOs (a) L = 4, (b) 10, (c) 20, (d) 30.

Fig. 3, it is clear that for all EMOs chosen, the proposed method provides better estimates than the one proposed byMartinelli et al. [3]: almost identical results with true parameters are obtained by the suggested method. Note also thatour method gives the satisfactory results even for the small EMOs. On the other hand, it is appeared from tables andfigures that the Martinelli et al.’s method [3] requires using larger EMOs to converge on the true parameters.

From Table 3, the standard deviations of parameters estimated by the proposed method get close to the CRLBsfor the increasing EMOs. It is worthwhile to note that there is an incongruity between the standard deviations of MAparameter b2 estimated by the Martinelli et al.’s method [3] and the CRLB of this parameter. According to the theoryof CRLB, standard deviations/variances of any parameter estimates are not small of CRLB.

4.3. Remarks

As a result of Tables 1–4 and Figs. 2–3, the following comments can be made for the examples simulated by thealgorithms mentioned here:

(1) From the estimation accuracy viewpoint, the proposed EMA model approach gives superior parameter estimatesto those EAR model approach introduced by Martinelli et al. [3], for all EMOs under consideration.

(2) From the viewpoint of consistency, the Martinelli et al.’s method seems to give small bias as compared to ourmethod; on the other hand, for the increasing EMO values, the standard deviations of parameters estimated by theproposed method have converged to the theoretical CRLBs. It should be noted that in the simulated examples weexploited the popular Burg algorithm for the estimation of the EAR model parameters in [3].

(3) The proposed EMA model approach generates satisfactory and acceptable parameter estimates even for the smallEMOs. However, the parameter estimates obtained by the Martinelli et al.’s EAR model approach [3] have con-verged to the original values when the EMO is increased.


Table 3Statistics of parameter estimates for Example 2

EMOs Methods Parameters a1 a2 a3 a4 b1 b2 σ 2w

True −2.595000 3.339000 −2.200000 0.731000 0.556000 0.810000 1.000000(CRLB)0.5 0.022909 0.051999 0.051906 0.022750 0.023112 0.026258 –

L = 6 min 1 Mean −2.674248 3.381704 −2.117514 0.591163 0.332544 0.270009 1.272635Std. dev. 0.076711 0.202901 0.236643 0.113758 0.050782 0.044237 0.089912

2 Mean −2.611281 3.387943 −2.263516 0.767770 0.543944 0.808727 1.672844Std. dev. 0.122608 0.373477 0.505885 0.296634 0.121176 0.053083 1.037108

L = 10 min 1 Mean −2.389262 2.601424 −1.036438 −0.129477 0.743113 0.615931 1.051714Std. dev. 0.045248 0.131214 0.183594 0.133876 0.039137 0.018583 0.062580

2 Mean −2.597142 3.346058 −2.208013 0.735633 0.558083 0.811732 1.021215Std. dev. 0.055939 0.139341 0.151719 0.073827 0.057784 0.055059 0.074593

L = 20 min 1 Mean −2.558128 3.192878 −1.943048 0.511943 0.587109 0.762631 0.976125Std. dev. 0.041513 0.130415 0.200131 0.172982 0.033842 0.020870 0.053392

2 Mean −2.601399 3.351519 −2.210730 0.734946 0.553826 0.804165 1.007565Std. dev. 0.044141 0.103070 0.103460 0.045565 0.054582 0.059720 0.073229

L = 30 min 1 Mean −2.565472 3.233223 −2.026477 0.596640 0.581491 0.788555 0.957743Std. dev. 0.037029 0.117469 0.186677 0.172213 0.029608 0.023682 0.054190

2 Mean −2.608505 3.364401 −2.220710 0.736765 0.546720 0.794527 1.005222Std. dev. 0.043219 0.099841 0.099708 0.043971 0.051920 0.058353 0.070712

Std. dev: standard deviation, Method 1: method of [3], Method 2: proposed method.

Table 4Norm values and I–S distance measures for Example 2

EMOs Methods Performance criteria

L1-norm L2-norm L∞-norm Frobenius-norm dIS

AR MA AR MA AR MA AR MA

L = 6 min 1 0.344275 0.763447 0.185640 0.584400 0.139837 0.539991 0.185640 0.584400 0.7166452 0.165511 0.013329 0.089704 0.012123 0.063516 0.012056 0.089704 0.012123 0.007757

L = 10 min 1 2.967353 0.381182 1.637267 0.269581 1.163562 0.194069 1.637267 0.269581 7.2330122 0.021846 0.003816 0.011836 0.002709 0.008013 0.002083 0.011836 0.002709 0.000121

L = 20 min 1 0.659004 0.078478 0.369759 0.056671 0.256952 0.047369 0.369759 0.056671 0.5525752 0.033593 0.008009 0.018121 0.006227 0.012519 0.005835 0.018121 0.006227 0.000204

L = 30 min 1 0.443188 0.046936 0.245405 0.033312 0.173523 0.025491 0.245405 0.033312 0.2330742 0.065380 0.024753 0.035913 0.018043 0.025401 0.015473 0.035913 0.018043 0.000844

dIS: I–S distance measure, Method 1: method of [3], Method 2: proposed method.

It should be note that the proposed method provides the satisfactory results for the low-order narrowband ARMAprocess mentioned in [3], as well. One can simulate it and compare the results to be obtained by both of methods. Thisexample is excluded from this text because of the space limitation.

5. Conclusions

In this paper, we are concerned with considering the relation between the parameters of an ARMA model and thoseof the equivalent MA (EMA) model. Based on this relation, a new method for the estimation of the parameters of anARMA model from the coefficients of the EMA model is proposed.

The performance of the proposed method was evaluated by several computer simulations. Further, over someperformance criteria, comparisons were made by the method available in the literature. From simulation results andperformance criteria, it can be conceived that a satisfactory and high-accuracy results which agree well with trueparameters are provided by the proposed method.


Fig. 3. Magnitude plots of power spectra for the narrowband ARMA (4, 2) model, with EMOs (a) L = 6, (b) 10, (c) 20, (d) 30.

Some useful comments and suggestions are summarized as follows:

(1) A MA model is nonlinear in nature. Thus, it requires nonlinear optimization so as to estimate its parameters.However, linear procedures are also available in the literature (e.g., Durbin’s method [20]). This method usesa high-order AR model to find the MA model parameters. A by-product of this paper is a method suggestedto estimate the parameters of a MA model, without requiring a large-order AR model. MA-cepstrum recursionequation given by (24) can be used to estimate the MA model parameters easily.

(2) The algorithm presented here can be extended for the 2-D ARMA models valuable for image processing andhigh-resolution spectral estimation applications.

(3) Estimation of the ARMA model parameters with great accuracy and consistency is highly dependent upon theEMA parameter estimates. In this study, EMA parameter estimation procedure requires the computation of thecepstral coefficients of a given data record. We used the periodogram technique for this aim. In order to performhigh accuracy in the algorithm developed, this step may be fulfilled by different techniques, or a more efficientmethod can be developed to obtain the EMA model parameters.

(4) In the simulated examples, the EMOs L needed for the EMA and EAR approaches were selected empirically,except that its minimum value L = p + q . Although it is indicated in [6] that choosing L to be about three or fourtimes of p + q would be a good choice, choosing a proper value for L usually changes according to the processunder consideration, and it is still an open-ended problem.

Acknowledgment

The authors are thankful to the anonymous referees for their constructive comments and suggestions on the pre-sentation of this paper.


References

[1] Z. Shufang, L. Renjie, A rapid algorithm for on-line and real-time ARMA modeling, in: IEEE Proc. 5th Int. Conf. Signal Processing, Beijing,China, 2000, pp. 230–233.

[2] W.B. Mikhael, A.S. Spanias, G. Kang, L. Fransen, A two-stage pole-zero predictor, IEEE Trans. Circuits Syst. 33 (1986) 352–354.[3] G. Martinelli, G. Orlandi, P. Burrascano, ARMA estimation by the classical predictor, IEEE Trans. Circuits Syst. 32 (1985) 506–507.[4] S. Li, B.W. Dickinson, Application of the lattice filter to robust estimation of AR and ARMA models, IEEE Trans. Acoust. Speech Signal

Process. 36 (1988) 502–512.[5] B. Friedlander, B. Porat, A spectral matching technique for ARMA parameter estimation, IEEE Trans. Acoust. Speech Signal Process. 32

(1984) 338–343.[6] S. Li, Y. Zhu, B.W. Dickinson, A comparison of two linear methods of estimating the parameters of ARMA models, IEEE Trans. Automat.

Control 34 (1989) 915–917.[7] L.D. Paarmann, M.J. Korenberg, Estimation of the parameters of an ARMA signal model based on an orthogonal search, IEEE Trans. Automat.

Control 37 (1992) 347–352.[8] X.-D. Zhang, H. Takeda, An approach to time series analysis and ARMA spectral estimation, IEEE Trans. Acoust. Speech Signal Process. 35

(1987) 1303–1313.[9] D. Graupe, D.J. Krause, J.B. Moore, Identification of autoregressive moving-average parameters of time series, IEEE Trans. Automat. Con-

trol 20 (1975) 104–107.[10] G. Kothapalli, S. Lachowicz, K. Eshraghian, A parameter search technique to build an ARMA model, in: IEEE Proc. 2nd Int. Conf.

Knowledge-Based Intelligent Electronic Systems, Adelaide, Australia, 1998, pp. 298–301.[11] A. Kaderli, A.S. Kayhan, Spectral estimation of ARMA processes using ARMA-cepstrum recursion, IEEE Signal Processing Lett. 7 (2000)

259–261.[12] M. Kaveh, High resolution spectral estimation for noisy signals, IEEE Trans. Acoust. Speech Signal Process. 27 (1979) 286–287.[13] J.A. Cadzow, High performance spectral estimation—A new ARMA method, IEEE Trans. Acoust. Speech Signal Process. 28 (1980) 524–529.[14] A. Kızılkaya, A.H. Kayran, Estimation of 2-D ARMA model parameters by using equivalent AR approach, J. Franklin Inst. 342 (2005) 39–67.[15] D.G. Manolakis, V.K. Ingle, S.M. Kogon, Statistical and Adaptive Signal Processing, McGraw–Hill, Singapore, 2000.[16] A. Kızılkaya, A.H. Kayran, Matrix-based computation of the exact Cramer–Rao bound for the ARMA parameter estimation realized with

relatively short data records, Frequenz 58 (2004) 278–281.[17] C.D. Meyer, Matrix Analysis and Applied Linear Algebra, SIAM, Philadelphia, 2000.[18] A. El-Jaroudi, J. Makhoul, Discrete all-pole modeling, IEEE Trans. Signal Process. 39 (1991) 411–423.[19] M.H. Hayes, Statistical Digital Signal Processing and Modeling, Wiley, Canada, 1996.[20] J. Durbin, Efficient estimation of parameters in moving-average models, Biometrika 46 (1959) 306–316.

Aydin Kizilkaya was born in Augsburg, Munich, Germany, in 1972. He received the B.Sc. degree fromthe Black Sea Technical University, Trabzon, Turkey, in 1994 and the M.Sc. degree from the University ofPamukkale, Denizli, Turkey, in 1997, both in electrical and electronics engineering. He received the Ph.D.degree in electronics and communication engineering from the Istanbul Technical University, Istanbul, Turkey,in 2006. In 2002, he joined the Istanbul Technical University, and he was appointed as a Research and TeachingAssistant from 2002 to 2006. He is now in the University of Pamukkale as an Assistant Doctor. His researchinterests include statistical digital signal processing, multidimensional signal processing, Cramer–Rao lower

bounds for parameter estimation, spectrum estimation, system identification, and circuits and systems.

Ahmet H. Kayran was born in Kayseri, Turkey, in 1955. He received the B.Sc. degree with high honorsfrom the Istanbul Technical University, Istanbul, Turkey, in 1977, the M.Sc. degree from the University of Cal-ifornia, Berkeley, in 1979, and the Ph.D. and D.I.C. degrees from Imperial College of Science and Technology,University of London, London, UK, in 1981, all in electrical engineering. From 1982 to 1984, he was a Na-tional Research Council Research Associate with the Naval Postgraduate School, Monterey, CA. In 1985, hejoined the Istanbul Technical University, where he is now a Professor of electrical engineering, where he wasDean of the Faculty of Electronics and Electrical Engineering from 1996 to 1999. His research interests include

multidimensional lattice filter theory, applications of signal processing, nonlinear system identification and adaptive filtering. Hehas served as an Associate Editor of Circuits, Systems, and Signal Processing in the area of multidimensional signal processing.

Documents

ARMA model parameter estimation based on the equivalent MA