A SIMPLE TEST FOR THE EQUALITYOF CORRELATION MATRICES

by

Kinley Larntz and Michael D. Perlman

TECHNICAL REPORT No. 63

May 1985

Department of Statistics

University of Washington

Seattle, Washington 98195

A SIMPLE TESTFOR THE EQUAIJ1Y OF CORRELATIONMATRICES

by

Kinley Larntz

School of StatisticsUniversity of Minnesota

St. Paul, Minnesota 55101

Michael D. Perlman

Department of StatisticsUniversity of Washington

Seattle, Washington 98195

May 1985

This research report was supported in part by National Science Founda-

lion Grant No. MCS 83-01807.

A SIMPLE TESTFOR THE EQUAlJTY OF CORRELATIONMATRICES

by

Kinley Larntz

School of StatisticsUniversity of Minnesota

St. Paul, Minnesota 55101

Michael D. Perlman

Department of StatisticsUniversity of Washington

Seattle. Washington 98195

May 1985

1. INTRODUCTION

1.1. The testing problem

Suppose R(l) •...• R(k) are sample correlation matrices from k: independent P:

variate normal populations (k :;?; 2). Each R(J3) == (r/p) is based on a random sam-

ple of size np. Denote the f3th population correlation matrix by p(J3)

this paper we consider the problem of the hvpothesis

(piP). In

. e'» = P) ,

where P is an~~~~Q nonsingular correlation matrix, against the ,,""""'OY''''

alternative.

- 2 -

that "unfortunately there is presently no criterion available to test the

H o. As pointed out Jennrich p. 904). computation of the likelihood ratio test

(LRT) statistic would require numerical maximization of a function of

p + P (p - 1) /2 == pZ/2 variables under H 0 • a difficult task unless the dimension p

is very small. In 1967 Kuilback [7] proposed the test which rejects H 0 for large values

of the statistic

k R:z= N (3 log ! R I'f3=! i

where N (3 = n(3 - 1 and R = :z= N (3R«(3)/:Z= N (3. Kullback asserted that under H 0 •

T 1 has an asymptotic xZ distribution with (k - 1)P (p - 1 )/2 degrees of freedom.

However. Aitken ([lJ. Section 5) and Jennrich ([4]. Section 2) have shown that this

assertion is incorrect, the asymptotic distribution of T 1 under H 0 in fact being a

linear combination of independent ~ variates with the weights depending on the unk-

nown value of the common correlation matrix P. One might attempt to estimate

these weights from the data. as Aitken suggests. but this would entail difficult compute-

tions and would alter the significance level in an unknown way for finite sample size.

1.2. Jennrich's test procedure

In 1970 Jennrich proposed a test statistic Tz which suffers from neither of the

above drawbacks -- the computation of T 2 is only of moderate ri1ffirnltv (involving the

inversion of two p x P and Tz does possess an asymptotic t distribution

under For .,UlJ.I-'lJ.....;l~), we confine attention to the case k: 2 now, r",'hll'nina to

the case ~ 3 in the final section. Let denote the p - 1

column vector consistmg of the elements of R (f3) 1 ~ i < j ~ p)

c As 00 random

- 3

nonsingular covariance matrix r = r(p) which depends on P in a known way (see

Section 3). Also, the matrix R = nI3R({3) is a consistent estimator of P under

n«, so r r (R) is a consistent estimator of r. Thus, Jennrich' s test statistic

has an asymptotic X2 distribution with p (p - 1 )/2 degrees of freedom under He,

and He is rejected for large values of T2 . Fortunately, inversion of the

[p (p 1 )/2] x [p (p - 1 )/2 J matrix r is not necessary, for Jennrich ([4], equa-

tion (4.6)) has provided a computationally simpler formula for T2 which only requires

inversion of two p x p matrices.

Although Jennrich' s test has much better computational and distributional proper-

ties than the LRT and Kullback's test, it is basically a large sample test and can perform

poorly for small samples, as we show in Section 3. The main purpose of this paper is to

propose a test statistic Ts which can be readily computed by hand, and which deter-

mines a test with reasonable small sample properties and with power comparable to

that of Jennrich's test for large samples. Furthermore, our test remains valid even if

one or more of the population correlation matrices p(i) are singular, a situation which

may occasionally arise in application and under which Jennrich's statistic

undefined.

'T' ..l. 2 1S

Section 3 contains a summary of our recommendations. In the final section we

discuss the application of our procedures in the case k: 3 , and also discuss the one-

we show how our procedure can be aopuec to obtain simultarie-

ous confidence intervals for the population correlation coefficients in the

- 4 -

2. OTIIER TESf PROCEDURES

2.1. A new test procedure

The basic idea is to apply the Fisher z -transforrnation to each sample correlation

coefficient rit) , obtaining

Z(8) =tJ

and to consider the p (p - 1) /2-dimensional random column vector z (f3) consisting of

the arranged in lexicographic order. Let

d 1/2 C~(l) - ~(2)) is asymptotically normal under H 0 with zero mean vector and

covariance matrix DrD , where D D (P) is the [p (p - 1)/2] x [p (p - 1)/2]

diagonal matrix whose (ij, ij)th entry is (1 - Pi])-l. The significant aspect of the

matrix DrD is that only its off-diagonal elements involve the unknown matrix P , the

diagonal elements of D r D each being 1 By a result of Sidak [13J, therefore, one has

that for any constants bi j > 0,

f

Prob 1d 1/2 zi~l) - Zi~2)l

lJ

)

- 5 -

In particular, if we define the test statistic T3 by

d V 2 maxl::ii<j::ip

then for any b > 0

Prob

asymptotically under H.

;2 bS :2: [<P(b) - <p(-b)]p(P-l)/2

we propose the (possibly conservative) level ex. test

under which Ho is rejected if Ts > bet., where b a > 0 is chosen such that

(2.1)

In effect, this procedure simultaneously tests each of the p (p - 1) /2 univariate

hypotheses p ~p = p ~J) , treating the differences z ~p - z ~J) as if they were indepen-

dent, and relying on Sidak's result to provide an upper bound for the significance level,

Our procedure has several potential advantages over Jennrich's test especially for

small sample size: (a) the value of Ts is easily computable with only a hand calcula-

tor; (b) the z-transform of a sample correlation coefficient r converges in distribu-

tion to normality faster than r itself; (c) larger sample sizes are needed to guarantee

the approximate Yr",lti"",..;;",to normality required of the vectors r(f3) by Jennrich's pro-

cedures T2 (and required of the vectors z(f3) by and T5, two variants of

Jennrich's procedure to be introduced in Section 2.3) than are needed for the approxi-

of each z sample

sizes for1"' _

1 - to be an accurate estimate of ) under n, same is

true for T4 and T 5 ). These may explain the breakdown

the sample SIzes dimension p Sec-

- 6 -

parameters and does not break dm'ITI for small sample sizes. In

may be applied when np < p + 1 for one or more of the "U.Uq..lH.n:l

this procedure

which case R (P)

is singular ), and even when, under H 0 ' the common population correlation matrix P

is singular, in which case the statistics

hence I' will be singular.

2.2. An example

, T4 ' and T 5 are not defined since Rand

Jennrich ([ 4J, Section 6) applied his test statistic T2 to two 12 x 12 sample

correlation matrices R (1) and R (2) computed from test scores of a group of 40

retarded children and a group of 89 normal children respectively

(p = 12, nl = 40, nz = 89). The matrices R(1) and R(2) appear at the bottom

of page 910 of [4]. He found that T 2 = 74.8 which, when referred to the X2 distribu-

tion with 12 ( 11) /2 = 66 degrees of freedom, corresponds to an attained significance

level of .22, hence is not significant at the 5 per cent level. Jennrich attributes this

lack of significance to "the relatively small sample size, 40, of the sample of retarded

children" and to "the similarity of the observed correlation matrices." Inspection

shows, however, that R(l) and R(2) are not very similar. In fact, the average of the 66

absolute difference is .255, and only 4 of the 66 signed differences are

negative. The maximum absolute difference of the z -transforms is

.706, occurring for (i,j) = (9,12), whence we find T 3 = 3.59. Setting b a = 3.59

in (2. and to a table of the normal distribution, one finds that this

corresponds to an attained level of ex = .02 . when to

Jennrich's our procedure is far more sensitive than Jennrichs nrocedure

another indication of the satisractory power nrnnprr.iF'" of

small" sampie sizes.

- 7 -

2.3. Two variants of Jennrich's procedure

The test statistics and are similar to Jennrich' s T2. but are based on the

vectors z{{3) rather than rep) is an attempt to exploit the more rapid convergence of

the z -transtorrns to normality. Still. T 4, and T 5 share with T2 the potential draw-

backs (c) and (d) discussed earlier. The statistics T 4, and T5 are defined by the for-

mulae

where r rUf) s fj = D(R), r = r(R*), fJ = D(R*), and R* = (rtj ) is an

alternate estimator of P under H o defined in terms of z(l) and z(2) by the relation

where Zij = ~p ( np - 3) z ~J) / L;p ( np - 3). The statistics T 4, and T 5 differ only in

the way the asymptotic covariance matrix D r D of d 1/2 (z (1) - Z (2)) is estimated.-- -

Under H 0, both T 4, and T5 have asymptotic X2 distributions with p (p - 1) /2

degrees of freedom. It should be kept in mind that as the sample sizes approach 00,

the tests based on T 2 , s and are asymptotically exact under H 0 , whereas the

test based on will be asyrnptotrcany conservative under unless P = J .

2.4. Summary of Monte Carlo study

In we conducted a Monte Carlo of the actual levels and

""'iin""" of p

- 8 -

n I = nz = 10, 20, 40, 100, and 200.. The results indicated that for small sample

only has an actual significance level which can be trusted to lie in the mf"ll-"t"

of the nominal value 0:, the levels of Tz, T4 ,and T5 deviating greatly from 0: for

some values of the matrix P under H 0.. As n increases, the rejection rates for Tz ,

T4 , and T5 appear to approach the nominal value, whereas the slightly conservative

nature of Ts becomes apparent. Thus, for small samples sizes Ts comes closest to

the nominal level, whereas any of T z , T 4 , or T 5 could be selected for the larger sam­

ple sizes. As expected, for a given sample size, the rejection rates tend to be less accu­

rate and more variable as p increases.

As to power, no one of the four tests is uniformly better than the others. Prelim­

inary investigations showed that the powers of T z , T4, and T 5 are comparable for

moderate to large sample sizes, so in [8] emphasis was placed on comparing the powers

of T z and Ts under a variety of alternatives. When p = 3, we found that in those

cases (75 out of 210 pairs of alternatives (p(I), p(Z) ) ) where the powers of Ta and Ta

differed significantly, more often the advantage lay with T z (59 cases) than "with Ts

(16 cases) (due to the arbitrary natures of the population correlation matrices selected

for the study, the importance of these numbers should not be overemphasized). In

addition, when the power of Tz was significantly larger than that of Ts , the magnitude

of the difference was occasionally quite substantial, whereas this is not the case when

Ts dominates Tz. Part of the reason for the often lower power of Ts is, of course,

its conservative nature, but this does not prevent the conclusion that when p is small

and the sizes are moderate or is test

from considerations of ease of implementation, where has a considerable advan-

When the dimension p 5 or 10, the relative status of T z and

in power often --

and 10 , we found

- 9 -

pairs of alternatives ( , p(2)) where the estimated power of

was np,:,r!v 1 , while that of was much lower, often nearly O. Each of these

pairs of alternatives involved some very large correlation coefficients pij,0.9 or

There were no cases where the estimated power of was nearly 1 while

that of Ta was substantially lower. It should be mentioned that unlike T2 , the statis­

tics T 4 and T5 did not exhibit instances of extremely low power.

3. RECOMMENDATIONS

Because the nominal significance levels of the tests based on T 2 , T 4, and T 5

have been found to be unreliable for small sample sizes, Ts is the clear choice in this

situation. Based on the cases studied, we propose the following rule-of-thumb: when

the ratio of sample size to dimension does not exceed 4, i.e., when

min ( n 1, n2) /p :::;; 4, then Ta is recomm ended. (As p increases, it is likely that

the constant 4 should be increase also.) Thus, for the values of (p,n) studied in

[8], Ts should be used in the following cases: (p,n) = (3,10), (5,10), (5,20),

(10, 10), ( 10, 20), ( 10, 40). (Note too that in the example of Section 2.2.,

min(nl,n2)/p = 40/12:::;; 4,and Ts was found to be more sensitive than Tz .)

For moderate and large sample sizes the choice is less clear-cut. Certainly T3 is

the easiest procedure to implement, and it readily provides a set of simultaneous

confidence intervals for all differences ,uU) - ,u~J)' 1:::;; i, j :::;; p, where ,u~f)

denotes the z -transform of Plf) -- also see Section 4. For sample sizes we sug-

that any of T5 should be since are exact

is conservative and more "''''Arc,rf, d for most alternatives. Of course, for

the convenience ofmoderate sample

outweich a r elativelv small loss in power.

may

we remark that as the sample sizes

- 10 -

4. THE k-SA.\£PLE AND ONE-SAMPLE CASES

When the number k: of populations is 3 or more, the hypothesis H 0 can be tested

by means of Jennrich's extension of T2 , given in equation (7.2) of Alternatively,

our procedure can be extended to the k-sample case as described next, and should be

expected to retain its advantageous small-sample properties. For fixed i < j let

t t , - (fl{\) l1(k) z .. - (zO) z(J;) and let H·, denote the univariater:=..1.J - \ ~tJ " .. , 1'""'1-) J _ tJ - c: 1.) " .. , 1.) ,..l l.J v

hypothesis

(1)H ij : f-Lij

11(1<;)r-tJ ' l~i<j~p.

AE min ( n l' ... , nk) -> 00, the random vector }£ij has an asymptotic k -variate normal

distribution with mean vector and covariance matrix

b.. = diag [(n1 - 3)-1, ... , (nk - 3)-1]. The hypothesis H i j can be tested by means

of the statistic

S ij

r k 12

I~ (n{J - 3) z ff) j1{J=1. ,

k

~ (n{J 3){J=1

(see Rao [l1J, p. 434-5), which reduces to

k(n - 3) ~ (z

{J-1

when n ri , where Zij z Under H ij' S ij has as

distribution. Since is the intersection of the p /2

univariate H ij , we introduce the statistic

- 11 -

and propose the test which rejects H a if T; > Xk~t,e(a)' where Xk-l,e is the upper t;

point of the Xk2_ t distribution and t; (Cf.) = (1 - rx )2/p(P-t) .

[5],

Corollary 5' of Khatri

Prob ITs 1 - ex

asymptotically under H a ' so this procedure yields a (possibly conservative) level ex

test.

Our proposed procedure for the k-sample p-variate problem may be regarded as

a generalization of the Pearson-Wilks procedure for testing equality of the correlation

coefficients from k: independent bivariate normal populations (see [10] or [11], pp.

434-5). For moderate sample sizes, the Xk2_ t approximation can be improved if S ij is

modified as indicated in the formula for H on page 436 of Rao [1].

Next. we consider briefly the one-sample case. Let R = (r ij) be a sample corre-

lation matrix based on a random sample of size n from a p -variate normal distribu-

tion with population correlation matrix P = (Pij)' Consider the problem of testing

the hypothesis H o: P = Po, where Po = (Pij,O) is a specified correlation matrix. If

Po is the identity matrix, H a is the standard hypothesis of independence of a set of

p-variates and may be tested by means of Bartlett's modified LRT [9]. See also M. Hills

[3J for a graphical method. For other values of Po, Jennrich's method may be applied

as follows. Let r denote the p (p - 1) /2-dimensional column vector consisting of

the elements r ij (i < j) of R s and similarly define E!.- and 120 in terms of the ele-

ments of P and As n -;. 00 r- is normal with zero

mean vector and covariance matrix 1f;

P in a known way. Thus the statistic

1f; P) . where 1/1 depends on the elements of

- 12-

has an asymptotic X2 distribution under H 0 with P (p 1) /2 riPicrrF'p" of freedom,

-where 1/Jo = 1/J(Po) , and Jennrich proposes rejection of Ho for large values of 12.

He gives a computationally simpler formula for T2 in equation (7.1) of

Our method applied to the one-sample testing problem leads to the statistic

(n 3 )1/2 m.ax Z ij - fJij ,0 ,t;:;i<j :O;p

where Z ij is the z -transform of r ij and fJij ,0 is the z -transforrn of P ij ,0 . By

Sidak's Theorem [13] the test which rejects H o if 1§ > b a • where b a is determined

by (1.1), is a (possibly conservative) level ex test of H o . Our procedure may be

expected to behave slightly better than Jennrichs for small samples since T3 relies.....,

only on the appropriate univariate normality of the z ij whereas 12 relies on the rnul-

tivariate normality of the vector I-. However. the difference should not be expected to

be as pronounced here as in the k-sample case (k :f:: 2), since the statistic 12 does

not require the estimation of a covariance matrix, unlike T2. The procedure based on

T3 has also been discussed by Morrison ([9], pp. 119-120); he uses the somewhat

weaker Bonferoni bound rather than the Sidak's inequality,

On the other hand, our procedure docs have the advantage that it readily yields a

set of simultaneous confidence intervals for the p (p - 1) /2 population correlation

coefficients Pij' If we let fJij denote the z-transform of Pij , asymptotically the pro-

bability is at least 1 ex that the inequalities

(l~i<j~p)

are simultaneously satisfied. These me.quanties are then inverted to obtain simult.ane-

ous confidence bounds the Jennrichs procedure

perform

dimension p .

(as in the k-sample

- 13 -

if the sample size n is small relative to the

Finally, we remark that in both the one-sample and k: -sarnple case, our method

can be applied much more readily than Jennrich's procedure if it is desired to restrict

attention to some arbitrary subset L of the p (p - 1) /2 indices (i, j) and test the

hypothesis

P OltJ

(in the k-sample case) or

D-­, tJ

(in the one-sample case).

Pij,O

P (~ )tJ ' (i,j)c:L

(i,j)c:L

- 14-

References

[1] Pitkin, M. 1969. Some tests for correlation matrices. Biometrika. 56: 443-6.

[2] Bartlett, M. S., 1933. On the theory of statistical regression. Proceedings of the

Royal Society, Edinburgh 53: 260-83.

Hills, M., 1964. On looking at large correlation matrices. Biometrika. 56: 249­

253.

[4] Jennrich, R. 1., 1965. An asymptotic X2 test for the equality of two correlation

matrices. Journal of the American Statistical Association. 65: 904-12.

[5] Khatri, C. G.. 1967. On certain inequalities for normal distributions and their

applications to simultaneous confidence bounds. Annals of Mathematical Statis­

tics. 38: 1853-67.

[6J Knuth, D. Eo, 1969. The Art of Computer Programming, Vol. 2: Semiriurnericoi

Algorithms. Addison-Wesley. Reading, 1vfA.

[7J Kullback, S., 1967. On testing correlation matrices. Applied Statistics. 16: 80­

5.

[8] Larntz, K.. and M. D. Perlman. 1985. A simple test for the equality of correlation

matrices. Unpublished rp,">("\lrt

St. Paul. MN.

Department of Statistics. University of Minnesota,

Morrison,

York.

. 1976. Multivariate Statistical Methods, 2nd ed. New

E. S., and S. S. 1933. Methods of statistical anaIV'SIS "lnnrnnri;,rp for

k samples of two variables. Biometrika. 25: 353-78.

1 c. 1973. Linear Statistical and its A;z:rpliccLti,ons, 2nd ed, John

and NI.

- 15 -

[12] Seal, H. L., 1964. Multivariate Statistical Analysis for Biologists. John Wiley and

Sons, Inc., New Nr'.

[13] Sidak, Z., 1967. Rectangular confidence regions for the means of multivariate nor­

mal distributions. Journal of the American Statistical Association. 62: 626-33.