Journal of Applied Psychology1978, Vol. 63, No. 6, 689-697

Uses of Discriminant Analysis Following MANOVA:Multivariate Statistics for Multivariate Purposes

Fred H. Borgen and Mark J. SelingIowa State University

When a test of multivariate analysis of variance is found to be significant, itmust be followed by other analyses before a researcher can arrive at an accu-rate understanding of the data set. Two possibilities for follow-up analysesinclude univariate analysis of variance and discriminant analysis. This articlepresents the results of a Monte Carlo study wherein typical, but simple, multi-variate data were analyzed by the two techniques. The results demonstrate thatdiscriminant analysis is capable of showing the underlying dimensionality of thedata as well as determining the contribution of individual variables to theunderlying dimensions, whereas analysis of variance is limited to specifying thecontribution of each variable to group separation. The authors present the argu-ment that when researchers analyze multivariate data, primary goals becomeinterpretation and understanding the data set. It is concluded that discriminantanalysis is most suitable for this purpose.

Multivariate statistical methods for psy-chological research are now generally avail-able (e.g., Bock, 1975; Tatsuoka, 1971).Since the methods are complex and interre-lated in numerous ways (see below), there isa risk that such techniques as multivariateanalysis of variance (MANOVA) will be appliedsimplernindedly. At the most basic level, thisarticle attempts to clarify the single issue offollowing up a significant MANOVA in an effortto determine the simple effects. In so doing,we present the case for using discriminantanalysis. However, in order to adequatelytreat this topic, we begin with a more generaldiscussion of multivariate statistics, univari-ate statistics, and their relationships. Further,we have found it necessary to make generalcomments on the goals of data analysis toillustrate clearly some principles to guideusers of MANOVA in their follow-up analyses.

Uses of A NOVA and MANOVA

MANOVA may be distinguished from uni-variate ANOVA by the fact that with theformer one may simultaneously analyze the

Requests for reprints should be sent to Fred H.Borgen, Department of Psychology, Iowa State Uni-versity, Ames, Iowa 50011.

experimental variables' effect on a set of de-pendent measures (cf. Bock, 197S). BothANOVA and MANOVA provide overall, omnibustests for significant differences among groups.Therefore, if significant results are obtainedfor the omnibus test, follow-up analyses arerequired in order to determine which of thevarious single effects are significant.

Techniques for following up ANOVA arenumerous and well established (e.g., Kirk,1968; Winer, 1971). The state of the currentliterature for following up significant MANOVAresults, however, is neither tidy nor simple.The area is complex, and no Winerlike figurehas emerged as a widely referenced authority.Thus, we do not presume that all of the an-swers are in. Rather, the present study wasdesigned to provide an empirical demonstra-tion of the utility of discriminant functionanalysis for following up MANOVA. The moti-vation for the empirical work reported herearose from Specter's (1977) overstated casefor the use of one-way F tests for the samepurpose. A primary aim of this article hasbeen to illustrate the special interpretive fea-tures of discriminant analysis when used inconjunction with other multivariate tech-niques such as MANOVA,

Consider the typical multivariate research

Copyright 1978 by the American Psychological Association, Inc. 0021-9010/78/6306-0689S00.75

689

690 FRED H. BORGEN AND MARK J. SELING

study, in which the investigators have ob-tained measurements for each of n variableson each subject (persons, environments, etc.)in each of the p groups under study. In theset of n variables, some will'be related to eachother. That is, n, and n2 might be related(correlated) variables, while neither is asso-ciated substantially with ;;,;. In this simplecase, a factor analysis of n\, n->, and «;j mightshow that there are two factors, or dimen-sions, that account for the relationships in thedata set. This, then, is the point we wish toemphasize. A set of rnultivariate data usuallyconsists not only of many variables, but thesevariables reflect a simpler set of underlyingdimensions. Multivariate data occupy multi-variate space.

Using only ANOVA, a researcher would beforced to approach such data only in piece-meal fashion. A multivariate strategy, on theother hand, makes it possible to encompassthe data all in one analysis. Bock (1975)states that the advantages of the multivariateapproach are (a) providing a clearer, betterorganized account of the investigation and(b) yielding more realistic probability state-ments in hypothesis testing and interval esti-mation (p. x i i ) .

The major use of MANOVA is in makingoverall tests of group differences on a multi-variate set of, data. Although the basic sta-tistics have been present in the mathematicalstatistics literature since the 1930s and 1940s(Anderson, 1958; Rao, 19S2, 1973), practicalapplications of the techniques have becomepossible only with the emergence of moderncomputer technology. Several books have beenwritten for the applied researcher in the be-havioral sciences. Bock (1975) is probablythe most comprehensive treatment, but it isat a fairly high level of difficulty. Tatsuoka(1971) has a simpler text that is quite suc-cessful at conveying an intuitive understand-ing of MANOVA and related techniques; Tatsu-oka (1970) is a brief treatment designed forready understanding, and Tatsuoka (1973)is a highly readable and informal descriptionof the relationships and applications of multi-variate methods. Other basic texts for the be-havioral scientist that treat MANOVA areOverall and Klett ( 1 9 7 2 ) , Cooley and Lohnes

( 1 9 7 1 ) , Van cle Geer ( 1 9 7 1 ) , Timm (1975 ) ,Finn (1974), and Harris (1975).

The key issue necessitating the whole com-plex development of multivariate statisticalmethods has been the recognition that multi-ple dependent measures actually reflect asimpler statistical set of underlying dimen-sions. When following up a MANOVA, what isneeded is a technique that f i ts this situationand that yields information about group sepa-ration as well as the underlying dimension-ality of the variables. It will be shown thatunder these typical multivariate conditions,discriminant analysis is a helpful follow-uptechnique, while ANOVA may not be adequate.

Relations Between MAX OVA andDiscriminant A nalysis

Over the past decade, training of behavioralscientists has increasingly acknowledged thefundamental statistical equivalence of uni-variate ANOVA and multiple regression (Dar-lington, Weinbcrg, & Wai berg, 1973, pp. 439-440). Each is a case of the general linearmodel when groups in the ANOVA are repre-sented by "'dummy variables." Analogously,though at a greater level of multivariate com-plexity, the statistical tools of MANOVA,canonical correlation, and discriminant func-tion analysis have a fundamental base in thegreater linear model (Bock, 1975; Darlingtonet al., 1973; Tatsuoka, 1973). Thus, bothIIANOVA and discriminant analysis are meth-ods for looking at mult ivariate differencesamong groups. When the groups are convertedto ' 'dummy variables," MANOVA and discrimi-nant analysis become special cases of canoni-cal correlation, and thus the general linearmodel. The equivalence of these three tech-niques is highlighted implicitly b)' Bock's(1975) major text in which all are mergedinto his presentation of the general multi-variate linear model. Although the methodsshare a mathematical core, differences remainin the kinds of concepts and questions eachaddresses.

R. A. Fisher (1936) developed and popu-larized discriminant function analysis as atechnique for multivariate classification ofindividuals (objects) in groups. However,subsequent work, often by applied psycholo-

DISCRIMINANT ANALYSIS 691

gists, has extended the uses of discriminantanalysis well beyond classification. Key workderives from a Harvard group led by PhillipRulon (Rulon, Tiedeman, Tatsuoka, &Langmuir, 1967). Rulon's students who havebeen at the forefront of advocating and dem-onstrating the versatility of discriminantanalysis are Tatsuoka (1970, 1971) andCooley and Lohnes (1971). They view dis-criminant analysis as a prepotent conceptualtool for analysis of multivariate group differ-ences. Cooley and Lohnes (1971) explicitlyincorporate Tukey's (1962, 1969) ideas aboutusing data analysis for exploration. Theirskill in merging this viewpoint with dis-criminant analysis is well exemplified in theirmastery of the massive Project Talent data(Cooley & Lohnes, 1968); this work is wellworth stud)' by someone wishing to learn touse discriminant analysis with versatility andcreativity. The points of view expressed hereabout discriminant analysis follow closelythose of Tatsuoka and Cooley and Lohnes; inour view they also provide the best currentmodels for effectively applying discriminantanalysis in applied research. Huberty (197S)provides the mose comprehensive and modernreview of the statistical developments andmany uses of discriminant analysis.

In a multivariate context, discriminantanalysis and its recent elaborations provideanswers to the following questions (cf . Cooley& Lohnes, 1971; Huberty, 197S; Tatsuoka,1971): (a) How many basic underlying di-mensions are represented by the multivariatedifferences between groups? (b) How mightthese underlying dimensions be psychologi-cally interpreted? (3) What are the individualcontributions of the original variables to theseunderlying dimensions? (d) How can thegroups be spatially represented? The answersto these questions give the researcher a richconceptual understanding of a data set thatenhances a full exploratory data analyticstrategy (Tukey, 1977).

With the emergence of the data analyticstrategy, which implicitly assumes no singleone right technique for a given problem, theneed for sophisticated statistical training be-comes all the more obvious. There is no waythat a data-analytic strategy can be "canned"

with a user merel}' using a cookbook to selecta brief set of computer control cards.

Spector's Advocacy of Follow-tip AN OVA

Spector (1977) argues that discriminantanalysis is not the appropriate technique foridentification of significant differences follow-ing a MANOVA. He advocates the use of aseries of univariate ANOVAS for follow-upanalyses after a significant MANOVA omnibustest. His empirical results, based on MonteCarlo data he generated, appear to buttresshis conclusions. Central to his argument isthe fact that the weights derived from dis-criminant analysis are misleading for interpre-tive purposes because they are distorted bythe intcrcorrelations among the dependentvariables. His summary Table 2, which pro-vides the evidence, shows that univariateANOVA F tests are clearly superior to the dis-criminant weights in reflecting "true" differ-ences in the case of correlated dependentvariables.

Critique oj Spector's (1977) Study

At first blush, Spector's evidence appearsconvincing and the case would seem to beclosed. But he has presented only a special,and atypical, case of multivariate data and alimited use of discriminant analysis. We wishto show that when more complex and typicalmultivariate data are modeled and when thef u l l capabilities of discriminant analysis areused, the apparent advantages of ANOVA fol-low-up disappear and the utility of discrimi-nant analysis is evident.

Spector has made some points with whichwe heartily agree. Among the most practicallyimportant of these is that the discriminantfunction weights have little interpretive utilitywhen the predictor (dependent) variables arecorrelated. Like multiple regression weights(e.g., Darlington, 1968), the discriminantweights are mathematically derived to bestpredict, in the least squares sense, the newlinear composites (discriminant variates). Inthe fitting of these weights with correlateddata, the prediction is maximized but the sub-stantive significance of the weights is oftenlost. Frequently where there are two highly

692 FRED H. BORGEN AND MARK J. SELING

correlated variables with similar relationshipsto the criterion, only one of the variables willbe weighted heavily because of the predictorredundancy. Yet it would be incorrect toconclude that the weakly weighted variableis unrelated to the criterion. The problemexists whether the raw or the standardizeddiscriminant weights are used. Unfortunately,some of the popular packaged programs fordiscriminant analysis (e.g., Nie, Hull, Jen-kins, & Steinbrenner, 1975, p. 443) suggestgreater interpretive util i ty for the weightsthan is justified (cf . , Hubert}', 1975). Theweights are clearly useful only when the origi-nal variables are uncorrelated, such as withorthogonal exact factor scores; here the stan-dardized weights must be used to adjust fordifferences in scale variances.

Spector (1977) is also correct in advocatingANOVA follow-up when an investigator is in-terested in determining which specific vari-ables may have contributed to an overall sig-nificant MANOVA. However, as we noted in aprevious section, repeated use of univariatetests will yield unrealistic probability esti-mates for hypothesis testing. This happensbecause many of the dependent variables willbe intercorrelated. In the face of correlatedvariables, results may appear very differentlyif treated univariately or multivariately. Ifall outcome measures were orthogonal, thenunivariate tests on the many dependent vari-ables would be quite adequate. Yet we knowempirically that this is rarely the case. Toomany investigators run univariate tests with-out stopping to evaluate the dependencies(correlations) among the variables. Thus, inthe worst examples of such work, we see re-ports of several significant outcome measures,all of which may be tapping a single, generalunderlying factor, such as general intelligence,job satisfaction, or response style.

The crucial limitation of Specter's articleis that the theoretical model he uses presumesthat the groups are differentiated along asingle underlying dimension. Because his datahave been developed for a limited case, uni-variate ANOVA is adequate to handle unequiv-ocally the follow-up analyses to MANOVA. Butby definition, typical MANOVAS will involvedata with structure best represented by mul-

tiple underlying dimensions. Thus, follow-upanalyses to MANOVA will be most effectivewith techniques able to reflect the multivari-ate structure. Discriminant analysis is such apotentially useful approach.

Another central limitation of Specter'scomparative analysis is that he fails to applyall of the accepted interpretive tools of dis-criminant analysis. Specifically, several es-tablished authors in the area (e.g., Cooley &Lohnes, 1971; Tatsuoka, 1970, 1971) dem-onstrate the interpretive value of the dis-criminant structure matrix, which shows thecorrelations between the original predictorvariables and the derived discriminant func-tion scores. This matrix, which has the formof a factor matrix, directly shows the rela-tionship of each variable to the underlyingdiscriminant dimensions without the con-founds of scale intercorrelation. This ap-proach was developed to avoid the knowninterpretive limitations of discriminantweights, limitations that are so central toSpecter's arguments against the use of dis-criminant analysis.

The use of the discriminant structure ma-trix has become fairly common (e.g., Adams,Laker, & Hulin, 1977), although unfortu-nately it has not been available in many ofthe popular computer analysis packages (e.g.,Barr, Goodnight, Sail, & Helwig, 1976; Nieet al, 1975). The Statistical Package for theSocial Sciences (SPSS: Nie et al., 1975)added structure matrix output in the fall of1978. Structure matrix output is availablefrom the Cooley and Lohnes (1971) programfor discriminant analysis. Moreover, it can becalculated fairly readily if the user has somecomputing skill. In a two-stage process thediscriminant scores first could be calculatedfor each individual from the discriminantweights and the original predictor variablesand then the derived discriminant scorescould be correlated with the original predictorvariables.

Objectives of This Study

The empirical study that follows, usingMonte Carlo data with known multivariatestructure, is designed to permit comparisono f ' discriminant analysis and ANOVA as fol-

DISCRIMINANT ANALYSIS 693

Table 1Theoretical and Obtained Means for Nine Groups and Six Variables

Variable

Group/source

1

2

3

4

5

6

7

8

9

TheoreticalObtainedTheoreticalObtainedTheoreticalObtainedTheoreticalObtainedTheoreticalObtainedTheoreticalObtainedTheoreticalObtainedTheoreticalObtainedTheoreticalObtained

XI

35.032.460.062.085.086.935.035.060.064.985.092.335.033.960.058.185.082.1

X2

50.050.875.071.5

100.0100.250.0S1.675.075.9

100.0105.650.053.475.071.9

100.0101.1

X3

65.062.190.090.3

115.0112.365.069.090.093.1

115.0113.765.065.190.089.8

115.0114.2

Yl

60.049.260.057.460.058.085.088.585.082.185.086.497.599.497.592.797.596.4

Y2

75.078.775.072.375.076.4

100.0103.7100.099.9

100.0103.1112.5112.5112.5115.4112.5110.7

Y3

90.090.890.092.390.092,4

115.0114.8115.0112.0115.0111.4127.5127.6127.5128.5127.5127.5

low-up analyses for identifying and interpret-ing specific effects within multivariate data.This comparative study parallels Specter's(1977) but uses more complexly structureddata designed to illustrate the full advantagesof discriminant analysis.

Method

Monte Carlo Sample

Our procedure was to develop random samples ofdata fitting specified population structure and thencompare the utility of the statistical methods forrecovering the structure. This study includes ninegroups of 50 each, separated on six variables whichmap into an underlying (wo-dimensional space.

Table 1 shows the population means specified forthis study and the obtained means for samples con-structed with a random number generator. The theo-retical model for the means of nine groups in atwo-dimensional space is depicted in Figure 1. Aconstant standard deviation of 25 was applied withineach of the nine groups on each of the six variables.This created a ratio of within-group standard devia-tion to mean differences of groups of 1: 1.5 acrossthe span of the vertical space and 1: 2 across thespan of the horizontal space.

Variables Yl, Y2, and Y3 were constructed to re-flect the vertical dimension of the underlying space,and Variables Xl, X2, and X3 similarly mapped thehorizontal dimension. Theoretical within-groups cor-relations were specified as follows: .60 between XIand X2 and between Yl and Y2; .25 between Xland X3 and between Yl and Y3. The pooled within-

groups intercorrelations of the six variables obtainedfor the randomly generated sample of 450 are shownin Table 2.

Analyses

The group differences across the multivariate datawere analyzed separately by discriminant functionanalysis and univariate ANOVA. The results from thetwo methods were compared with respect to theircapacity for recovering known features of the data.

Results

Analyses of Variance

Six univariate ANOVAS were used to calcu-late F ratios reflecting the separation of the

37.5

25.0

25 SO

Figure 1, Theoretical model of nine groups in a two-dimensional space defined by six variables.

694 FRED H. BORGEN AND MARK J. SELIXG

Table 2Pooled Witliiii-Croiips Correlation Matrix

Table 4Discriminant Analysis Results

Vari-able

XIX2X3YlY2Y3

XI

1.00.60.23.05.01.06

X2

1.00.18.01

-.04.02

X3

1.00.01

-.01.03

Yl Y2 Y3

1.00.64 1.00.28 .24 1.00

Note. N = 450.

nine groups on each of the original six varia-bles. These Fs are shown in Table 3. All arehighly significant, implying that each varia-ble makes a contribution to group separationin the multivariate space. The Fs for varia-bles XI, X2, and X3 are somewhat higherthan those for Yl, Y2, and Y3, suggesting(correctly) that group separation is somewhatgreater for three of the six variables. No fu r -ther information about either the group sep-aration or the structure of the data is readilyextractable from these ANOVA results. Theysay nothing definitive about the dimensional-ity of the underlying space.

Discriminant Function Analysis

Using primarily the SPSS program (Nie etal., 1975), the nine groups were comparedby discriminant analysis, with results shownin Table 4. Theoretically, six variables andnine groups could be separated along six dif-ferent discriminant dimensions, provided allvariables were independent. Table 4 resultsclearly show two significant discriminantfunctions, indicating an underlying two-di-mensional space for the data. Having estab-

Discrim-inant

function

1234S6

Percentvariance

63.733.8

1.1.7.6.1

Canonicalcorre-lation

.75

.64

.15

.12

.11

.05

Significance ofdiscriminant

X2

622.1254.3

22.713.16.71.1

P

.000

.000

.535

.589

.569

.788

discriminant analysis are directed to showinghow the original variables separately contrib-ute to the two underlying discriminantvariates.

Shown in Table 5 are the standardized andunstandardizecl weights for the two discrimi-nant functions. Neither set of these weightsprovides particularly useful information aboutthe structure of the data. (The SPSS manual(Nie et al., 1975) errs in stating the stan-dardized weights provide routinely useful in-formation about the relative contribution ofthe variables to multivariate separation.] Forboth the unstandardizecl and standardizedweights, the highest relative values appear forvariables X3 and Y3, despite the fact thatthese variables have the weakest correlationswith their respective underlying dimensions.These apparently anomalous weights illus-trate the common paradox with linear com-posite weighting systems (Darlington, 1968).Since the X1-X2 and Y1-Y2 pairs are quiteredundant, the discriminant weights assigned

Table 5Discriminant Weights

Table 3Univariale ANOVA F Ratios forDifferences Between Nine Groups

Variable

Statistic

1'

Note, df =

XI X2 X3

42.1 41.6 35.1

8. 441. All Fs sis:

Yl Y2 Y3

22.9 21.7 21.3

ilficant at .001 level.

Unstandardi/.eddiscriminant

Variable

XIX2X3YlY2Y3

Constant

I

.010

.012

.014-.002

.001-.002

-2.546

I I

-.005.005.002.011.012.018

— 4.321

Standardizeddiscriminant

I

.343

.377

.449-.072

.041-.051

I I

-.161.170.064.341.351.499

DISCRIMINANT ANALYSIS 695

to each of these separate variables are lower.These results illustrate how the pattern ofintercorrelations among the variables may of-ten affect the weights so that they do not di-rectly reflect the relationships of the originalvariables and the criterion. Thus, the weightsare not a secure basis for inference. Fortu-nately, the discriminant structure matrix pro-vides information which is more directly in-terpretable.

The discriminant structure matrix was notprovided by the SPSS program but was cal-culated with an ad hoc Fortran program.Using the unstandardized discriminant weights(Table 5), two discriminant scores (I andII) were calculated for each of the 450 "sub-jects" in this study. The six original predictorvariables were then correlated with each ofthe two discriminant scores for the sample of450 to produce the discriminant structurematrix shown in Table 6. This matrix showsthe direct correlation between the originalvariables and the derived discriminant scores.It is apparent that Variables XI, X2, and X3contribute to separation on Discriminant I,and that Yl, Y2, and Y3 load on Discrimi-nant II. Although the within-factor loadingsare all quite high, the relative contributionsof the original predictors are quite differentfrom the relative discriminant weights inTable 5. The correlations are rank orderedin the same way as the underlying theoreticalmodel. However, the absolute differences be-tween the correlations have been sharplytruncated. This reflects the fact that therewas equal group separation on the original

Table 6Discriminant Structure Matrix

Discriminant

Variable II

XIX2X3YlY2Y3

.873

.866

.824-.059-.052-.043

.Oil

.081

.076

.838,823.819

Note. Values shown are correlations between theoriginal six variables and the two discriminantvariates for the total sample of 450.

II

-1

-1 0

I

+ 1

Figure 2. Nine empirical groups plotted in a two-dimensional space defined by the discriminant di-mensions.

variables, despite the differences in intercor-relations of these variables with the under-lying dimensions. The discriminant functionsare weighted to maximize group separation,and thus they gave relatively heavy weightsto Variables X3 and Y3.

Finally, the SPSS discriminant analysisprogram produced output showing the multi-variate means (centroids) of each of the ninegroups on the discriminant dimensions. Theseresults are shown in Figure 2 in a plot of thediscriminant space. This figure directly showsthe capacity of the discriminant method torecapture the structure of the data, as origi-nally modeled in Figure 1. A full understand-ing of the data is possible when this graphicinformation is combined with knowledge ofthe dimensionality of the space (Table 4)and the contributions of the original variablesto the discriminants (structure matrix, Table6). It becomes apparent from the graphic in-formation that the groups are not equidis-tant in the space but vary in their relativedistance and thus statistical separation.

Discussion

This study has contrasted univariate ANOVAand discriminant analysis as potential follow-up techniques to a significant omnibus testin MANOVA. The focus has been on the amountof useful information each yields about typi-cally complex multivariate data. Unlike Spec-tor's (1977) analysis with a simpler data set,

696 FRED H. BORGEN AND MARK J. SELING

the results of this comparison show that dis-criminant analysis has several advantages foradequately reflecting the multivariate char-acter of data. When data truly are multi-variate, as implied by the application ofMANOVA, a multivariate follow-up techniqueseems necessary to "discover" the complexityof the data. Discriminant analysis is multi-variate; univariate ANOVA is not.

Spector draws a dichotomy between hy-pothesis testing and prediction/classification,concluding ANOVA is useful for the former pur-pose and discriminant analysis for the latter.In our view, this dichotomy incompletely rep-resents the goals of statistical analysis. Wesubmit that interpretation and understandingof data, particularly when multivariate, areother important functions of methods in ap-plied psychology. This goal includes the "de-tective" and "discovery" mode as a data-analytic strategy, first articulated by Tukey(1962, 1969) and explicitly implemented withmultivariate methods in the Cooley andLohnes (1971) text. Tukey's data-analyticviewpoint is presented in a text (Tukey,1977) that one reviewer (Wiley, 1978) seesas announcing a conceptual revolution in sta-tistics which could be equivalent to thatplayed by R. A, Fisher's Statistical Methodsjor Research Workers. Wiley (1978) sees thatTukey is part of a "revolution currentlyshaking the field of statistics," based on acompletely new set of principles, including thefollowing: (a) For a single category of prob-lem, there is no single best technique andthere never will be, (b) there is much to befound in any data set which cannot be antici-pated, and (c) a summary picture is wortha thousand summary numbers. According toWiley, Tukey's data-analytic viewpoint is"based on the notion that data are rich andcomplex; if they are worth having in the firstplace, they are usually worth systematic ex-ploration going far beyond the immediate pre-conceptions which led to their collection"(p. 153).

ANOVA is generally conceived as the appro-priate method for hypothesis testing. ANOVAis useful, as Spector states, for specific hy-potheses about single variables and groupseparation. However, discriminant analysis

provides the test for the number of significantunderlying dimensions accounting for groupdifferences. This is a central issue for an ade-quate understanding of multivariate data, andthe formal equivalent of the test has been in-corporated in the general MANOVA procedure(Bock, 1975). When conducting multivariateresearch, it is usually of interest to examinethe relations among the dependent variables.We would argue that even when these inter-relationships do not have theoretical interest,they may have great statistical import, andthus should be routinely examined for an ade-quate understanding of multivariate data.

In his computer example, Spector givesshort shrift to the value of the discriminantstructure matrix. He presents the results(labeled as "factor structure matrixes") inTable 2 but does not mention them until theconcluding section. In the earlier section inwhich he contrasts results of the two ap-proaches, he attends only to the discriminantweights, showing they are faulty for recover-ing the "true" relationships. But when hebelatedly refers to the discriminant structurecoefficients in his conclusions, he acknowl-edges that they, like the F tests, give "cor-rect" information for both the uncorrelatedand correlated cases. Thus, by attending tothe discriminant weights but ignoring the dis-criminant structure coefficients, he reaches afaulty conclusion about the deficiencies of dis-criminant analysis.

The recent study by Adams et al. (1977)illustrates how omnibus MANOVA results canbe effectively followed by both univariateANOVA and discriminant analysis techniques.Appropriately, they used ANOVA F tests toindicate the separate contributions of singlevariables to group separation. Then, discrimi-nant analysis was used to interpret the multi-variate structure of their data. The qualityof the interpretations they were able to makewas considerably enhanced by their use ofthe test for the number of significant dis-criminant dimensions, a discriminant struc-ture matrix, and a plot of the means for their15 groups in discriminant space. Their in-terpretations could be made with particularconfidence because of their large sample size(N — 1,313); typical samples for other in-

DISCRIMINANT ANALYSIS 697

vestigators will be smaller, and discriminantanalysis, while still useful, will require morecautious interpretation. Finally, it is note-worthy that Adams et al. use discriminantanalysis solely as an interpretive tool; theyare not explicitly concerned with the pre-dictive or classificatory role which Tatsuoka(1977) believes has been too narrrowlyascribed to discriminant analysis by somewriters (e.g., Bock, 1975). Tatsuoka advo-cates for discriminant analysis the kind ofinterpretive role exemplified by the work ofAdams et al.

As a data-analytic and interpretive tech-nique, discriminant analysis is uniquely use-ful for identifying the underlying dimension-ality of multivariate data, for showing thecontribution of the individual variables to theunderlying dimensions, and for providinggraphic displays of the data which directlyaid interpretation. Univariate ANOVA does notprovide such information for multivariatedata. ANOVA is useful and desirable, however,for specifying the individual contribution ofeach variable to group separation. As such,ANOVA results should be combined with thoseof discriminant analysis to indicate groupseparation in multivariate space. Since mostdiscriminant analysis programs also routinelyprovide the results for univariate ANOVA, thediscriminant approach seems to be the mostcomprehensive method of data analysis avail-able for following up a significant MANOVA.

References

Adams, E, F., Laker, D. R., & Hulin, C. L. An inves-tigation of the influence of job level and functionalspecialty on job attitudes and perceptions. Journalof Applied Psychology, 1977, 62, 335-343.

Anderson, T. W. An introduction to multivariatestatistical analysis. New York: Wiley, 1958.

Barr, A. J., Goodnight, J. H., Sail, J. P., & Helwig,J. T. A user's guide to SAS'76. Raleigh, N.C.:Sparks Press, 1976.

Bock, R. D. Multivariate statistical methods in be-havioral research. New York: McGraw-Hill, 197S.

Cooley, W. W., & Lohnes, P. R. Multivariate dataanalysis. New York: Wiley, 1971.

Darlington, R, B. Multiple regression in psychologicalresearch and practice. Psychological Bulletin, 1968,69, 161-182.

Darlington, R. B., Weinberg, S. L., & Walberg, H. J.Canonical variate analysis and related techniques.Review oj Educational Research, 1973, 43, 433-454.

Finn, J. D. A general model for multivariate analysisof variance. New York: Holt, Rinehart & Winston,1974.

Fisher, R. A. The use of multiple measurements intaxonomic problems. Annuals oj Eugenics, 1936, 7,179-188.

Harris, R. J. A primer of multivariate statistics. NewYork: Academic Press, 1975.

Huberty, C. J. Discriminant analysis. Review ofEducational Research, 1975, 45, 543-598.

Kirk, R. E. Experimental design: Procedures for thebehavioral sciences. Belmont, Calif.: Brooks/Cole,1968.

Nie, N. H., Hull, C. H., Jenkins, J. G., Steinbrenner,K., & Bent, D. H. Statistical package for the socialsciences (2nd ed.). New York: McGraw-Hill, 1975.

Overall, J. M., & Klett, C. J. Applied multivariateanalysis. New York: McGraw-Hill, 1972.

Rao, C. R. Advanced statistical techniques in bio-metric research. New York: Wiley, 1952.

Rao, C. R. Linear statistical inference and its applica-tions (2nd ed.). New York: Wiley, 1973.

Rulon, P. J., Tiedeman, D. V., Tatsuoka, M. M., &Langmuir, C. R. Multivariate statistics for person-nel classification. New York: Wiley, 1967.

Spector, P. E. What to do with significant multi-variate effects in multivariate analysis of variance.Journal of Applied Psychology, 1977, 62, 158-163.

Tatsuoka, M. M. Discriminant analysis: The study ojgroup differences. Champaign, 111.: Institute forPersonality and Ability Testing, 1970.

Tatsuoka, M. M. Multivariate analysis: Techniquesfor educational and psychological research. NewYork: Wiley, 1971.

Tatsuoka, M. M. Multivariate analysis in educationalresearch. In F. N. Kerlinger (Ed.), Review ofresearch in education. Itasca, 111.: F. E. Peacock,1973.

Tatsuoka, M. M. Review of R. D. Bock's Multi-variate statistical methods in behavioral research.Applied Psychological Measurement, 1977, 1, 457-461.

Timm, N. H. Multivariate analysis with applicationsin education and psychology. Belmont, Calif.:Brooks/Cole, 1975.

Tukey, J. W. The fu ture of data analysis. Annals ofMathematical Statistics, 1962, 33, 1-67.

Tukey, J. W. Analyzing data: Sanctincation or de-tective work? American Psychologist, 1969, 24, 83-91.

Tukey, J. W. Exploratory data analysis. Reading,Mass.: Addison-Wesley, 1977.

Van de Geer, J. P. Introduction to multivariateanalysis for the social sciences. San Francisco,Calif. : Freeman, 1971.

Winer, B. J. Statistical principles in experimental de-sign (2nd ed.). New York: McGraw-Hill, 1971.

Wiley, D. Review of Exploratory data analysis byJ. W. Tukey. Applied Psychological Measurement,1978, 2, 151-155.

Received June 13, 1978 •