DOCUMENT-RESUME

ED 137 374 TM 006 178

AUTHOR-- Dziuban, Charles D.; Shirkey, Edwin C.TITLE Sampling_Adequacy and the Semantic Differential..PUB DATE [Apr 77]NOTE 18p.; Paper presented at the Annual meeting of the

American Educational Research Association (61st, NewYork, New York, April 4-8, 1977)

EDRS PRICE MF-$0.83 Hc-$1.67 Dlus Postage.DESCRIPTORS Correlation; *Factor Analysis;-Interaction; Matrices;

*Psychometrics; *Semantic DifferentialIDENTIFIERS *measure of Sampling Adequacy

ABSTRACTThe Kaiser Measures of Sampling Adequacy (MSA) were

derived for a typical six-concept Semantic Differential.-- The overallindices indicated that both .concept and total correlation-matriceswbUld lead to comparable-decisions regarding the psychometric- gdalityof- the sample data sets. The individual MSAls, however, revealedconsiderable variability for some scaleS placing..several in.a rangewhiCk would make them_suspect in a-psychometric sense. rt wasrecommended that the concepts of psychometric adequacy be-used in-determining the efficacy of one's Semantic Differential data forfactor analytic-procedures. (Author)

.Documents acquired by ERIC include many informal unpublishedmaterials not available-from other sources. ERIC makes every effortto-obtain the'best copy available. NevertheleSs, items-of marginal.

* reproducibility are often encountered and this affects the quality* of the midrofiche and-hardcopY reproductions'ERIC-makes available* ..via the ERIC.Document. Reproduction Service (EDRS)...EDRS is not, ** responsible for theAuality of the-original document,-ReproductionsA,

-* supplied.by- EDRS- are the-best ihai can be .made from the origina

U5 DEPARTPABNT OF HEALT,t',EDUCATION &WELFARENATIONAL INSYTE OF

EDUCATION

THIS DOCUMENT HAS BEEN REPROOUCED EXACTLY AS RECEIVED FROMTHE PERSON OR ORGANIZATION ORIGIN-ATINO IT POINTS OF VIEW OR OPINIONSSTATED 00 NOT NECESSARILY REPRE-SENT OFFICIAL NATIONAL INSTITUTE OFEDUCATION POSITION OR POLICY

SAMPLING ADEQUACY AND THE SEMANTIC DIFFERENTIAL

Cnarles D.. Dziuban

and

.Edwin C. Shirkey

Florida Technological University

Paper Presented at the Annual Meeting of theAmerican Educational .Research Association

New York City

April 4-8, 1977

Abstract

The Kaiser Measures of Sampling Adequacy (MSA) wure derived for a typical

six-co _ept Semantic Differential. The overall indices indicated that both

concept and total correlation matrices would lead to comparable decisions

------ regarding the psychometric quality of the sample data sets. The individual

MSA's., however, revealed considerable vriabi1ity for some scales placing

sev- al in a range which would make them suspect in a psychometric sense. It- -

was recommended that the concepts of psychometric adequacy be used in deter-

ning the efficacy of one Semantic Differential data_ for factor analytic

procedures.

Of the many asse--ment techniques developed in the past decades the

Semantic Di ferehtial (SD) has bean incorporated into an astounding number

f studies. The device presents some attractive alternatives to mo e tra-

ditional instruments in that for one administration the responses yielded

by the SD produce a large amount of data. Customarily the semantic space

is port eyed as three-dimensional and Euclidean in nature. Through effective

selection of scales and/or concepts, the instrument may be tailored to he

.specific situation encountered by the investigator and with proper con-

stri;ction, lmpleted with a rnnimum of tIme and effort. These with ad-

dltionally favorable. consideratIons .have made it poseible to find in almost

every behavioral research journal at lea t some studies for which the SD

bad been the fundamental data- collection device. See for example: Aiken,

1970; Aiken, 1972; Aiken and Dreger, 1961; Anttonen, 1969; Divesta, 1966;--

Dutton, 1956; Husek.and Wtttrock, 1962; Jenkins and SUci 1958; McOallon

and Brown, 1971; Neale, 1969; Neale and Proshek, 1967;.Tanaka, Oyama and

Osgood, 1963; Yamamato, Thomas and Karns, 1969; Yamamato, Thomas, and

Weirsma 1969.

addition it has been subject to a large number of studies regarding

its psychometric pr per_ies. This is best.attested,to by the recent volume

edited by Snider and 0 good (1969) . Many investigators such- as Osgood and.

Sneci (1955) haVe been con- ned with facto ial methods of structuring

Semantic DifferentIal dimensiorsality.

the 1Iteratire of Semantic DIfferential research over a period

Mi on and Osgood (1966) noted that

elve

years indicated that the scale by co cept interaction was often documented

but found little evidLnce for person by scale structure intet ction. Deutsch-

man (1959) reported that n facto- analytic cases studied there vere s lated

three fac - valuation, potene- and activity. UcKie and Foste (1972)

proposed a model in which evaluative scales were determined through factor

analytic methods wLth subsequent determination Of concept clusters achievedthrough the analysis of p_ s n by concept data. Komarita and Bass (1967)

rted.finding threeevalu tive factors while Bashook and Foster. (1973)rep

argued that there was onlycine effective factorin those data. Mos.S (19

found various sets of concepts to alter scale fact pa ns, while Peabody

(1967) Ind cated that the typically found evaluative factor resulted from a

confounding of evaluative and descriptive dimensio

Tanaka and Osgood (1965) idertified a good deal of cross-cultural- pattern*

stab lity in the Semantic Differential. Rosenbaum enhaum, a d McGinn

(1971) indicated that patterns differed cons derably with varyin- cepts.

Recently Maguire (1973) in a comprehensive review-of Semantic Differential

methodology recommended that the method of principal components be abandoned

and replaced with the factor anal7tie-or image model. From these results it

ems clear that factoring-procedures'have played a large part in structuring

the dimensionality of Semantic Differ ntial data.

Recently Kaiser (1970) announced the dev lap ent of a Measure of Sampling

Adequacy (MSA) which Is intended to assess the degree to which a et of

variables under con ideration comprise an adequate sample (psychometric) from

the domain of interest. The index Tay be used as the basis of a decision

rule for determining whether a given sample correlation matrix should be

factored in an explorato y sense (Cerny and Kaiser, In Press).% The fundamental

theory underlying the Meast. (Guttman, 1953) Is that as the sample data con-

form to factor analytic tenets the matrIx of their corclac1ons should have

an inverse R-1 which approaches a diagonal Diag). Kaiser and Rice(1974) using this property defined the MSA as a fun tion ef the anti image

correlation matrix Q 5R-1S where 52 (Diag R-1)- and the observed sample

correlatIon matrix (R)

MSA

EEr2jk 4-.EEci2jkjOk -j0k

The index lies between zero and one (0 < MSA 1) with values increasing with--the psychometric,quali y of the data. The present accepted calibration for

..

.

MSA is as folloWs:

the .90's - marvelousIn the .80's - meritoriousIn the .70's - middlingIn the .60's - mediocreIn the .501s - miserableBelow .50 - unacceptable

A similar measure may be defined forindividual variables.

Er2jkIc

A(j)-

Er2Jk Eq2jkkkOJ . 104j

The i dividual index is intended to assess wh ther a Pa ticular variable

represents the domain of interest and thus should be included in the sampledata s

While holdi _

number of variables incr

number of subjects increases, and (d) the general

s constant, the M A appears to improve as the

h) the (effective) number of fact rs decreases,

evel of correlation

increases. Recent studies with the MSA (Dziuban and Shirkey, 1974A; Dziuban

and Shirkey, 1974; B Shirk y and Dziuban, 1976) have shown that it signals

random variables in a data set and that it would readily guard against data

where the population correlation matrix was an iden y (R Monte car

studies of the index (Cerny and Kaiser, In Press; Meyer, Kaiser, and Cerny,

In P s Dziuban and Shirkey, 1976) have shown it to be most influenced by

the number of variables P.

The purP- e of this s -dy was to assess variations in the saMpling

adequacy of Semantic Differential sca es when used with differing concepts,

In a general sense it -as a study of the psychometric interaction -f scales

and concepts.

P- cedures

A six concept Semantic Differential was administered to a sample of five

hundred fifteen public school teachers in the Central Yloride area. The

concepts Death, Hero, Quicksane, Success, Gentleness and White Rosebuds were

amo g those used by Osgeod, Succi and Tannenbaum (1957) to develop the instru-

meflt. Thirty scales (evalurtion, potency, and activity ) were selected for

use from the thesaurus published by the devel_pers. The order and polarity

ale presenttion was randomly assigned, The order if concept presentation

was varied randomly from a list of twelve possibilities. The scale inter-

correlation matrice were derived for each of the concepts as well as for

strung out or collapsed matrix The overall and individual Measures of

Sampling Adequac (MSA) were computed for each of the seven ma-

:not reported in

Althou

Nper, the resealed image components were computed for

5

each matrix. The number of components retained (q) was equal to the numbereigenvalues of R greater than one and was a quantity of interest to this

study.

Resu

The overall Measures of SamplingAdequacy- (MSA) and number o- components

are presented in Table I.

Insert Table I About Here

It may be observed that, had the MSA criterion been used exclusively as afactorability decision rule, all marrics would have been evaluated as "appro-priate." The highest overall

Death (MSA .85).

MSA (.98) was obtained for Hero and the lowest

worthy of'note that the total sample did not pro-duce the largest value (MSA .88) although the effective number of subjectsappeared to be 3090. Further it may be observ d that tliere was substantialvariability in the number of retained component8. The largeSt number (eightwas obtained for Quicksand and tho lowest (three) orHero while the totalsample produced six components-.

--The individual MSA's are presented in Table II.

Insert Table II About Here

were to view as suspect variables with values below- 0, a series

psychometric interactions may be observed. A swum y of those scales-presented in Table III.

Insert Table III About Here

For at least one concept eight ( 07.) of the original thirty scales

produced an MSA which would be considered very suspect. Five of those wererelated to the originally defined activity factors with one each for potencyand evalua ion. None of the scales, however, ichin the context of the

trung out matrix producedan individual. MSA which normally would Mandate

further scrutiny. Of the six concepts, Death and White Rosebuds produced4the largest number of unacceptable values, four and five respectively, while

Quicksand produced three. Success and Gentleness produced one low value eachwhile Hero was the only concept,which failed to yield any clearly unacceptable

MSA levels.

Disc u ion

Through the use of some kind of factoring procedure, it has been and stillis common practice to assess the underlying dimensionality Semantic Dif-ferential data. Much of this work has been explo atory in nature, although

often the intent has been to retrIeve the evaluation, potency, and activitydimensions. Because of its three-dimensional nature exploratory factor

analysis of Semantic Differential data has p esented complex analysis pro-blems. If one wishes to analyze a two-dimen ional array of scales, however,two primary options arc available. The first, which is rarely done,

analyze the scales for each concept separately and subsequently compare thesimilarity of the factors The second method involves deriving the factors

the strung out matrix by collapsing the concepts and forcing non-

7

independent observations. We have assessed the overall psychometric adequacy

of the one data set in both situations and found it to be acceptable.

addition, individual indices for the strung out matrix showed all.of the

variables to be at 1 ast minimally adequate. From this one _ight assume

that the- could proceed with the proposed factoring procedures.

The results of the 'co c pt factoring, however, indicated that certain

precautions should be taken in theSe,circumstances. The change in psycho-

metric adequacy of some scales, when used

a change i

h different concepts, may indicate

domain relatedness. This becomes evident'whea one "dissect

__e Semantic Differential. When the Individual MSA's for-the-everall.matrix

were computed the scale pair Weak-Str__g produced a value (.79) which would

by present calibration standards 1 one _o decide that had an acceptable_

domain relatedness. tt did, however, produce decidedly inferior values for

o of the concepts (.46, .59). fect of collapsing the concepts was

obscure the infe ior MSA's. Since this Oco-U- d with eight scales,

seems_ wQrthy of consideration.

Tae- results of this study suggest that,13

ca ion of a common decisith

.pending on the -oncept, appli-

1) to Semantic Differential d 'a will

produce largely disparate numbers of components. With as many as eight and

as few as three dimensions found in the

in the

same decision rule-wL have over or under factored each one of,the concepts.

The data from the Semantle Differential presentS a natural three-dimensi64

overall matrix suggested "averag

separate concepts the total of six

' components. Apparently hrOugh the

data box. Analyzing the components separa -ly for each concept tends to

ignore the basic structu the inst u en_ as does-collapsing the'data over

concepts. An .ideal analysis shohld in olve some form of a three-mode pro-

cedure such as the one proposed by Tucker (1972).

Should one, however, choose a conventional analysis_ strategy, the

overall_and individual Measures of Sampling Adequacy will be useful in reaching

appropriate decisions regarding the quality of one's data set. The.overall

measures provide important information as to whether the data should be

factored at all. The individual measures might--proVide information as to

which variables should:be included in.subsequent data collection and analysis_

strategies.- A variable which is.universally poor is probably a good candidate

for deletion. On the other hand, scales with erratic sampling adequacy

characteristics may be well suited for some concepts and not for others. This

scale by concept interaction has 1 ng been recognized so we re-emphasize that

scale selection is a c itidal step in the assembly of a part cular Semen ic

Differential. The individual MSA's might be further helpful in this context

since with the SD one is factoring single scale- with their pr -u ed low

reliability instead of batteries of tests. Accordingly, we feel the concept

of- psychomet 'c sampling adequacy should provide a helpful guide to the analysis

f Semant c Differential and rec5mmend that one scrutinize his/her data by

exa ining the associated overall and individual MSA's.

ii

9

References

Aiken, L. Attitudes Toward Mathematics. Review of Educa ional Re earth,1970, 40, 551-596.

Aiken, L. Research on Attitudes Toward Mathematics. Arithmetic Teacher,March, 1972, 229-234.

Aiken, L. & Dreger, R. The Affect of Attitude on Performance in Mathematics.12L3-1121.1_,:_hlaj_19iipj_Kisis.LIaloio 1961, 52, 19-24.

Anttonen, R. A Longitudinal Study in Mathematics Attitude.Edueational Research, 1969, 62, 467-471.

Journal c

Bashook, P. G., & Foster, S. F. How Many E's Are There? 7 A Critical Analysisof Problems Concerning Determination of Evaluative Factors of SemanticDifferential Scales. A paper presented at the Annual Meeting of theAmerican Educational Research Association, New Orleans, February 25-March 1, 1973.

Cerny, B. A., & Kaiser, H. A. A-Study of a Measure of Samplin- Adequacy forFactor-Analytic Correlation Matrices. Multivariate Beha- oral Research,(In Press).

Deutschman, P. The Semantic Differential Its Uses and Abuses. PublicO2Inion Quarterly, 1959,

Divesta, F. The Test Retest Reliability of Children's'Ratings on the SemanticDifferential.klu_s_a_tionalandPschoMpasurement, 1966, 109,205-229.

Dutton, W. Attitudes.of Ji-inior High School Pupils Toward ArithmeResear- January, 1956, 18-22.

School

Dziuban, C. D., & Shirkey, E. C. On the Psychometric Assessment of CorrelationMatriceS. American Educational Research Journal, I974A, 11, 211-216.

D.ziuba- C. D., & Shirkey, E.C. When is n,Correlation-Matrix Appropriate forFactor.Analysis? -Some Decisigo Roles. Lyslis!lpiaLaLL11s1n., 1974B-6, 358361-,

Dziuban, C. D. & Shirkey, E.G. An Investigation of Some DistributionalCharacteristics of the Measure of Sampling Adequaey. Paper Presentedat the Annual "-eeting of the American Educational Research Association,San Francisco, April, 1976.

Cutcman, L. Image TheorY-for the Structure of Quantitative Variates.EELftallElslica, 1953 18,-277-296.

-The Dimensions-of-Attitudes Toward Teachers-as.'"keasuredj)y.the Semantic-Differential." Journal of Educational Psychol_1962,:53, 209-213. -

enkins,: S., & Sucil C. An Atlasi of Semantic P-journal -f Psycholoy, 1958, 71, 688499.i

10

for 360 Words. Anierican

Kaise_, . F. A Second Generation Little Jiffy. Psychometrika, 1970,401-416.

. Raise _Rice,. . J. Little Jiffy, Mark IV. Educational and PsYchoIog caMeasurement, 1974, 34, 114--1 7. --

S. &:Bass. A. R. Attitude_Diffe entiation and Ev luativof the SeMintic Differential. Journal of Personality and cial-Psycho ogy,--1967,.-6,-241-244.

Maguire, T. Semantic Differential MethodologyAmerican Educational Research Journal, 1973 0, 295-306.

McCallon, E., & Brown, J. A Semantic Differential Instrument for MeasuringAttitudes Toward Mathematics. The Journal of Experimental Education,- Summer, 1971, 69-72.

McKie, D. & Foster, S. F. A General Model for Multidimensional Analysis ofSemantic Differential Attitude Data. Proceedings of the Annual,AmeticanPsychological Association Convention, Honolulu, 1972, 45-46.

the Structuring of Attitudes

Meyer, E. P., Kaiser F., & Cerny, B. A. MSA for a Special Spearman Matrix.y_isl-1212-ika, (In Press).

M. S., &.0sgoodThe Multivariate:Structure:of Qualificat ona.In R. B,..Cattell Ed.), Handbook of Multivariate Experimental Psy-cholo y.. Chicago: Rand:McNally, 1966..

Mos C, Cur eet and ProjeCted Status of Semantic Differential Research.Psychological Renord, 1960, 10, 47-54.

The Role of AttitudesDecember, 1969 16.:

arning Mathematics. Arithme ic Teacher,

-Nealei- D & Proshek, j.. Sahool.Related _Attitudes of:Culturally: Disadvanta e-:EleMentary School- Children. Journal of'Edu ational. Psycholo y, 1967,

Osgood, C. E.,- & Sudoi J. .FaCtor Analysis of Meaning.Experimental Psyc Ology, 1955, 50, 325-328.:

Osgood, C. B., Succi,Urbana: University of Illinois Press, 1957.

ourna_

& Tannenbaum, P. H. The MeastireMent of Meanin

Peabody,.D.:_Trait Inferences:. EvalUative.and.Deseriptive AsPects °ureaof. PerSonality and Social P-sye1221agL, 1967, T (Monogr. -.Suppl. 4 Whole-

R6Sehbaum, Rosenbaum, W. R., & McGinnles, E. Semantic DifferentialFactor Struc ure:Stability Across Subject, Concept, and Time Differences,Multivariate Behavioral Research 1971, 451-469.

Shirkey, E. Q., & Dziuban-, C. D. A Note on Some SaMpling-Characteristicsofthe Measure of Sampling Adequacy MSA). MultiVariate BehavioralResearch 1976, 11, 1257128.

Snider, J. R..&. Osgood,- C. E. (Eds-0... The _SA Book of Readil s Chicago: Aldine 13 blishing Company,.1969.

Tanakai-Y.,& Osgood, .C. E. Cross-CultUre, Cross Concept, and .Cross SubjectCenera1ity.-of Affective 'Meaning Systems, -Journal-of PersonatclSosillmlipjuLy, 1965, 2(2), 1437153.

Tanaka', Y., Oyama, T., 4 osgood, C. A.Cross-Culture and Cross Concept Study.

. ofthe Cenerality-of Semantic Spaces, Journal o' Verbal Learnin- and--Verbal Behavior. 1963,-2, 392405.

Tucker, C. R. Some Mathematical Notes on Three- ode Factortetrika, 1972, 37 2728.

alysis

_Yamatheto, K., Thomas,. E., & Karns, E. School Related Attitudes in Middle7School Age Students. ,American Educational ResearCh -ournal, March1969,:191-206. :

Yamamoto, K., Thomas,'E., and' Weirsma, S. Dimensions_of School Related :

_Attitudea:im-Middle-SchoOl Age Students. Ls_--cl-o-yn_S_WQhcLI31s,1969,:10 375-378.-

Table I

Overall MSA's and Number-of Components

..Concept- MSA.- of Components

Quicksand = 8

Death .85

White Rosebuds .92

Hero 95

Gentleness .92 .

Succesa

Total .88

..Passive/Mtive..

Soft/Hard.

Complete/Incomp1ete

Excitable/CalM

.Good/Bad

.Weak/Strong,

Awkward/Graceful

Fast/gow

Tenacious/Yielding

.Free/coastraine(1

,Pleasurable/Painful

Eaergetic/Inert

Simpe/Complex.

.Serious/light

Cold/Hot

Thick/Thin..

-.Beautiful/Ugly--

,Severe/Leaient -

Easy/Difficult .-

large/SmalL

Eme..tional/Unemotional

ligat/Heavy

Meaningful/M4aningless

Masca1ine/Feminine:

_Unimporrant/Import.ant

?ositive/Nega4yq .

.Decimals..00.tted

keerliaed

Table II

Individual Measures of Sampling Adequacy

White

Rosebuds Hero Gentleness Success Quicksand Death

Total

Matrix59 94 67 90 86 68 7891 93 94 91 86 87 7593 95 96 94 79 8 9279 96 85 94' 92 68 8993 96 91 95 91 90 8946 92 88 92 85 59 7992 96

93 92 84 83 8889 97 91 64 50 65 8091 94 89 91 90 71 9096 96 96 72 92 87 9392 95, 94 95 .90 91 9167 97 8896 68 87 8991 93 87 93 71 78- 8091 96 80 95 94 87 8846 93 89 83 87 83 77,89 92 81 92 93 75 9194 96 94 94 92 86 9192 81 87 90 93 93 9195 90 94 91 90 86 9090 96 88 95 92 81 9190 95 95 94 88 85 8491 94 92 94 81 FT' 9093 90 94 93 89 89 .8093 95 90 85 83 89 8492 95 94 93 75 90 8763 97 89

'95 89 88 9094 97 92 95 62 75 919696 84 86

86 76 6995 93 92 9273 72 8196 95 94 95 19

94

18

_Table III

Summary of the Scalesor 1hjch Deficien MSA's Were Found

ite

Rosebuds Hero Gentleness Success Quicksand De th

,Passive/Active 59 67 68Excitable/Calm68Weak/Strong 4659Fast/Slow

64 50 65Energetic/Ine t 6768Cold/Hot 46

_oving/Still 63ieeningful/Mee 'ngless

62_

8