~wrrent@omm*nts@EUGENE GARFIELD

INSTITUTE FOR SCIENTIFIC lNFORMATION~3501 MARKET ST., PHILADELPHIA, PA 19104

Prestige Versus Impact: EstablishedImages of Journals, like Institutions,

Are Resistant to Change

Number 38

How closely does stature (reputation) cor-respond to objective measures of actual per-formance? In the following article, reprint-ed from Social Science Quarterly, 1 JamesA. Christenson, Department of Sociology,University of Kentucky, Lexington, and LeeSigelman, now dean. Faculty of Social andBehavioral Sciences, University of Arizona,Tucson, compare impact-factor data fromthe Social Sciences Citm”on Indexm (SSCF )with prestige rankings of journals insociology and political science. These jour-nal hierarchies were obtained in previousstudies in which sociologists and politicalscientists were asked to rank journals on thebasis of their importance to their respectivefields. Christenson and Sigelrnan, compar-ing these rankings with SSCI data, noted anonlinear relationship between reputationand impact, particularly at the low and highends of the prestige ranking, or “peckingorder. ” In other words, the authors conclud-ed that some journals have a reputation thatis disproportionate to their impact, whileother, newer joumrds are presumably notrewarded with the reputation that their im-pact warrants.

The reprinted paper is reminiscent ofother studies that have attempted. throughboth subjective and unobtrusive “objective”means, to establish or examine hierarchiesbased on prestige. For example, in 1970Kemeth D. Roose and Charles J. Andersen,American Council on Education, Washing-ton, DC, published their report based on asurvey of some 6,000 scholars. They mea-sured the quality and effectiveness of grad-

September 21, 1987

uate programs in more than 30 disciplinesat over 100 American institutions.z A 1971study by Warren O. Hagstrom, Universityof Wisconsin, Madison, compared depart-mental prestige (based on a 1966 AmericanCouncil on Education study that rated thequality of graduate faculty) with such mea-sures as department size, research produc-tion, and faculty awards in 125 departmentsof mathematics, physics, chemistry, and bi-ology. 3 Other similar studies include a1977 paper by Julie A. Virgo, vice presi-dent, Carroll Group, Chicago, Illinois, thatcompared the citation frequency of scientif-ic papers with their judged ‘‘importance,”4and a 1982 assessment of US research-doc-torate programs edited by Lyle V. Jones,University of North Carolina, Chapel Hill,and colleagues. s

Prestige, in the words of Thomas Vocinoand Robert H. Elliott, Auburn University,Montgomery, Alabama, is’ ‘an abstract psy-chological concept” that is ‘‘multidimen-sional. ”They discussed two aspects of jour-nal prestige: “intensity,” which they gaugedby asking respondents to assign weights tojournals using a single journal as a referencepoint (Public Administration Review, in thiscase); and’ ‘extensity,” which refers to howwidely a journal is known within aprofession.b

In a 1978 paper, Thomas Roche, Penn-sylvania State University, University Park,and David L. Smith, Virginia PolytechnicInstitute and State University, Blacksburg,used the SSCf to establish methods of rank-ing sociology journals and departments on

263

the basis of citation data.T The authorscompared their citation-based ranking withvarious other quantitative rankings. TheRoche and Smith paper, like the study re-printed here, compared a subjective rank-ing of political-science and sociology jour-nals, obtained through a questionnaire, withobjective SSCI data on the impact of thesejournals.

As I’ve noted previously, a frequent, ifunwarranted, criticism of citation indexingis that raw citation counts do not take intoaccount the standing or prestige of the jour-nal in which the cited work appears. s How-ever, that leaves open the question of whatdetermines the meaning of prestige. TheChristenson-Sigelman study suggests thatthe reputation of a journal is not always jus-tified. Its actual, current intluence may bemore accurately demonstrated through suchmeasures as citation impact. As the authorsconclude, citation impact may provide amore meaningful indication of a journal’simportance than the number of articles it haspublished or a total reliance on what maybean obsolete reputation, On the other hand,we must guard against simply equating cita-tion impact with reputational standing. As

David L. Tan, Pahner College of Chiroprac-tic, Davenport, Iowa, noted in a review ofthese quality-assessment studies, “No oneis certain what reputationrd studies are mea-suring—reputation or quality?”9

In a recent critique of the study reprintedhere, Gillian Stevens, University of Illinois,Urbana-Champaign, points out flaws in partof the Christenson-Sigelman methodology,specifically in their use of residuals. Thesemeasures, Stevens notes, do not complete-ly support the conclusion that’ ‘journal reputations survive past the time that they arewarranted. ” 10 In a reply, the authors,while defending their methods and conclu-sions, acknowledge Stevens’s criticisms andexpress agreement that “better evidence isneeded to document the assertion that jour-nals rated high on the prestige scale are re-ceiving too much credit and those on the lowend tw Iitt]e.” 1I However, they ~so Point

to the larger issues discussed in their paper.These include the extent to which common-ly accepted prestige scales reflect the utiliza-

tion of scholarly works as documented in theSSCI. Such issues, according to the authors,are valid and worthy of further study.

c ,%?,s,

REFERENCES

1. CfrrSaterraorr J A & SSgelrnmr L. Accrediting knowkdge: journal srature and citation impact in socialacierrce. .%c. Sci. @rrn. 66:964-75, 1985.

2. Rowe K D & Arrderaerr C J. A raring of graduote programs,Waahirrgton, DC: American Cormcif on Education, 1970, 111 p.

3. Hagatrorrr W O. Inputs, outputs, and the prestige of university science departments.SOciol. E2Juc. 44:375-97, 1971.

4. Virgo J A. A statistical procedure for evaluating rk importance of scientific papers.Libr. Quart 47:415-30, 1977.

5. Jones L V, Lhrdaey G & CoggeahaSl P E, eds. An assessment of research-docrorore program in theO’rri/edSrafes. Washington,DC: NationalAcademy Press, 1982.5 VOIS.

6. Voctno T & Elffott R H. Journalprestige in public adminisrcation. A&nin. SW. 14:5-14, 1982.7. Roctte T & Srrrfth D L. Frequency of citations as criterion for tk ranking of departments, journals, and

individuals. Sociol. Inq. 48:49-57, 1978.8. Garfteld E. Citation indexing-its theory and application in science, technology, and hronarrities.

New York WiSey, 1979. p. 247.9. Tm D L. The assessment of quality in higher education: a critical review of the litcramre and research.

Res. High. Educ. 24:223-65, 1986.10. Stevens G. A note on journal stature and citation inrpact. Sot, Sci. Quart. 68(2):418-9, 1987.11. CfrrMenaon J A & Stgehurn L. Journal stature and citation impact: a rejoinder,

Sot. Sci. Quart. 68(2):419-20, 1987.

264

Reprinted from .%aal Science Quarterly, Vol, 66, No, 4, tkemrber 1985, Copyright 19S5,by permission of rhe Wriverairyof TexasPress.

ACCREDITING KNOWLEDGE: JOURNAL STATURE AND CITAtiONIMPACT IN SOCIAL SCIENCE1James A. CHRISTENSON, University of Kentucky

Lee SIGELMAN, University of Kenruc~

The impact of the nerwork of journals through which acholara disseminate and accredit their ideas is comparedto the presrige hierarchy of journals in aoeiology arrd politieat science, A comparison of prestige rankings of jour-nals with Socia/ Sciences Citation Aro’e.@ impact scores suggests a nonlinear rdationship: many reputed “top”journals receive inordiire credt and many new and less prestigious joumrds reeeive Iesa credk than their impactwarranta.

The institutional goal of science is the exten-sion of accre&xi knowledge (Merron, 1942). Inorder to achieve this goal, scholars need to com-municate with one another through regular, openchannela. This exchange of ideas is thus an ines-capable aspect of aeientificresearch and develop-ment. The question for scholars is how and whereto disseminate and thus to accredit their work.

Aecredhed knowledge is grounded in collegialrecognition of the indlviduaf and her/his work.Not all ideas win equaJ acceptance, and neitherdo all the scholam who generate these ideaa orall the institutions that house these aeholars. Forthla reason, mnsiderabie research has been doneon the stratification of Bcholarlyfieldg-researchdesigned to pinpoint the beat, or at least the mostreputable, aeholarsand programs in various fields(e.g., Allison and Stewart, 1974; Bingham andVertz, 1983; Cole, 1983; Cole and Cole, 1973;Crone, 1%5; Long, 1978;Merton, 1968;Reskirt,1977). However, the network of uunrmmicationsundergirding driaatratifrcationsystem remains lit-tle understood (Garfield, 1972). IdeaJly, goodideas, insights, theories, and findhga wouldachieve the impact they deserve on the basis oftheir merits; but, in accrediting knowledge, themedium of dissemination maybe as important asthe message.

Journals, along with booka, are the prime me-dium for accrediting knowledge. Cole (1983) ob-aerwti that’ ‘we read papers itt jotsrttaiaofdyafterthey havebeersevaiuated [accredited] by others.We givepeoplemore credit for publishingin pres-tigious joumrds” @ 137). But what do we meanby “accredited’? And to what extent is prestigeindependent of quaiity or impact? A paper pub-lished in a refereed joumai haa met the standardsof that journal. As Glem (1971:298) has noted,it is widely recognized that there are status dif-ferentials amongtilourrrrds in any field. To pub-Iiah a paper in eertr& journals maybe a Idghlyvisible badge of snccea$. If a paper appears in a“top” journal, the presumption is that it must begood. Journals, beeause they are refereed, pro-vide accreditation.But some referesdjoumrdspr-ovidemuch more accreditation than others.

A different form of accreditation is providedby one’s peers when they make use of one’s work.Seen from this perspective, good work is workthat others find useful and eorrsequently cite intheir own work. Hargena and Fehrdee (1984)summarized their literature review by assertingthat “the number of citations to a scientist’s workis ofien recommendedas the best sin81eindicator

of scholarly recognition” (p. 686). so the accred-itation of one’s work can be measured in at leasttwo ways: the prestige of the journal in which itis published and the frequency with which it iscited. Of course, work can be widely cited pre-ciselybecauseof where it was published,but thesetwo aspects of accreditation are at least e4meep-tually dlatinet. Our researeh questioneoneems theextent to which they are empirically distinct. Howdoes the latter form of accrexfhation(citation im-pact) relate to the former (journal prestige)?

Questions concerning accredited knowledgehave both theoretical and practical implications.A journal can achieve the status of a “top” pub-lication outlet for reasons unrelated to the quaJi-ty or impact of the articles it publishes—reasonsthat include, but are by no means restricted to,its sponsorship, age, the quantitative/qualitative,theoreticallempirieal, and professionrd/practi-tioner orientations of its articles, the visibility ofits edhor and editorial boards, and its past reputa-tion. It seemslikdy thatjournals, fike departmentsand universities,establishimagesthat are relarive-Iy resistant to change. Thus, jourtrsdX, a long-es-tablished, discipline-supportedjournal, may out-rank new journal Y in lerms of prestige eventhough Y is publishing more important articlesthan X is in terms of citation impact.

Martyprofessionalsare interestedin the accred-itationof knowledgefor praetieal reasons. For ex-ample, the interests of librarians and informationsystem designers stem from their aaaumptionthatthe quality of a joumai affects user demand forthe journal. Seierreeplatmergfind journal ratingshelpfut in aaacssingthe payoffsof variousresearchprogrrunsand the productivityof variousresearch-ers and research teams. Journal editors and spon-sors use ratings as performance indicators and

1Audrmr are Iisvd atphabsricatly.Each has made an equal contribution. Thk researchwaapartially fucdedby the KentuckyAsri-cukural Experiment Ration.

265

TABLE 1Prestige, Impact, and Related Measuresfor SaciologyJourna!s

SSCPImpact

Glenn Scord prestigePrestige (Average, Residual

Joursud scorrY 19?7-79) SeOr~

Amen”con Sociological ReviewArneticm Jourrrd of Sociology

Social ForcesSocial Psychology Qrsorierly (formerly Sociorrr.wy)British Jourrd of SociologyAmerican AnthropologistSocial ProblemsAmen”can Political Scienre ReviewDerrrographyAnnals of the Amen”can Aca&?my of Political ond Social SciencePublic Opinion QuarterlyAmen”con Ecoaorrric ReviewJourrrol of Personality orrd Social PsychologyEuropem Jourrrol of SociologyBehavioral ScienceRural SociologyHurnon OrganizationJournal of Social Ps~ho/ogyAa%rinistrotive Science QuorterlyMilkmk Memonid Fund QuorterlyIntenrationol Jourrrol of Comparative SociologyAmerican Behavioral ScientistJournol of Social IssuesSocial ResearchDaedalraHumon Relorion.rPoprdorion StudiesHarvard l?ducationol ReviewCurrent SociologyConodion Review of Soriology and Anthropology.$ociologid ReviewIrrtenrational Social Science JourrrolAmen”con SociologistJourrrol of Marriage aad FornilyJournol of Conflict ResolutionJounrol of Health ond Social ResearchSociology of Education

10.09.68.17,87.87.77.67.57.47.27.17,17.16.96.86.76.76.76.76.76.76.66.66.66.56.56.56.46.46.46.36.36.26.26.26.26.1

3.3672.0340.9710.9440.5351.8151.0411.9731.1330.4250.8511.5522.39U0.4350.5870.7980.4360.2832.2931.1920.1710.4831.0310.3950.9580.5191.0172.8160.0950.2330.2440.2300.7400.9880.6381.6020.403

1.6622.3101.&t5i. 3671.6880.5821.0900.2580.8181.1740.7400.189

-0,4700.8670.6470,3810,6660.786

4.7940.0720.8740.5290.0980.5980.0560,4010.009

-1.5050.6340.5250.4170.428

-0.073-0.2680.007

-0.7510.092

planning guides. Researchers themselves want to United .%tes., and Giles and Wright (1975)ur!&r-pursue a sensiblemanuscriptsubmissionstrategy, took a similar survey of political scientists. Glennwhfie department chairs and deans are faced with askedhis respondentstojudge 63journrdsin termsthe need to document the quality of facuiry pub- of’ ‘the average im~mmce of their contributionslications in conjunction with tenure and promo- to the field” of soaology, instructing them to usetion decisions, departmental reviews, and the fike the American Sociological Review as an anchor(Gordon, 1982). for their evaluations. The American sociological

Review was given an arbitrary score of 10, andrespondents were told to assign a score of 5 to

The Lhsfs between Reputation and a journaI they considered onfy half as implantPerformance as Ihe Anrcricon Soci&@al Review, 20 to a jour-

nal they considered twice as important as theIn the fields of sociology and political science American Soeiofogieaf Review, and so on. Giles

fairiy clear-cut journal prestige hierarchies have and Wright’s respondents. who rated 63 joumafsbeen documented. Giem (1971) soiicited evrdua- commonfy used by political scientists, alsotions of professional joumais from a sampie of employeda lo-point rating system, but their scalesociologists at Ph.D.-granting programs in the was marked by verbal descriptors (O = pmr,

266

TABLE l--continuedPrestige, Impact, and Related Meaaureafor SociologyJoumata

Glennprestige

Journal SrOr@

SSCPImpactScoreb Prestige

(Average, Residual1977-79) Ssor@

Sociological QuarterlyAc?a .$ociologicaSocial Science QuarterlySouthwestern Journal ojAnrhropologySociology and Social ResearchSociologySoci0108ical InquirySociety (formerly Transaction)Sociological Perspectives (formerly Pacific Socio[08ical Review)Lrw and Society ReviewSociological AnalysisJournal of Grrontdogy

Journal of Research in Crime and DelinquencyAmcricarr Journal of Economics and SaciologyBritish .Iourmd of CriminologyGerontologistCrime and DelinquencyScience and SocietyJournal of Crime I.aw, Criminology, and Police SciencePhylon.$ocia! tfio[08yJewish Journal of SociologyAmerican Journal of CorrectionEugenics ReviewJournal of Negro Eakotion

New !joriety

Federal Probation

6.16.16.06.05,95.95.85.75.75.75,75.45.45.35.35,35.25.25.15.05.04.94.84.74,54,53,8

0.221 0.2350.174 0.2720.479 -0.068NA’4 NA

0.103 0.1270,694 -0.3370,187 -0.0390.198 -0.1470.222 -0.1661.760 -1.3750.197 -0.1461.316 -1.3260.735 -0.8690.237 -0.5780.394 -0,7010.877 -1.0810.831 -1.1450.309 -0.7341.921 -2,1010.098 -0.7690.571 -1.1400.288 -1.018

NA NANA NA

0.076 -1,2510,065 -1,2430.326 -2.148

Wource: Glenn (197 I ).

b!%urce: Social Sciences Citation hake Annual, VOIS. 1-3.

CThis is the actual value of the Glenn prestige score, less the prestige score predicted from the regression ofprestige scores on impact factor scores.

‘NA: Not available,

2 = fair, 4 = adeqyrde, 6 = good, 8 = very of work” (Cole, 1983:116). Number of citationsgood, and 10 = outstanding) rather than being is also highly correlated with other measures ofanchored by a prominent journal. quality that miologisfa of sciencehave employed

Sociologists’ and politiwd scientists’ ratings of (e.g., access to resources, status of degree-grant-their professional journafs, as determined by the ing institutions, initiaf appointments, mobility).Glenn and GJea-Wrightsurveys,are summuimd However,qualityin thk contextis defined as in-

in the first cohunn of T3&s 1 and 2, respective- tellecrrralinfluence-the impact of one’s ideas asly. Two of the top five ~ourrttdson the political aeeredked by others through use in their own

scientists’list(the American SociologicalReview work. Citationa are a measure of qurdity, in thatand the American Journal of Sociology) were the they suggest that other professionrds working intop-rated sociology journals. Sociologists, for the aarne area have found one’s ideas valuable.their part, also gave high marks to the principal The Sociai Sciences Citation ktdexa (SSCP )journals of their sister dkciplines, ranking the provides “impact factor” scores for more thanAmerican Anthropologist sixth and the American 1,300 social science joumafs. Journals from thePdiricd Science Review eighth. diaeiplin~ of psychologyfolfowedby psychiatry,

How closely are these reputational ratings re- econornica,and law generaflyhave higher impactlated to the actual influence or quality of these scores. Sociologyjournals rank about loth, withjoumafs? “Extensive past research indicates that political science joumsfs about 25th. Such differ-citationsare a valid indicatorof the relativequality ences among disciplines reflect, among other

267

TABLE2Prestige, Isnpaet, and Related Measuresfor Polltkd Bcience Jourmds

SSCPImpsct

Giles-Wrigtst 8eoreb PrestigePrestige (Average, Residual

Journsl score 1977-79) ScOr.#

World PoliticsAmerican Sociological ReviewAIOCriConJouml of [nternutiond .!.awAmerican Journal of SociologyAntckm Pofilicfd Science ReviewJournal of PoliticsComparative PoliticsAmerican Journal of Political ScienceAdministrative Science QuarterlyPublic Opinion QuarterlyDaedolusJournal of Public LawPublic Aa?ninistrotion ReviewBritish Jounud of Po/itica( SciencePublic InterestPolitical 21e0QLaw and society Review[ntewmtional OrganikxionSocial ForcesPolitical Studies

Social Science QuurterlySage Professional PapersGOvemment and OppositionPolitics and SocietyBehavioral SciencePublic ChoicePublic PolicyPolityCanadian Journal of Polirica[ ScienceJournal of Conflict ResolutionIntem”onal AffairsComparative Political StudiesUrban Affairs QaarrerlyForeign AffairsWestern Political QuorterlyAahrinistration and SocieiyA&rinistrative law Review

7.3 0.9707.1 3.3677.0 1.3237.0 2.0347.0 I .9736.7 0.3786.6 0.7086.6 1.0276.5 2.2936.5 0.8516,4 0,9586.4 NAd

6.3 0.1956.2 0.7086,2 2.0936.2 0,2676.2 1.7606.2 0.%16.1 0.9716.0 0.3486.0 0.4796.0 NA6,0 0.3576.0 0.4126.0 0.5876.0 0.3746.0 0.76455,9 0.1755,9 0.4655.9 0.6385.8 0.8145.8 0.5235.8 0.5445.8 2.0505.8 o.3m5.8 0.3285.8 1.235

things, the relative size and professional diversi-ty of the disciplines.

The earliest journal impact scores SSCI pub-lishedare for 1977and are baaedon citationsfromarticlespublishedduring 1975-76.A journal’s im-pact factor wore for 1977is definedas the numberof citations during 1977 to articles that the jour-nal publishtxlduring 1975-76,dividedby the totalnumber of articles the joumaf published during1975and 1976(i.e., the ratio of citations to “cit-able” items for a given journal). Dividing thenumberof citationsby the number of citable itemscontrols for the journal’s size and the frequencywith which it is published. Gordon (1982) foundthat impact scores were highfy correlated overtime. For example, the correlation of impactscores between 1977and 1978 for 59 of Glenn’s

1.282-0.3220.7750.3590.3941.0280.7350.548

-0.2930,5510.489NA

0.7350.335

-0.4760.593

-0.2810.1870.0810.3460.269NA

0.3410.3080.2060.3310.1010.3470.1770.076

-0.1270.0430.03 i

-0.8510.1740.158

4,374

journals was .84, To mitigate the possibilities ofyearly fluctuations,a three-year(1977-79)averageis calculated in thk research for each joumaf. Ifjournal prestige influences submissiondecisions,the prestige ratings published in the early to mid1970s would influence publications in the mid1970sand citation counts in the latter 1970s, thetime of our assessment.

What is the relationship between citation im-pact andjoumrd reputation?The Glesmand Giles-Wright reputationalratingsare related to the SSC1impact factor scores (which are shown in the sec-ond column of Tables 1 and 2): the correlationbetwwenthe Glem (scwiology)and SSC1measuresis .526, artd the correlation between the Giles-Wright (political science) and SSCJ mcaaures is.572. This suggests that reputations are perfor-

TABLE 2-contbmedPrestige, Impact, and Related Measures for Politicaf Science Journals

Journal

American Politics Quorterly[ntew’onal Studies QaanerIyPubliu.rAsian SuweyPolitical .UedtodOlOgyPolirical ScienceDissentAmerican Behavioral ScientistPolirical Science Qr4uneriyPolitical QuarterlyJournal of Peace Researchhsterrratioaal Social Science Jouraal.lourrral of Irrterrratioard AfairsSiraalation and GamesAnrrtds of dre American Academy Of PO[itiCa[ ad .$OCia[ .!CienCe

Review of PoliticsInterrratiorrol InteractionsJournal of Developing. AreasExperimental Studies oj PoliticsPolicy Stuales JournalOrbisPsMidwest Review of Pablic AahrinisxratimrNatiomdCivicReviewJournal of Inter-Amen’can Studies and World AffairsSocial Science Journal

Gil&WrightPreatJgeScore

5.85.75,75.75.6S.65.65,65.6S.65.65.45.45.35.35.35,05.04.94.84.84,74.24.14. I3.8

SSCPImpactScore prestige

(Average, Residual1977-79) Scorr+

0,597 O.llxt0.581 -0.09[0.172 0.1490.446 -0.012

NA0.388 4N:780.205 0.0300.483 -0.1330.504 -0.1460.134 0.0710.557 -0.1770.230 -0.1850.312 -0.2330.268 -0.3070.425 -0.3990.217 -0.277

NA NA0.146 -0.536

NA NA0.106 -0.7120.457 -0.9180,518 -1.054

NA NANA NA

0.272 -1.5100,196 -1.765

%ource: Giles and Wright (1975).

bSource: Social Sciences Citation Irrdexa &suad, VOIS. I-3.

C’tlsis is the actual value of the Giles-Wright prestige score, less the prestige score predicted from the regres-sion of prestige scores on impact factor tiom-s. -

‘NA: Not available.

martce-baaedto some degree, for the journals thatare perceived as most prestigious in each disci-pline tend to be the ones that have the greatestscholarly impact. On the other hand, these cor-relation are not rteariy strongenoughto permitus to concludethatajottrnsd’sreputationis a sim-ple fonctionof schokarlyitdluence. Approximatelytwo-thirds of the variance in the reputed qualityof political sciencejournals and three-quarters ofthe variance in the repined importance of aociol-OSYjournals remain unexplainedby the SSCI im-pact scores,

The unexplainexivariance in journals’ reputa-tions might simply reflect the operation of ran-dom error in the reputatiottal measures. More-over, there is a lag of several years between thereputatiomd meaaures attd the impact measure.But we doubt that either random measurementer-ror w 4 time lag tells the whole story. Rather,we think it quite likelythat scholarlyjourrttds, lieacademic departments, tend to establish reputa-

tions that endure in spite of what they merit. Oncea journal has been placedon a diwiplii’s prestigeIadder, it tends to retain its place because ifa rep-utation is accepted at face value and is not con-tinuously reevaluated in light of changingcircumstances.

We certainly do not claim to possess deftitiveproof of this interpretation, but some intriguingevidence is available. For the 56 journala forwhich both Giles-Wright reputational and SSCIimpact data are available, the correlation betweenreputational scores and the reskfuul in these rep-utatiomdscores (theportian left over after regress-ing the Glles-Wright scores on the SSCI impactscores) is extremely high: r = .820. For the so-’.,ciology joumats, the correlation between the’tGlenn measure and the residual unexplained bythe SSCI impact score is even higher: r = .851.(These residuals are shown in the tfdrd columnof Tables 1 and 2.) These highly autocorrelatederror terms suggest that in each field high-atattta

269

journals tend to have better reputations than theirinfluence would warrant, while lower-statusjour-nals tend to have poorer reputations than their in-fluence would warrant. Subsequent examinationof scatter plots supfmrtsthis argument. This is norto say that none of the highly reputable journafsdeserves its reputation. For example, the Giles-Wright reputational score for the American So-ciological Review, 7.1, is very close to whatwould be predicted from a regression of Giles-Wright scoreson the SSCIimpactscores. General-ly, however, both very good and very bad reputa-tions tend to be exaggerations of what the impactdata suggest are merited, Especially noteworthyin this regard among the political science jour-nals are World Politics and the Journal of Poli-tics, both of whose reputationaf scores are farabove what would be predicted from the citationdata: World Politics, whose score of 7.3 placesit first among afl the political science journals,has a predicred reputatioml score of 6.07, closeto the mean on the Giles-Wright scale, and thereputational rating of 6.7 for the sixth-rankedJournal of Politics is also much higher than thepredicted score (5.7). Among the journals ratedby sociologists, the American SociologicalRew”ew,the AmericanJoumaf of Sociology,SocialForces, and the British Journal of Sociologydisplay the largest positive residuals, i.e., thelargest “unearned” reputations, though the firsttwo would still be very highly rated even if theirratingswere exacrfyconsonantwith their intluenceas measured by the SSCf citation data.

In short, the residuals provide strong presump-tive evidence that reputational measures of jour-naf quality reflect persisting stereotypes ratherthan simply summarizing actuaf influence. Thissuggests at the very least that widely held stereo-types about some of the most prominent sociologyand political science journals may need to be re-considered. It also suggests that in thinking aboutthe role variousjournals play in accreditingkrmwl-edge, it would be well to incorporate a behavioralas well as a reputational dimension.

RatingSociologyand Pofitieaf &ience Jourrsafs

Since the prestige rankingsof sociologyand po-litical sciencejournals were published in the ear-ly 1970s, many new journals have been estab-lished, the stature of journals may have changed,and citation information has become available.This rwent citation information provides behav-ior-based comparative data for a wide range ofjournrds in the sociaf sciences.

The SSCI journal impact data do pose someproblems, which we need to acknowledge. Oneproblem is that of incompletecoverage. The SSCIdata base does not inchrde seversdjournals thatare increasingly important publication outlets in

sociology and political science, In political sci-ence, the list of exclusions includes Polirica/Be-havior, Micropo[itics, and Political Psychology,to name only three examples from one relativelysmall comer of the discipline. If journal ratingsare to be based on the SSCf impact scores, thenbeing excluded from the SSCI data base is tanta-mount to being excluded from considerationaltogether.

Exclusion of journals from the SSCI data baseis a problem, but it is a problem of limited scope:the journals that are not included in the SSCIdatabase are, for the most part, joumafs that wouldnot score very high in terms of impact if they wereincluded, The truly major problem stems from thedifficultyof defining the boundariesof a scholarlydiscipline. If we wish to determine which are thebest sociology or political science journals, wemust first be certain what we mean by a sociologyor political sciencejoumaf. This is a very difficultproblem, and it is by no means Peculiar to theSSCI data base; indeed, it affects every attemptto evaluate journals in any field. For example,Glenn’s list of 63 journals includes several topjournals from other disciplines(e.g., the AmericanPoliricalScienceReview, the American Ecorrom”cReview, and the Harvard Iiducaricma/Review) as

well as numerous interdisciplinaryjournals (e.g.,Public Opinion Quarterly, Behavioral Science,and Social Science Quarterly), The SSCI, for itspart, categorizes journals according to their dk-ciplinary affiliation, but its categories are hard]yauthoritative. To cite only three examples, shouldCurrent History, IPW Benchre, and the Journalof CanadianStuales redly be considered three ofthe 77 journals subsumed under SSCI’Spoliticalscience category?

Despite these problems, the SSCI impact dataseem to us to provide a firmer foundation for as-sessing the quality of sociology and political sci-ence journals than rm~ other method devised tothis point. On the bas]s of the SSCI impact data,we get a fresh picture of the quality of severalestablishedjournals. For example, SociologyandSocial Research, which has been published foralmost three-quarters of a century, has an impactscore of only 0.103, which places it about 58thof the 66 journals in the SSCI scciologycategory.Similarly, the impact score of the venerable Po-litical Science QuarTerly(0.504) places it well be-low the other established political science jour-nals. More dramatically, Worki Pofirics, the mostprestigiousjournal according to the Giles-Wrightratings, has an impact score of 0.970, whichwould not place it among the top 10 in the Giles-Wright rating. Many regional journals also havelower impact ratings than might have been ex-pected (e.g., Sociological Quarrerly, Sociologi-cal Perspectives, and others not reported such asSociological Spectrum and Sociological Focus)

270

(data not presented). And some specializedjour-nals (e.g., AdministrativeScienceQuarrerly,Jour-nal of Healrhond SocialResearch,PublicInterest)have a greater impact than their reputationsworddsuggest. Of course, these comments should notbe taken out of the context of the SSCI impactscores upon which they are based; any problemsassociated with the SSC1data to measure journalquality will have to be borne in mind in interpret-ing journrd ratings based on the SSC1data.

Conehssion

If the medium accredits knowledge, assessmentof the impact of joumafs that constitute the me-dium for the exchangeof scholarly ideas demandsmore scrutinythan it has previouslyreceived. Thisstudy indicates that the prestige accorded manyjournals se-emsout of line with the impact thesejournrds have had in the social science researchcommunity. The relationship between reputationand citation impact is nonlinear, best describedas a sigmoid curve. A fairly clear-art prestigehierarchy is present, but many of the most pres-tigious joumaJs have less impact than might beexpected, and many other journals have more im-pact than is attributed to them by the reputationalratings.

REFERENCES

Tbe availability of behavior-based journal rat-ings shouldmitigate the mmmon tendencysimplyto eormtnumber of articles publishedas a measureof scientificproductivityor to limitjournal evalua-tions to outdatedreputatiorudhierarchies, It is easyto count articles, but it is difficult to draw mean-ingfulcomparisons.We believethat impact shouldbe weightedmuch more heavily than simple num-ber of articles or stereotypic journal reputationsin assessing accreditation of scholarly work.

The SSC1citation data permit scholars to eval-uate the importanceofjournals based not on opin-ion but on the frequency of citations. While suchassessments do not directly measure the quslityof journals, frequencyof citationimplies scholarlyacceptance, or at least acknowledgmentof impor-tance through utilization of others’ work. How-ever, the SSCI should not become the litmus testfor qurdityof social research. Journals have pres-tige, but their prestige is only derived from theusefulness of the articles they publish. In the longmn, individual articles and t-woksbexmmethe lit-mus test of quality. But practically,most of uswork within very limited time parameters, Thus,in the short run journal citation data do providedeans, tenure committees, and those studyingstratification in sciencea more defensibleand lessstereotyped means of measuring “accredited”knowledge than any other method now available.

Alliwn, Paul D., and John A. Stewan. 1974. ‘‘Productwiry Differences atnons Scientists: Evidence for Accumulative AdvamaSe,”American Sociological Review, 39 (Ausus1):5%-6C6.

Bin@a.m, Richard D., and Laura L. Verrz 19S3. “TIM Social Stmcture of an Academic Dkcipline: Networks snd PrestiSe in PoliticsJScience, ” Social Science Quarterly, 64 (Junc):275-S7.

Cole, Jonathan R., and Stephen R. Cole. 1973. social Stmtgi’cation in Science (Chicaso: University of Chicago Press).

Cole. S1cphen. 19S3. ‘The Hieramhy of the sciences,’” American Journal of .%ciolqry, S9 (July): 111-39.

Crane, Diana. I%5. “scientists at Major and Minor Universities: A StudYin Productivityand Recosniticm,” AmericanSocidogicd

Review, 30 (October) :699-7 14.

Garfield. Eugene. 1972. “Citation AnaJysis as a TCKIIin JmIrnsf Evfduation,’” Science, 17S (3 November):471 -79.

Giks, Michael W., and Gerafd C, Wright, Jr. 1975. “Politicaf Scientists’ EvaJuatkms of Sixty-’three Journals, ”PS. 8 (Summer):25’P57.

Glem, Nomat D. 1971. ‘‘Amertcan Sociologists’ Evaluation of SIxtyThree Journals,” Anwican SociologJsr, 6 (November) :29S-303.

Gordon, Michael D. 19S2. “Citation Ran!@ versus Subjective Evaluation in tk Determination of JomttaJ Hierarchies in the SociafSciences,” Journal of the A?menedn .$oaefy for Information Science, 33 (Janusry):55-57.

Harsens, Lowell L., and Dkne Felndee. 19S4. ‘Wructumf Determinants of Stratification in Science, ” Ametizzn sociological Review,

49 (Gctober):6S5 -97.

La@, J. Scott. 197S. “Pmductiviry and Academic Positions in the fkientific Career,” American .%ciobgical Review. 43

(lkember):SS9-90S.

Merton, Robert K. 1942. “science ml Techtwlogy in a Democratic Order,” Joum/oftigddPolidd tiobgy, 1 (Gaober): J J5-26.

---------------- 1%S. ‘The MattJIew Effect in Science, ” Science, 159 (5 January ):56-63.

Reskm, Barbara F. 1977. “Scientific Pmiwtivity and the Reward Stmctttre of Science.” American .$ociological Review, 42(June) :491-504

271