Embed Size (px)
Citation preview
200
EUGENE GARFIELD, MEMBER, IEEE
A* "tiTfLc£ — Citation snslyss provides s tool for ^vaiiiaivwi tH*? r °l*-tive importance of journals as well as papers. A publication, the 1SI Journal Citation Reports, is issued primarily to permit users to make such evaluations by answering the basic questions: How often has a journal been cited? What journals have cited it and how frequently? What journals has it cited and how frequently? These data have been conditioned by the inclusion of a "relative impact factor" to give additional weight to the significance of the articles it publishes. The information thus developed is particularly attractive to librarians, journal editors and publishers, individual scientists, and science policy-
|T7* o r more than a decade, the Institute for Scientific In-J&. formation has produced a reference tool called tne Science Citation Index (SCI) [ 1 - 3 ] . Coverage of the SCI has grown from 600 to 2400 journals, and now includes ail o f t h e w o r l d ' s s i g n i f i c a n t SCient i fJ -^ a n H technical p e r i o d i
cals. It is one purpose of this paper to show how that fact can be demonstrated by use of citation analysis.
The effectiveness of the SCI in information retrieval has been well established [ 4 ] . Reports in the literature have described its use in various fields of science and technology. It may be desirable, however, to explain briefly the principle of citation indexing, and its application in the
THRRWIOJRT £ footnote, book enc^'clo^ed*0 or conventional word or subject index. He then enters the Citation Index section o f the SCI and searches for that particular author's name. When he locates the author's name, he then checks to see which o f several possible references fits the particular one that he is interested in. Under the year, journal, volume, and page number of mis particular reference, he iiicii iooKs to see who has currently cited tins particular work. Having noted the bibliographical citations of the authors who arc Citing the work with which lie started, the searcher then turns to the Source Index section and ob-j--z .l i i_:t_i: c -l _i
n i c C/UiiijJicic u i U i i u g i a p i n e a i u a i a i u i i n c w u i n s
which he has found. Tne fundamental question one can answer quickly
t h r o u g h th^. (^ifnt^rsy T*?r?s*y w n » r f » <*r>c\ f>v w h r v m tV>i«?
paper been cited in the literature? The SCI is also used by scientists to determine whether their work has been applied or criticized by others. It can facilitate feedbacks in the communication cycle. Any author may choose to ignore the citations to his own work and still use the Index to retrieve publications which cite work by other scientists. The SCI can be used to identify scientists currentiv
SCL The SCI is composed of three basic parts: the Source
Index, the Citation Index, and the Permuterm Subject Index. The Source Index gives a complete bibliographic description of all source items from the journals processed for the SCI: authors; title; journal title; journal volume, issue, pagination, and year. Items are arranged alphabetically by first author. The Citation Index is an alphabetic list, arrsnCTed b w first author, o f items cited in the references given in the bibliographies and footnotes of source articles, each reference followed by brief ds scrip-J-1 / » ; a „ a : \ _ j . ' „1 __.L*_t- U 1 : 1
ll'OiiS V —1 - — -i —''1 - 7 MIUiLC UlllKslZ;* Willk'ii !I<|VC IJJILKIJ M.
The Permuterm Subject Index is an alphabetic subject index derived from words in the titles of source articles [ 5 ] . Every significant word in a title is paired with all other significant words- and r e f e r e n c e is made from each such NAIR T O T H E A N T H N R N F TFIP «OIIRR.P. ARTICLE i — — ~ — — — ~ ~ —
Using the SCI is a relatively simple affair. The searcher UVUl VC» l i l V i l u * v t v i v i i w \JM. u n u v. i u i n v n u o XUV1I1111WU
working on special problems or to determine whether a paper has been cited, whether there has been a review of a subject, whether a concept has been applied, a theory confirmed, or a method improved. Because indications of corrections are indexed like other items from the covered source journals, the SCI is useful as an aid in following particular articles. Only the user's imagination limits the extent to which the SCI can be a useful tool for the scientist and librarian.
The SCI is published quarterly and cumulated annually and quinquennially. However, the data base from which the periodic volumes o f the SCI are derived has been rnaii;-tained on magnetic tape, and it is updated weekly. By the end o f 1972. ISI had processed more than 2 million source items from the journal literature — research reports, letters T O the E D I T O R . M E E T I N G Abstracts corrections ?*nd ern?tR editorials, reviews, bibliographies, etc. Along with complete bibliographical descriptions of these 2 million source
i t e i i i S , tlic d a t a baS6 h a d p i G C C S S C u i T i O r e t i lan 3(5 million i v i f i v i l V v D 11 W i l l n i l / o u u i v v x iv i i i l } i v / v i i i v t v j u n u u i u n v /
Manuscript received September 10, 1973. The author is with the Institute for Scientific information, Philadelphia, Pa.
19106.
This paper was presented at the Conference on the Future of Scientific and Technical Journals, New York, May 17-19, 1973. The manuscript arrived too lets to ba included in the September (Ccnfsrsr.es Record) iss«e of the PC TRANSACTIONS.
i- : - _ _ i* t_i_i_ _t JL '-i nni — ^ o —:ti?_..- '„.,..-.. .1 ^ i a p i i i o a , O i W i i i C i i d U O U t / OVO, O i ^ .O J i i i i i i u i i , UUllicllIlCU
citations of other items in the journal literature. Obviously, a citation index is based on the principle that
there is some meaningful relationship between one paper and another which cites it or by which it is cited, and thus between the work of the two authors who have published the articles. Such relationships can be extended, using the SCL backward and forward in time through the work of
IEEE TRANSACTIONS ON PROFESSIONAL COMMUNICATION, VOL. PC-16, NO. 4, DECEMBER 1973
What Scientific Journals Can Tell Us About Scientific Journals
GARFIELD: WHAT SCIENTIFIC JOURNALS C A N TELL US A B O U T SCIENTIFIC JOURNALS 201
other men, to create citation networks showing the development o f particular ideas and concepts in research, as that development has been recorded in the literature [6 , 7 ] .
Such networks are graphic representations of the relationship between the work of individuals. Using the SCI
ships between journals. As noted above, the entire SCI data base is stored on
macmetir. tane. and it is thus amenable to extensive m s n l D -
ulation and analysis. In the case of individual authors, we have been able to identify the frequency with which individual authors and individual papers are cited in the literature. Counts o f this sort are strictly quantitative and objective, but even admitting this limitation, an author's or a paper's frequency of citation has been found a useful measure o i the man's or ibe nanprs contribution to or impact upon the field. It has been found to correlate well with professional standing [ 8 j . It is certainly not the only measure, nor one that can be used, for any purpose, in isolation; nor do we claim for it the absolute reliability that critics o f citation analysis have wrongly imputed to us when they have attacked it. The fact does remain, however, that it provides a useful objective criterion that we have previously lacked.
We have used basically the same principle to evaluate journals, rather than papers. When a scientist cites a previously published article, he tells you, in a sense, that he has read it and has some reason for bringing it to your attention. The more frequently a journal's articles are cited- the more the world's scientific community is expressing the fact that it finds the journal to be a carrier o f useful information.
It has always been and still remains difficult to assess the relative importance of scientific and technical journals. There have been few, if any5 totally objective criteria by which to measure them. Many librarians learn, in time, which of those journals are most "important" for particular disciplines, and there have been countless studies to establish core-journal lists. But when choices must be made, when one must live within a tight acquisitions budget, considerations less relevant than scientific merit
habit, relative subscription cost, and so on. As the title o f this paper indicates, the journals themselves can tell us a lot about scientific journals. What their citation patterns tell us cdii now be deteririirieu i r o r r i u i c i&i joutuui i^tiu^ Hon Reports (JCR) [9, 10] .
Before describing the type of information provided by ICR vVC should ITlalce c ! e S r ^"HP /?pnnft? Tf* r * o i n ? > i l p H s
To create the first JCR., every reference citation from every item published during the last quarter of 1969 was extracted from the SCI data base. This sample from the total data base amounted to 926,928 citations. Through an assortment of computer manipulations, we determined the total number of times a particular journal was cited, and the number of times it was cited by any other journal. We determined in addition how many times it had cited other journals, and how often it had cited each. All o f these
counts were distributed by year of publication of citing and cited journals. With these counts and distributions we have been able to rank journals in various ways.
By vigorous sample validation, we established that the data taken from the last quarter of 1969 were representative of data recorded from the source journals for the entire year. Journal r s n k i n g f v produced would r^nisin esscn^ tially the same no matter whether data from the full year or from one calendar quarter were used. We also established that the rankings ^nd relationships indicated by the 1969 data would remain substantially unchanged i f they were to be recalculated from more recent data. Thus, the JCR data transcend the time period from which they were extracted and have value as a tool for objectively viewing, measuring, and evaluating the current journal literature. It is recognized, however, that periodic updating o f the ssinpis w i n O " requires i c assure accuracy tor Hie iuiure.
The JCR answers these basic questions: how often has a journal been cited? what journals have cited it, and how frequently? what journals has it cited, and how fre-n n p n t l v ^
In the first case, we are dealing with absolute citation frequency, which, without reference to other factors, reflects a journal's value and the use made o f it It is a rough measure, to be sure, because citation frequency is a function of other variables besides scientific merit. Some of them are authors' reputations, controversially of the subject, journal circulation, reprint dissemination, coverage by secondary indexing and abstracting services, availability in library collections, and so on. An immediate objection to such a ranking by absolute citation frequency may be that it reflects the size of a journal — the number of articles it publishes — as much, if not more, than it reflects the significance of those articles. In view of the undoubted relation between size and citation frequency, we have attempted to discount it. or at least minimize it, by calculating a relative impact factor - that is, by determining the number of times a journal has been cited in relation to the number of articles it has published. This journal impact factor thus reflects the frequency with which the "average" article in a particular journal is cited.
Impact can be measured in various ways. One method that described above — is to compute the ratio between
citations to particular years of a journal and the number of articles nublished in those years. ^In thr current JCR we have developed the impact factor by dividing the number of limes a journal was cited in 1969 by the number of articles it published during 1967 and 1968.) T o measure impact, one can also use the ratio of citations to the number o f articles actually cited one or more times, disregarding those which are not cited. A very different ratio expresses the fraction of articles cited. One can also try to discount the "inbreeding" effect of self-citations. Further, so-called journal utilization factors indicate the number of different citing journals involved [ 1 1 ] .
The ratio of citations to sources provides an overall measure of impact. Unless limited by chronological criteria, the ratio can be skewed by a few supercited classics. A single article cited 500 times has the same effect
202 IEEE TRANSACTIONS O N PROFESSIONAL COMMUNICATION, DECEMBER 1973
as 100 articles cited five times each — a very high impact factor. The putative impact factor mentioned above, citations in relation to cited sources rather than all published sources, discounts the negative influence o f articles which are never cited.
However useful absolute citation frequency may be, it is not sufficient for the task of journal selection, except perhaps to establish "core"' journal collections. It is important to be able to answer the second question: what journals have cited the journal in question, and how frequently? In other words, who is using the journal, and how often? in what type o f library does it belong? What kind of journal is it? Answers to these questions, as provided by the JCR, can be extremely useful in characterizing a journal, and determining the priority it should be accorded in an acquisition list, or in a reading list for that matter [ 1 2 ] .
Finally, the JCR provides answers to the third question: what journals has the journal cited, and how frequently? What other journals will interest authors who have published in this journal, and to what extent? Editors and contributors are frequently surprised to learn the citation practices in their own journals.
The usefulness of the JCR and its analyses of journal citation patterns depends to a great extent upon the intent and upon the insight of the user. As a tooi for journal evaluation, we believe the JCR may quickly prove indispensable to four groups: chief librarians and acquisitions librarians; individual scientists; journal editors and publishers; and, finally, science policy-makers.
Librarians can use the JCR immediately to determine the optimum makeup of special and general journal collections; determine the optimum size of back files of journals; provide a basis for cost-benefit analysis in managing subscription budgets; determine appropriate journal circulation and photocopying policies.
Individual scientists can use the JCR to identify lesser-known journals important to a given field; determine how long articles in a given journal are likely to be useful; identify journals from other fields that may be related to primary interests; finally, identify journals in which to publish.
Journal editors and publishers can use the JCR to evaluate editorial policies. Useful indicators o f a journal's per-formance are the extent of self-citation, the number o f times cited per year, and the distribution of citations viiiiV/ii viiiug JVUlllUlO TTlllilll U 1 1 U VUIOIUV 1 1 1 V / O p V / V I O l l ^ .
ence of science can use the JCR to evaluate research fronts; to model journal communication networks; to define disciplines and specialties through journal relationships; to determine the interdisciplinary or multidisciplin-ary character o f research projects; and to evaluate funding and subsidy requests for journal publication in the light o f library acquisitions and predicted use patterns.
In conclusion, we should like to stress the dual nature o f journal evaluation. It attempts first to characterize a journal, and second to judge its importance in relation to others, to say that it is more or less important than others. The issue o f importance is frequently difficult to discuss. After all, the mere fact that a journal is published must
mean that it is important to someone, or to some group. We mean by important significant in the transfer of information between research scientists. It has been shown that about 2000 journals account for almost 85% of all citations, that an even smaller core of journals accounts for the majority of articles published, and that as few as 300 journals provide more than half the references [ 13] . These facts, and the relationships displayed by the ISI Journal Citation Reports, allow one to determine importance as it has been assigned by the scientific community itself.
iAbLhi Interview Questions and Summary Data
1. Approximately what percentage of your research time do you work with security classified information? (n = 184)
0%-20% 21%-40% 41%-60% 61%-80% 81%-100%
149 2i 0 6 8
2. In your opinion, have security classification restrictions seriously impeded the flow of scientific and technical communication in your specialty? (n - 184)
Yes No
58 126
3. Do you receive abstracts of classified technical information in your specialty? (Asked only of the 14 respondents who indicated 41 percent or more in Question 1.)
Yes . No
0 14
4. With whom in your work group would you consult for the latest security classified technical information relevant to your current project? (n = 14)
Number of respondents naming no individuals — 14
5. In searching for classified technical information, to which sources (in order) do you normally turn? (n = 14)
(Data presented in Table II)
I'ABLE ii
Classified Technical information Source First
Number of Professionals Using Source:
Second Third Fourth
Office Classified Files 14 0 0 0
Library and Technical Information Office
0 10 4 0
Contract Bibliographies 0 4 8 2
Defense Documentation Center 0 0 ? 12 Abstract Lists
Kendall Coefficient of Concordance (W) = 0.57.
GARFIELD: WHAT SCIENTIFIC JOURNALS C A N TELL US A B O U T SCIENTIFIC JOURNALS 203
TABLE III Dissemination Statistics for Laboratory Documents
Number of Laboratory Documents Generated in 1970
Classified:
Top Secret Secret
Unclassified
Limited Distribution No Restrictions
Total
127 66
193
TABLE IV Request for Information from the General Public*
"Freedom of Information" (For Period 1 May-1 Nov 71)
Number of requests for technical information from the general public 347
Disposition of requests:
Requests referred to other military organizations 16 because of incorrect address
Foreign requests 35
General public 'unlimited distribution) requests 160 referred to Defense Documentation Center
DoD contractors (security classified) requests released 42 through the appropriate security agency
Total 347
Note: No denials for information during above period.
* In formation taken directly from lab report.
TABLE V Comparison of Security Classification Guidelines
Typical Subject Laboratory Guidance*
The work and developed data under this project and supporting tasks are generally Unclassified. However, new techniques which constitute a breakthrough in current state-of-the-art and have military application will warrant protection up to and including the Secret level. Decisions as to warranted level of classification to be assigned will be made by [laboratory project group].
Physical characteristics
Estimates or proven conclusions as to system capabilities and/or operational limitations
Vulnerability
^oniiaential
Unclassified
Applies to speed, altitude, maximum and minimum range, maneuverability limitations, and other specific performance parameters.
Applies to parameters not within scope of above. Performance when no figures are used is Unclassified.
Applies to length, height, weight, scale models, and photographs.
Secret Applies to any data relating to overall system vulnerability.
RE FERENCES
[ 1 ] Garfield, E. Citation indexes for science. Science 122:108, 1955. [2] Science Citation Index, a new dimension in indexing. Science
144:649, 1964. 13] Maliri- M/ 'V . The Scicv.ce Citation Index' 3 new concent in indexing.
Library Trends 16:374, 1968. [4] See, tor example: Cawkeil, A. E. Using references to retrieve current
articles. Radio & Electronic Engineer 35:352, 1968. Cawkeil, A. E. Citations in chemistry. Chemistry in Britain 6:414, 1970. Spencer C. C. Subject searching with Science Citation Index: preparation of a drug bibliography using Chemical Abstracts, Index Medicus, and Science Citation Index. American Documentation 18:87, 1967.
(5] Garfield, E. Permuterm Subject Index, the primordial dictionary of science. Current Contents 12(22):6, 1969.
{6} Garfield, E. & Cawkeil, A. E. Location of milestone papers through citation networks. Journal of Library History 5:184, 1970.
[7] Price, D. J. D . Networks of scientific papers. Science 149:510, 1965.
[8j See, for example: Hagstrom, W. O. Inputs, outputs, and the prestige of university science departments. Sociol. Educ. 44:375, 1971. Margolis, J. Citation indexing and evaluation of scientific papers. Science 155:1213, ;967. C . R. Myers. Journal citations and scientific eminence in contemporary psychology. Amer Psychol, 2.5:1041, 1970.
[9] Garfield, E. Is citation frequency a valid criterion for selecting journals? Current Contents No. 14, 5 April 72, p. 5-6.
[10] — . ISI's Journal Citation Index data base - a multi-media tool. Current Contents No. 16, 19 April 72, p. 5-8.
[ ! ! ] Garfield, E . &. Sher. I . H- New factors in the evaluation of scientific literature through citation indexing. American Documentation 14:195, 1963.
[12] The following provide useful examples in journal characterization using the JCR: Garfield, E. What is the "core" literature of biochemistry as compared to the "core" literature of chemistry. Current Contents No. 5, 2 February 72, p. 6-9. — . What is the "core" literature of chemical physics? Current Contents No. 9, 1 March 72, p. 5-8. — . Journal Citation Studies III. Journal of Experimental Medicine compared with Journal of Immunology, or, How much of a clinician is the immunologist. Current Contents No. 23, 7 June 72, p. Ml-4.
[13] Garfield, E. Citation analysis as a tool in journal evaluation. Science 178:471. 1972.
0 0
Typical Comparison Laboratory Guidance*
Specific performance Secret
Genpra l rw?rformLirtCC
Secret
*Ta!;cn directly from security classification guides. Oniy 5 of nearly 40 items for comparison laboratory project reproduced.
