1
Sir — In an earlier Correspondence 1 , I explained why impact factors are irrelevant for judging the quality of taxonomy. E. Garfield 2 requested quantitative arguments to support my hypothesis that taxonomy follows different “regularities” from most sciences, making the impact factors as calculated by the ISI (formerly the Institute of Scientific Information) inapplicable. If ISI impact factors are to judge research meaningfully, as discussed in a recent News Feature 3 , they have to be a roughly accurate estimate of the real impact of publications. This requires that: (1) the impact of a paper is expressed by citations and the citation impact is positively correlated with the quality and/or relevance of the paper; (2) most cited and citing journals are considered (easily possible if Bradford’s law of scattering applies to a field, as most cited papers are published in a few core journals); and (3) a paper gets most of its citations in the first few years after publication. These requirements are not met by taxonomy, for the following reasons. First, the number of taxonomists is declining. Consequently, taxonomy does not follow the ‘exponential curve’ of most sciences. The old literature is not overwhelmed by an avalanche of new papers. The peak of species descriptions (a rough surrogate for relevant publications) was before 1900 for most groups, so the average age of references in taxonomic publications is much greater than those in other scientific disciplines. I have analysed 2,091 references from seven randomly chosen, comprehensive taxonomic papers. The mean age of the references is 61 years, the median 36 years (details available direct from F.-T. K.), with 98.5% of cited papers being more than two years old. In shorter taxonomic journal papers, F. Köhler 4 found the average age of citations to be 46.7 years (s.d. 30.5 yr) and 41.8 years (s.d. 22.7 yr). It is therefore pointless to judge taxonomists according to the ISI method of analysing citations over the preceding two years. Second, the relevance of descriptive publications in this field remains the same over time; original descriptions have to be referred to for ever, independent of the paper’s quality. Outside taxonomy, referring to original descriptions becomes superfluous with time and/or revisions. Einstein’s papers are not always cited for his theory, unlike the original descriptions of, say, Escherichia coli or Drosophila melanogaster. This obsolescence under- estimates impact only for authors of well- studied species 5 . However, in taxonomy this obsolescence effect is negligible. Third, for any group of organisms there are at best a handful of (or frequently no) extant specialists. Therefore the chance to become cited by colleagues is relatively rare compared with other fields. The number of taxonomists and consequently the number of publications is this low for one reason, which has nothing to do with need or quality: decision-makers are generally more enthusiastic about other fields. For these reasons, taxonomic papers have a long-term impact. Sometimes taxonomists have to wait a generation to be heavily cited. The number of citations of their empirical taxonomic publications depends on the number of taxonomists working on the same field and whether these colleagues publish in the few taxonomic journals covered by the Science Citation Index (SCI). These things are a matter of luck. Fourth, there are no core journals for general taxonomy. These exist for cladistics, biogeography, chemical systematics, and so on, but not for species descriptions, revisions of genera, identification keys or inventories. Where to publish depends on which museum the material is in; which institute or learned society the taxonomist is affiliated to; or which serial has the funding to accept long monographs. Only 27 (42%) of the 64 entomological journals covered by the SCI Expanded publish taxonomic papers, whereas 898 (82%) of the 1,100 serials held by the entomology library of the Natural History Museum in London probably contain taxonomic information. Because of the less-developed relevance hierarchy of taxonomic journals, the low proportion covered by the ISI puts taxonomy at a disadvantage. Finally, the most important impact of taxonomy is the usage of identification keys which enable non-taxonomists to identify and work with a group of organisms. The use of such keys is generally not documented in reference lists, hence a crucial impact of taxonomy is missed by citation analysis. Consequently, taxonomy has no citation classics which the ISI would uncover. E. O. Wilson’s citation classics aren’t his taxonomic works, and R. Sokal’s treatises on biometry and on the method of numerical taxonomy (not an empirical taxonomic work, but a methodological textbook on classification) have far more citations than his taxonomic work. The SCI is not an appropriate means to judge taxonomy because taxonomy does not meet its requirements for a meaningful judgement. Frank-Thorsten Krell Department of Entomology, The Natural History Museum, London SW7 5BD, UK [email protected] 1. Krell, F.-T. Nature 405, 507–508 (2000). 2. Garfield, E. Nature 413, 107 (2001). 3. Adam, D. Nature 415, 726–729 (2002). 4. Köhler, F. Amateurwissenschaft: Entwicklung, Beschreibung und wissenschaftssoziologische Analyse am Beispiel der Koleopterologie p. 195 (Diploma thesis: Sociology, Univ. Köln, 1988). 5. van der Velde, G. Nature 414, 148 (2001). correspondence NATURE | VOL 415 | 28 FEBRUARY 2002 | www.nature.com 957 Why impact factors don’t work for taxonomy Its long-term relevance, few specialists and lack of core journals put it outside ISI criteria. Physics gets physical Sir — The readable and entertaining Words article “Coming to Terms” by J. L. Heilbron (Nature 415, 585; 2002) on scientific nomenclature is amusing, but potentially misleading as far as it concerns high-energy physics. It may be true that the first flippant names such as ‘quark’ and ‘gluon’ were bestowed by US researchers, but they have monopolized neither the discoveries nor the follies. The inspirationally named gluon — the elementary particle carrying the force that ‘glues’ quarks together — was discovered at Germany’s high-energy accelerator centre, DESY, in Hamburg in 1979, and the more prosaically named ‘intermediate boson’ was discovered at CERN, the European laboratory for particle physics, in Geneva in 1983. Less seriously, I plead guilty to coining TOE as a non-anatomical acronym for Theory of Everything in an article that appeared in Nature (323, 595–598; 1986). As for Grand Unified Theories or GUTs, the term was coined by my CERN collabo- rators Andrzej Buras (Polish/German), Mary K. Gaillard (American/French), Dimitri Nanopoulos (Greek) and myself (British). But we did not have the ‘guts’ to put the acronym in our paper on this subject (Nucl. Phys. B 135, 66; 1978) — instead, we weaselled out and used the equally non-anatomical GUM for Grand Unification Mass. To the best of my knowledge, GUTs were first spilled, metaphorically, by Dimitri Nanopoulos in a paper (Harvard Preprint HUTP-78/A062) published later in 1978. As well as making discoveries, we Europeans can be just as facetious, jocular and capricious as our US friends. John Ellis Theoretical Physics Division, CERN, CH-1211 Geneva 23, Switzerland © 2002 Macmillan Magazines Ltd

Why impact factors don't work for taxonomy

Embed Size (px)

Citation preview

Page 1: Why impact factors don't work for taxonomy

Sir — In an earlier Correspondence1, Iexplained why impact factors are irrelevantfor judging the quality of taxonomy. E. Garfield2 requested quantitativearguments to support my hypothesis thattaxonomy follows different “regularities”from most sciences, making the impactfactors as calculated by the ISI (formerlythe Institute of Scientific Information)inapplicable.

If ISI impact factors are to judgeresearch meaningfully, as discussed in arecent News Feature3, they have to be aroughly accurate estimate of the realimpact of publications. This requires that:(1) the impact of a paper is expressed bycitations and the citation impact ispositively correlated with the qualityand/or relevance of the paper; (2) mostcited and citing journals are considered(easily possible if Bradford’s law ofscattering applies to a field, as most citedpapers are published in a few core journals);and (3) a paper gets most of its citations inthe first few years after publication. Theserequirements are not met by taxonomy, forthe following reasons.

First, the number of taxonomists isdeclining. Consequently, taxonomy doesnot follow the ‘exponential curve’ of mostsciences. The old literature is notoverwhelmed by an avalanche of newpapers. The peak of species descriptions (arough surrogate for relevant publications)was before 1900 for most groups, so theaverage age of references in taxonomicpublications is much greater than those inother scientific disciplines.

I have analysed 2,091 references fromseven randomly chosen, comprehensivetaxonomic papers. The mean age of thereferences is 61 years, the median 36 years(details available direct from F.-T. K.), with98.5% of cited papers being more than twoyears old. In shorter taxonomic journalpapers, F. Köhler4 found the average age ofcitations to be 46.7 years (s.d. 30.5 yr) and41.8 years (s.d. 22.7 yr). It is thereforepointless to judge taxonomists accordingto the ISI method of analysing citationsover the preceding two years.

Second, the relevance of descriptivepublications in this field remains the sameover time; original descriptions have to bereferred to for ever, independent of thepaper’s quality. Outside taxonomy,referring to original descriptions becomessuperfluous with time and/or revisions.Einstein’s papers are not always cited forhis theory, unlike the original descriptionsof, say, Escherichia coli or Drosophilamelanogaster. This obsolescence under-

estimates impact only for authors of well-studied species5. However, in taxonomythis obsolescence effect is negligible.

Third, for any group of organisms thereare at best a handful of (or frequently no)extant specialists. Therefore the chance tobecome cited by colleagues is relatively rarecompared with other fields. The numberof taxonomists and consequently thenumber of publications is this low for onereason, which has nothing to do with needor quality: decision-makers are generallymore enthusiastic about other fields. Forthese reasons, taxonomic papers have along-term impact. Sometimes taxonomistshave to wait a generation to be heavilycited. The number of citations of theirempirical taxonomic publications dependson the number of taxonomists working onthe same field and whether these colleaguespublish in the few taxonomic journalscovered by the Science Citation Index(SCI). These things are a matter of luck.

Fourth, there are no core journals forgeneral taxonomy. These exist for cladistics,biogeography, chemical systematics, andso on, but not for species descriptions,revisions of genera, identification keys orinventories. Where to publish depends onwhich museum the material is in; whichinstitute or learned society the taxonomistis affiliated to; or which serial has thefunding to accept long monographs. Only27 (42%) of the 64 entomological journalscovered by the SCI Expanded publishtaxonomic papers, whereas 898 (82%) ofthe 1,100 serials held by the entomology

library of the Natural History Museum inLondon probably contain taxonomicinformation. Because of the less-developedrelevance hierarchy of taxonomic journals,the low proportion covered by the ISI putstaxonomy at a disadvantage.

Finally, the most important impact of taxonomy is the usage of identificationkeys which enable non-taxonomists toidentify and work with a group oforganisms. The use of such keys isgenerally not documented in referencelists, hence a crucial impact of taxonomy is missed by citation analysis.

Consequently, taxonomy has nocitation classics which the ISI woulduncover. E. O. Wilson’s citation classicsaren’t his taxonomic works, and R. Sokal’streatises on biometry and on the methodof numerical taxonomy (not an empiricaltaxonomic work, but a methodologicaltextbook on classification) have far morecitations than his taxonomic work. TheSCI is not an appropriate means to judgetaxonomy because taxonomy does notmeet its requirements for a meaningfuljudgement. Frank-Thorsten Krell Department of Entomology, The Natural HistoryMuseum, London SW7 5BD, [email protected] 1. Krell, F.-T. Nature 405, 507–508 (2000).

2. Garfield, E. Nature 413, 107 (2001).

3. Adam, D. Nature 415, 726–729 (2002).

4. Köhler, F. Amateurwissenschaft: Entwicklung, Beschreibung und

wissenschaftssoziologische Analyse am Beispiel der Koleopterologie

p. 195 (Diploma thesis: Sociology, Univ. Köln, 1988).

5. van der Velde, G. Nature 414, 148 (2001).

correspondence

NATURE | VOL 415 | 28 FEBRUARY 2002 | www.nature.com 957

Why impact factors don’t work for taxonomyIts long-term relevance, few specialists and lack of core journals put it outside ISI criteria.

Physics gets physical Sir — The readable and entertainingWords article “Coming to Terms” by J. L. Heilbron (Nature 415, 585; 2002) onscientific nomenclature is amusing, butpotentially misleading as far as it concernshigh-energy physics. It may be true thatthe first flippant names such as ‘quark’ and‘gluon’ were bestowed by US researchers,but they have monopolized neither thediscoveries nor the follies.

The inspirationally named gluon — theelementary particle carrying the force that‘glues’ quarks together — was discoveredat Germany’s high-energy acceleratorcentre, DESY, in Hamburg in 1979, and themore prosaically named ‘intermediateboson’ was discovered at CERN, theEuropean laboratory for particle physics,in Geneva in 1983.

Less seriously, I plead guilty to coiningTOE as a non-anatomical acronym for

Theory of Everything in an article thatappeared in Nature (323, 595–598; 1986).As for Grand Unified Theories or GUTs,the term was coined by my CERN collabo-rators Andrzej Buras (Polish/German),Mary K. Gaillard (American/French),Dimitri Nanopoulos (Greek) and myself(British). But we did not have the ‘guts’ toput the acronym in our paper on thissubject (Nucl. Phys. B 135, 66; 1978) —instead, we weaselled out and used theequally non-anatomical GUM for GrandUnification Mass. To the best of myknowledge, GUTs were first spilled,metaphorically, by Dimitri Nanopoulos ina paper (Harvard Preprint HUTP-78/A062)published later in 1978.

As well as making discoveries, weEuropeans can be just as facetious, jocularand capricious as our US friends.John EllisTheoretical Physics Division, CERN, CH-1211Geneva 23, Switzerland

© 2002 Macmillan Magazines Ltd