What is wrong with Journals' Impact Factor

Ahmed-Refat Refat- -Journals Impact FactAhmed-Refat Refat- -Journals Impact Factoror

The Impact FactorThe Impact Factorof Journalsof Journals

ب ب ليس تقيم IFIFليس تقيم وحدة وحدةاالبحاثاالبحاث

Dr.-Ahmed-Refat-AGDr.-Ahmed-Refat-AG 12-01-200912-01-2009


In This presentation In This presentation

Ranking of Universities and its relation to Quality of journals & Publishing

IF Alternatives IF Alternatives H-indexH-indexGSGSSJRSJR

Impact Factor FeverHistory of IFCalculation of IFUses and Misuses of IF of IF

Raising Star Raising Star H-H-Index !!!Index !!!What is your H-What is your H-index ??index ??

1

2

3

14


Moed HF, Van Leeuwen TN (2005) Citation analysis inMoed HF, Van Leeuwen TN (2005) Citation analysis inresearch evaluation. Springer, New Yorkresearch evaluation. Springer, New York

‘‘It was no longer enough just It was no longer enough just to to publish in bulk:publish in bulk:

it had to be demonstrable that it had to be demonstrable that your your publications were also heavily used,publications were also heavily used,

hence useful and importanthence useful and important..

A direct indicator of usage was the fact A direct indicator of usage was the fact that your research was that your research was

cited by subsequent researchcited by subsequent research.’.’


Citation Fever & Citation Fever & Impacities !!!???Impacities !!!???

Just Only Two Examples: Just Only Two Examples: <<<<<<<<<<<<<<<<<< Impact fever > Impact fever >>>>>>>>>>>>>>>>>>>

1.1. Oxford JournalsOxford Journals

2.2. BioMed Central “BMC” JournalsBioMed Central “BMC” Journals



<<<<<<<<<<<<<<<<<< Impact fever > Impact fever >>>>>>>>>>>>>>>>>>>

Oxford University JournalsOxford University Journals

mini survey ~ to know some features of IF mini survey ~ to know some features of IF as:as:

Magnitude , Variation, Ranking of specific J,Magnitude , Variation, Ranking of specific J,

Position of IF “ culture” in editorial policy Position of IF “ culture” in editorial policy and more….and more….Just have a look , take a Just have a look , take a

note , hold your questions , and note , hold your questions , and wait for the second example wait for the second example BMCBMC....



Do journals published by Do journals published by BioMed CentralBioMed Central have have Impact Factors and are their citations tracked?Impact Factors and are their citations tracked?

Yes; for any journal to have an Yes; for any journal to have an Impact FactorImpact Factor, however, it must be tracked , however, it must be tracked by by Thomson Reuters (ISI)Thomson Reuters (ISI) for three years.. for three years.. The following list shows those journals The following list shows those journals that are already tracked by Thomson that are already tracked by Thomson Reuters (ISI) and so already have, or will Reuters (ISI) and so already have, or will soon have, soon have, Impact FactorsImpact Factors..


Breaking News …Good Breaking News …Good News ??? News ???



It is a great honor to be It is a great honor to be indexed in Pub-Med ???????indexed in Pub-Med ???????


Please Recommend me to Please Recommend me to ISI ???ISI ???

Additional BioMed Central journals Additional BioMed Central journals are currently being considered for are currently being considered for tracking by Thomson Reuters (ISI). If tracking by Thomson Reuters (ISI). If the journal you are interested in is the journal you are interested in is not currently tracked, you can not currently tracked, you can recommendrecommend it to Thomson Reuters it to Thomson Reuters (ISI) for tracking(ISI) for tracking




Approaches to Ranking JournalsApproaches to Ranking Journals


Impact FactorImpact Factor

An Impact Factor is one measure of the An Impact Factor is one measure of the relative importance of relative importance of a journala journal, , individual individual articlearticle or or scientistscientist to to science and social science literature and science and social science literature and research. research.

Each index or database used to create an impact factor uses a Each index or database used to create an impact factor uses a different methodology and produces slightly different results, different methodology and produces slightly different results, revealing the importance of using several sources to judge the revealing the importance of using several sources to judge the true impact of a journal's or scientist’s worktrue impact of a journal's or scientist’s work


Impact FactorImpact Factor

Journals RankingJournals RankingAuthor RankingAuthor Ranking


Journal Impact FactorJournal Impact Factor

Journal Impact Factor is frequently Journal Impact Factor is frequently used as a proxy for the used as a proxy for the importanceimportance of a journal to its field. of a journal to its field.



In general we can distinguish two broad In general we can distinguish two broad approaches to ranking journals:approaches to ranking journals:

1.1. Stated preferenceStated preference (or peer review) and (or peer review) and

2.2. Revealed preferenceRevealed preference



StatedStated preferencepreference involves members of particular academic involves members of particular academic

community ranking journals on the basis of community ranking journals on the basis of their own expert judgementstheir own expert judgements


StatedStated Preference Preference RankingRanking

e.g. the British e.g. the British ABS Quality Journal GuideABS Quality Journal Guide and and Journal Quality ListJournal Quality List). ).

Opinions might be based on anything from a Opinions might be based on anything from a large-scale worldwide survey of academics to large-scale worldwide survey of academics to a small group of individuals with decision-a small group of individuals with decision-making power, but will always contain some making power, but will always contain some element of subjectivityelement of subjectivity


Revealed Preference RankingRevealed Preference Ranking

Revealed preferenceRevealed preference rankings are based on rankings are based on actual actual publication behaviour and generallypublication behaviour and generally

measure the measure the

citation rates of journalscitation rates of journals


Several indicators of productivity or quality of Journals Several indicators of productivity or quality of Journals have been proposed, which correlate with different have been proposed, which correlate with different aspects of productivity and quality. aspects of productivity and quality.

The first of these indicators, The first of these indicators, overall citation overall citation countscounts (‘citedness’),(‘citedness’),will remain with us will remain with us as as an index of the overall scientific impactan index of the overall scientific impact (Garfield 1977–1993),(Garfield 1977–1993), while while second order indicatorssecond order indicators can be used to can be used to explore subtly-grained aspects of this impact. explore subtly-grained aspects of this impact.

Revealed PreferenceRevealed PreferenceFirst Order RankingFirst Order Ranking


A well-known second-order indicator A well-known second-order indicator

is theis the

JJournal ournal IImpact mpact FFactor (JIF)actor (JIF)

Revealed Preference 2Revealed Preference 2ndnd order order RankingRanking


What is Impact Factor?What is Impact Factor?

Impact Factor Impact Factor IFIF is is one of one of the quantitative tools for ranking, the quantitative tools for ranking, evaluating, categorizing, and evaluating, categorizing, and comparing journals. comparing journals. . .



It is a measure of the frequency with It is a measure of the frequency with which the "average article" in a which the "average article" in a journal has been cited in a particular journal has been cited in a particular year or period. year or period.



The annual impact factor is a ratio The annual impact factor is a ratio between citations and recent citable between citations and recent citable items published. items published.



Thus, the impact factor of a journal is Thus, the impact factor of a journal is calculated by dividing the number of calculated by dividing the number of current year citations to the source current year citations to the source items published in that journal during items published in that journal during the previous two yearsthe previous two years



Impact factor was first proposedImpact factor was first proposed by by Eugene GarfieldEugene Garfield

who is a chemist, librarian, and linguist who is a chemist, librarian, and linguist by training by training in a 1955in a 1955

article in the journal article in the journal Science.Science.

Chairman Emeritus, Institute for Scientific Information, ISI Chairman Emeritus, Institute for Scientific Information, ISI Philadelphia.Philadelphia.


Calculation of IF


Example Example The specific calculations for The specific calculations for OMJ OMJ 's 2007 's 2007

impact factor are displayed below.impact factor are displayed below.(Numerator)(Numerator)

Articles published in Articles published in 20020066 that were that were citedcited in 2007: 98in 2007: 98

Articles published in Articles published in 20020055 that were that were citedcited in 2007in 2007: : 103103

98+103=98+103=201 201

Calculation of IF


(Deneminator)(Deneminator)

Total Number of articles Total Number of articles published in 200published in 20066: 67: 67Total number of articles Total number of articles

published in 200published in 20055: 48: 4867+48=11567+48=115

IFIF=201/115= =201/115= 1.7481.748

Calculation of IF


201201 (articles published in 2006 and 2005 that were cited in 2007)(articles published in 2006 and 2005 that were cited in 2007)

115 (115 (total number of articles published in 2006 and 2005)total number of articles published in 2006 and 2005)

= 1.748= 1.748

The 2007 IF for the OM journal means that, The 2007 IF for the OM journal means that,

on average, articles published in this on average, articles published in this journal from one or two years ago have journal from one or two years ago have been cited around 1 and three-quarter been cited around 1 and three-quarter

times.times.

Calculation of IF


Calculation of IF



. .


Origin of Impact FactorOrigin of Impact Factor







Impact Factor DebateImpact Factor Debate

Impact factors have been much Impact factors have been much debated in the literature in terms of debated in the literature in terms of their value for evaluating research their value for evaluating research quality. quality.



The impact factor is a citation measure The impact factor is a citation measure produced by produced by

Thompson Scientific's ISI Thompson Scientific's ISI

Web of Knowledge databaseWeb of Knowledge database. .



Impact factors are published annually Impact factors are published annually in ISI's in ISI's Journal Citation ReportsJournal Citation Reports

Database.Database.



Impact factors are only available for Impact factors are only available for journals that are indexed in ISI journals that are indexed in ISI

databases. databases.



One journal's impact factor on its own One journal's impact factor on its own doesn't mean muchdoesn't mean much. Instead, it's important . Instead, it's important

to look at impact factors of multiple to look at impact factors of multiple journals in the same subject area. This journals in the same subject area. This way, one can determine if the impact way, one can determine if the impact

factor of the journal of interest is high or factor of the journal of interest is high or low compared to other journals in a low compared to other journals in a

subject area. subject area.



Using the Impact Factor Using the Impact Factor WiselyWisely

The Institute for Scientific The Institute for Scientific Information® (ISI®) does not depend Information® (ISI®) does not depend on the impact factor alone in on the impact factor alone in assessing the usefulness of a journal, assessing the usefulness of a journal, and neither should anyone else. The and neither should anyone else. The impact factor should not be used impact factor should not be used without careful attention to the many without careful attention to the many phenomena that influence citation phenomena that influence citation rates, rates,


Using the Impact Factor Using the Impact Factor WiselyWisely

The impact factor should be used with The impact factor should be used with informed peer review. In the case of informed peer review. In the case of academic evaluation for tenure it is academic evaluation for tenure it is sometimes inappropriate to use the sometimes inappropriate to use the impact of the source journal to estimate impact of the source journal to estimate the expected frequency of a recently the expected frequency of a recently published article. Again, the impact factor published article. Again, the impact factor should be used with informed peer review. should be used with informed peer review. Citation frequencies for individual articles Citation frequencies for individual articles are quite varied.are quite varied. ..


Factors that Influence Impact Factors 1- Factors that Influence Impact Factors 1-

Date of PublicationDate of Publication

The impact factor is based solely on The impact factor is based solely on citation data and only looks at the citation data and only looks at the citation frequency of articles from a citation frequency of articles from a journal in their first couple years of journal in their first couple years of publication. Journals with articles that publication. Journals with articles that are steadily cited for a long period of are steadily cited for a long period of time (say, 10 years) rather than only time (say, 10 years) rather than only immediately lose out with this immediately lose out with this calculation. calculation.



Factors that Influence Impact FactorsFactors that Influence Impact Factors

Large vs. Small JournalsLarge vs. Small Journals

Large and small journals are Large and small journals are compared equally. Large journals compared equally. Large journals tend to have higher impact tend to have higher impact factors--nothing to do with their factors--nothing to do with their qualityquality


Factors that Influence Impact Factors Factors that Influence Impact Factors

Average CitationAverage Citation It’s important to remember that the It’s important to remember that the

impact factor only looks at an average impact factor only looks at an average citation and that a journal may have a citation and that a journal may have a few highly cited papers that greatly few highly cited papers that greatly increase its impact factor, while other increase its impact factor, while other papers in that same journal may not be papers in that same journal may not be cited at all. Therefore, there is no direct cited at all. Therefore, there is no direct correlation between an individual correlation between an individual article’s citation frequency or quality article’s citation frequency or quality and the journal impact factor. and the journal impact factor.



Review ArticlesReview Articles Impact factors are calculated using Impact factors are calculated using

citations not only from research articles citations not only from research articles but also review articles (which tend to but also review articles (which tend to receive more citations), editorials, receive more citations), editorials, letters, meeting abstracts, and notes. letters, meeting abstracts, and notes. The inclusion of these publications The inclusion of these publications provides the opportunity for editors provides the opportunity for editors and publishers to manipulate the ratio and publishers to manipulate the ratio used to calculate impact factor and used to calculate impact factor and falsely try to increase their number. falsely try to increase their number.



Changing / Growing FieldsChanging / Growing Fields

Rapidly changing and growing fields Rapidly changing and growing fields (e.g. biochemistry and molecular (e.g. biochemistry and molecular biology) have much higher biology) have much higher immediate citation rates, so those immediate citation rates, so those journals will always have higher journals will always have higher impact factors than nursing, for impact factors than nursing, for instance. instance.


Factors that Influence Impact Factors Factors that Influence Impact Factors ISI's ISI's

Indexing / Citation FocusIndexing / Citation Focus There is unequal depth of coverage in There is unequal depth of coverage in

different disciplines. In the health different disciplines. In the health sciences, the Institute for Scientific sciences, the Institute for Scientific Information (ISI), the company which Information (ISI), the company which publishes impact factors, has focused publishes impact factors, has focused much of their attention on indexing and much of their attention on indexing and citation data from journals in clinical citation data from journals in clinical medicine and biomedical research and medicine and biomedical research and has not focused on nursing as much. has not focused on nursing as much.



Research vs. Clinical JournalsResearch vs. Clinical Journals In some disciplines such as some In some disciplines such as some

areas of clinical medicine where areas of clinical medicine where there is not a distinct separation there is not a distinct separation between clinical/practitioner versus between clinical/practitioner versus research journals, research journals research journals, research journals tend to have higher citation rates. tend to have higher citation rates. This may also apply to nursing. This may also apply to nursing.





Journal impact factorJournal impact factor ImpactitisImpactitis

P J Van Diest, H Holzel, D Burnett, and J CrockerP J Van Diest, H Holzel, D Burnett, and J Crocker

Impactitis: new cures for an old Impactitis: new cures for an old diseasedisease

J. Clin. Pathol., November 1, 2001; 54(11): 817 - 819. J. Clin. Pathol., November 1, 2001; 54(11): 817 - 819.


Proper and improper uses of Proper and improper uses of impact factorimpact factor

Evaluating individual scientistsEvaluating individual scientistsAwarding higher academic positionsAwarding higher academic positionsEvaluating research groupsEvaluating research groupsEvaluating institutionsEvaluating institutionsResource allocationResource allocationEvaluating journalsEvaluating journals


Some examplesSome examples

In In JapanJapan people walk around boasting people walk around boasting of their own individual impact factorof their own individual impact factor

In In the UKthe UK (and elsewhere) university (and elsewhere) university funding is awarded on the basis of funding is awarded on the basis of impact factors (research assessment impact factors (research assessment exercise)exercise)

DeanDeans instruct authors to publish in s instruct authors to publish in highest impact factors, though may not highest impact factors, though may not be the best fitbe the best fit


1 What is wrong with impact 1 What is wrong with impact factors?factors?

IF not statistically representative of of IF not statistically representative of of individual journal articlesindividual journal articles

IF correlates poorly with actual IF correlates poorly with actual citations of individual articlescitations of individual articles

Authors not only use IF when Authors not only use IF when submitting to journalssubmitting to journals

The ISI database is imperfect and The ISI database is imperfect and contains citations to “non-citable” contains citations to “non-citable” articlesarticles

Per Seglen BMJ 1997;314:497Per Seglen BMJ 1997;314:497



Self citations are allowedSelf citations are allowedReview articles are heavily cited and Review articles are heavily cited and

inflate the impact factorinflate the impact factorLong articles collect many citationsLong articles collect many citationsShort publication lag allows short Short publication lag allows short

term journal self citationsterm journal self citationsSame language citations are Same language citations are

preferred by authorspreferred by authors



Selective journal self citationSelective journal self citation Coverage of the database is not complete Coverage of the database is not complete

(3,200 out of 126,000 journals)(3,200 out of 126,000 journals) Database has an English language biasDatabase has an English language bias Database is dominated by American Database is dominated by American

publicationspublications Journals in database may vary from year to Journals in database may vary from year to

yearyear IF is a function of the number of articles in IF is a function of the number of articles in

the research fieldthe research field



Research fields with literature that rapidly Research fields with literature that rapidly becomes obsolete are favouredbecomes obsolete are favoured

Impact factor depends on expansion or Impact factor depends on expansion or contraction of research fieldcontraction of research field

Small research fields lack journals with Small research fields lack journals with high impacthigh impact

Relations between fields (eg clinical v Relations between fields (eg clinical v basic science) strongly determine IFbasic science) strongly determine IF

Citation rate of article determines journal Citation rate of article determines journal impact, but not vice versaimpact, but not vice versa


Why the IF Why the IF of journals of journals should should

notnot be used for be used for evaluating researchevaluating research


What are the alternatives?What are the alternatives?


IF-VariantIF-Variant


WebmetricsWebmetrics


Google Scholar (Beta)Google Scholar (Beta)

Google Scholar (Beta) Cited Google Scholar (Beta) Cited ReferencesReferences

(GS) covers peer-reviewed papers, (GS) covers peer-reviewed papers, theses, books, abstracts, and other theses, books, abstracts, and other scholarly literature from all broad scholarly literature from all broad areas of research and from a wide areas of research and from a wide variety of academic publishers and variety of academic publishers and professional societies, as well as professional societies, as well as scholarly articles available across the scholarly articles available across the web. web.


Google Scholar (Beta) Cited Google Scholar (Beta) Cited ReferencesReferences

Each Each Google ScholarGoogle Scholar search result search result contains bibliographic information, such as contains bibliographic information, such as the title, author names, and source of the title, author names, and source of publication. At the end of the search result publication. At the end of the search result is a “Cited by” link, which will display a list is a “Cited by” link, which will display a list of articles and documents that have cited of articles and documents that have cited the document originally retrieved in the the document originally retrieved in the search. Note that this only includes search. Note that this only includes resources indexed by resources indexed by Google ScholarGoogle Scholar..


P&PP&P


EigenfactorEigenfactor

Ranks journals similar to Ranks journals similar to GoogleGoogle ranking ranking of websites. It uses the structure of the of websites. It uses the structure of the entire network (instead of purely local entire network (instead of purely local citation information) to evaluate the citation information) to evaluate the importance of each journal. importance of each journal.

Measures journal price as well as Measures journal price as well as citation influence. The Cost-citation influence. The Cost-Effectiveness Search orders journals by Effectiveness Search orders journals by a measure of the value of the dollar a measure of the value of the dollar they provide. they provide.



Ranks scholarly journals as well as Ranks scholarly journals as well as newspapers, theses, popular magazines, etc. newspapers, theses, popular magazines, etc.

Adjusts for citation differences across Adjusts for citation differences across disciplines, allowing for better comparison disciplines, allowing for better comparison across research areas. across research areas.

Calculations are based on the citations Calculations are based on the citations received over a 5-year period vs. 2 years in received over a 5-year period vs. 2 years in JCRJCR. .

Available free of charge on the web. [Available free of charge on the web. [eigenfactoreigenfactor.org, 2007.org, 2007] ]




H-IndexH-Index


H-IndexH-Index

The h-index quantifies the actual scientific The h-index quantifies the actual scientific productivity and the apparent impact of productivity and the apparent impact of the scientist. The h-index is based on the the scientist. The h-index is based on the author’s most cited papers and the author’s most cited papers and the number of citations they have received number of citations they have received from other articles.from other articles.

"A scientist has index h if h of his/her Np "A scientist has index h if h of his/her Np papers have at least h citations each, and papers have at least h citations each, and the other (Np − h) papers have no more the other (Np − h) papers have no more than h citations each."than h citations each."


H-IndexH-IndexAn h-index of 16 means, for example, An h-index of 16 means, for example,

that a researcher has published 16 that a researcher has published 16 papers that each had at least 16 papers that each had at least 16 citations. Therefore, the h-index citations. Therefore, the h-index reflects both the number of articles reflects both the number of articles as well as the number of citations per as well as the number of citations per article.article.


H-IndexH-Index







More & More about IF More & More about IF


ISIISI


H-IndexH-Index

GS Google search engine~~~~~~~GS Google search engine~~~~~~~Scopus Database SJRScopus Database SJR


H-index of Garfielf EH-index of Garfielf E The Founder of IF The Founder of IF

Eugene Garfield, PhD Eugene Garfield, PhD

Dr. Garfield is Chairman Emeritus, Dr. Garfield is Chairman Emeritus, Institute for Scientific Information, ISI Institute for Scientific Information, ISI

Philadelphia. Philadelphia.


Eugene Garfield, PhDEugene Garfield, PhD


H Index Garfield EH Index Garfield E


Zewail AHZewail AH

. .


H-Index Zewail H-Index Zewail


GSGSof of

Ahmed-RefatAhmed-Refat



H-indexH-indexof of

Ahmed-RefatAhmed-Refat


H index Ahmed-RefatH index Ahmed-Refat


Any Any Question ????Question ????

Thank Thank YouYou


Education

What is wrong with Journals' Impact Factor