Transcript
  • Middlesex University Research RepositoryAn open access repository of

    Middlesex University research

    http://eprints.mdx.ac.uk

    Harzing, Anne-Wil ORCID: https://orcid.org/0000-0003-1509-3003 (2016) Microsoft Academic(Search): a Phoenix arisen from the ashes? Scientometrics, 108 (3) . pp. 1637-1647. ISSN

    0138-9130 [Article] (doi:10.1007/s11192-016-2026-y)

    Final accepted version (with author’s formatting)

    This version is available at: http://eprints.mdx.ac.uk/19984/

    Copyright:

    Middlesex University Research Repository makes the University’s research available electronically.

    Copyright and moral rights to this work are retained by the author and/or other copyright ownersunless otherwise stated. The work is supplied on the understanding that any use for commercial gainis strictly forbidden. A copy may be downloaded for personal, non-commercial, research or studywithout prior permission and without charge.

    Works, including theses and research projects, may not be reproduced in any format or medium, orextensive quotations taken from them, or their content changed in any way, without first obtainingpermission in writing from the copyright holder(s). They may not be sold or exploited commercially inany format or medium without the prior written permission of the copyright holder(s).

    Full bibliographic details must be given when referring to, or quoting from full items including theauthor’s name, the title of the work, publication details where relevant (place, publisher, date), pag-ination, and for theses or dissertations the awarding institution, the degree type awarded, and thedate of the award.

    If you believe that any material held in the repository infringes copyright law, please contact theRepository Team at Middlesex University via the following email address:

    [email protected]

    The item will be removed from the repository while any claim is being investigated.

    See also repository copyright: re-use policy: http://eprints.mdx.ac.uk/policies.html#copy

    http://eprints.mdx.ac.ukhttp://eprints.mdx.ac.uk/19984/mailto:[email protected]://eprints.mdx.ac.uk/policies.html#copy

  • Microsoft Academic (Search): a Phoenix arisen from the ashes? Anne-Wil Harzing Version June 2016 Accepted for Scientometrics Copyright © 2016, Anne-Wil Harzing All rights reserved. Prof. Anne-Wil Harzing Middlesex University The Burroughs, Hendon London NW4 4BT Email: [email protected] Web: www.harzing.com

  • 1

    MicrosoftAcademic(Search):aPhoenixarisenfromtheashes?

    ANNE-WILHARZINGMiddlesexUniversity

    TheBurroughs,Hendon,LondonNW44BTEmail:[email protected]:www.harzing.com

    AbstractIncomparisontothemanydozensofarticlesreviewingandcomparing(coverageof)theWebofScience,Scopus,andGoogleScholar,thebibliometricresearchcommunityhaspaidverylittleattentiontoMicrosoftAcademicSearch(MAS).Animportantreasonforthebibliometriccommunity’slackofenthusiasmmighthavebeenthatMAScoveragewasfairlylimited,andthatalmostnonewcoveragehadbeenaddedsince2012.Recently,however,Microsoftintroducedanewservice–MicrosoftAcademic–builtoncontentthatsearchengineBingcrawlsfromtheweb.ThisarticleassessesMicrosoftAcademiccoveragethroughadetailedcomparisonofthepublicationandcitationrecordofasingleacademicforeachthefourmaincitationdatabases:GoogleScholar,MicrosoftAcademic,theWebofScience,andScopus.Overall,thisfirstsmall-scalecasestudysuggeststhatthenewincarnationofMicrosoftAcademicpresentsuswithanexcellentalternativeforcitationanalysis.Ifourfindingscanbeconfirmedbylarger-scalestudies,MicrosoftAcademicmightwellturnouttocombinetheadvantagesofbroadercoverage,asdisplayedbyGoogleScholar,withtheadvantageofamorestructuredapproachtodatapresentation,typicalofScopusandtheWebofScience.Ifso,thenewMicrosoftAcademicservicewouldtrulybeaPhoenixarisenfromtheashes.

  • 2

    MicrosoftAcademic(Search):aPhoenixarisenfromtheashes?

    IntroIncomparisontothemanydozensofarticlesreviewingandcomparing(coverageof)theWebofScience,Scopus,andGoogleScholar(forthelatestseee.g.Delgado-López-Cózar&Repiso-Caballero,2013,Wildgaard,2015,Harzing&Alakangas,2016),thebibliometricresearchcommunityhaspaidverylittleattentiontoMicrosoftAcademicSearch.AGoogleScholarsearchforjournalarticleswithMicrosoftAcademicSearch(MAS)inthetitleprovidesonly5results.ThesamesearchforGoogleScholar,theWebofScience,orScopusprovidesmanyhundredsofjournalarticlesforeachdatabase.Thisisquitesurprisinggiventhatin2014NaturereporterRichardNoorden,afrequentcommentatoronbibliometricdevelopments,wrote:“Afewyearsago,MicrosoftAcademicSearch(MAS)wasvyingwithGoogleScholartobetheweb’spre-eminentfreescholarlysearchengine.Bothproductsindexedtensofmillionsofscholarlydocuments,trackedtheircitations,andmadeprofilepagesforacademics.[…]Thestagewassetforbibliometricbattle.” Jacso(2011)wasthefirsttowriteaboutMAS,providingareviewofthemajorcontentandsoftwarefeaturesanditsshortcomings.Hisverdictwas:“thisfreebibliometricserviceisaprojectofgreatinteresttothoseinterestedinmetrics-basedresearchperformanceevaluation”(Jacso,2011:983].Surprisingly,afullthreeyearspassedwithoutanyarticlesdealingwithMASuntilOrtegapublishedtwoarticlesin2014.Thefirstcompared771authorprofilesbetweenGoogleScholarCitationsandMAS(Ortega&Aguilo,2014)andconcludedthatGoogleScholarreportedmorepublicationsandcitations.Thesecond(Ortega,2014)usedMAStostudyco-authornetworksandhighlyrecommendedMASforcollaborationstudies,providedproblemswithduplicateprofilesandinfrequentupdatingcouldberesolved. SowhyhasthebibliometriccommunityalmostcompletelyignoredMAS?Oneofthereasonsmighthavebeenthatitsnativeinterfacewasnotverysuitableforcitationanalysis.However,thesameistrueforGoogleScholarandbibliometricresearchershaveturnedenmassetoPublishorPerish(2007)todobibliometricresearchwithGoogleScholar.PublishorPerishhasincludedasearchoptionforMASsince2013,whichwasusedbyHaley(2014)tocompareEconomics&FinancejournalsinGoogleScholarandMAS.HaleyfoundcitationslevelstobesubstantiallyhigherinGoogleScholarthaninMAS,withthemeanh-indexroughlytwiceashigh.Rankcorrelations,however,werefoundtobeveryhigh. The–todate–lastarticlepublishedonMASmightexplainthebibliometriccommunity’slackofenthusiasm.Orduña-Malea,Martín-Martín,Ayllon,&DelgadoLopez-Cozar(2014)publishedacomprehensiveanalysisofMAScoverageandshowed–asmanyusershadnodoubtnoticedthroughincidentalsearches–thatalmostnonewcoveragehadbeenaddedsince2012.However,fastforwardtwoyearsandMicrosoftAcademicSearchhasarisenfromtheasheswithanewservice–MicrosoftAcademic–builtoncontentthatsearchengineBingcrawlsfromtheweb,includingpublisherwebsites,universityrepositories,researcher,anddepartmentalwebpages.Citationcountsarethesumsofthereferencelinksbetweenthepapers.

  • 3

    However,thebigquestionthatwillburnonbibliometricians’mindsis:Isitscoverageanybetterthanitspreviousincarnation?Thisarticleprovidesafirstattempttoanswerthisquestionthroughacomparisonofthepublicationandcitationrecordofasingleacademicforeachofthefourmaincitationdatabases:GoogleScholar,MicrosoftAcademic,theWebofScience,andScopus.

    AnindividualacademicrecordasacasestudyInordertoassessthecoverageofthenewMicrosoftAcademicincomparisontoGoogleScholar,Scopus,andtheWebofScience,Iconductedadetailedanalysisofmyownpublicationrecord.AlthoughthisisobviouslyalimitedtestofMicrosoftAcademicasanewdatasourceforcitationanalysis,thereareseveralreasonswhyIthinkmyownpublicationrecordpresentsanappropriatetest.

    First,itincludesenoughpublications–varyingfrom47intheWebofScienceto124inGoogleScholar–toavoididiosyncraticresults.Inaddition,withover10,000GoogleScholarcitationsandrelativelyfewpublicationswithoutcitations(generally2016publicationsandconferencepapers),citationlevelsarealsohighenoughtoavoididiosyncraticresults.

    Second,covering22years(1995-2016),itincludesbotholderandyoungerpublications,includingsomepapersonlyavailableinonlinefirst.ThisshouldallowustoassesstowhatextentMicrosoftAcademiccoversolderpublicationsaswellasveryrecentones.

    Third,itincludesawidevarietyofpublications.Lookingatthe124GoogleScholarpublications,47areinjournalsthatcouldbeconsideredtobemainstreamintheirfield,22areinsecondaryjournals,20arebookchapters,15areconferencepapers,12arewhitepaperspublishedonlyonmywebsite,3arebooks,and3non-refereedpublications(2newsletterarticles,1companyreport).Thetwofinalonesareajournalrankingavailableonlyonmywebsite(TheJournalQualityList)andasoftwareprogram(PublishorPerish).ThisvarietyshouldallowustoassesstheextenttowhichMicrosoftAcademiccoversnon-traditionalpublications.

    Fourth,virtuallyallofmyacademicpublicationsareincludedinGoogleScholar,includingallofmyjournalarticles,booksandbookchapters,aswellas12ofmy14whitepapers.TheonlytwowhitepapersthatarenotlistedinGoogleScholarrelatetoteaching(“Writingcourseworkassignments”and“Howtoaddressyourteacher”).NotallofmyconferencepapersarelistedinGoogleScholar,butthisisonlynatural,asmanyofthemneverappearedonline.Hence,myGoogleScholarpublicationrecordprovidesanexcellentbaselineforourcomparisonacrossdatabases.

    DatacollectionAlldatawerecollectedonthe16thofMay2016.SearchesforGoogleScholarandMicrosoftAcademicwereconductedwithPublishorPerish(2007).PublishorPerishisusedprimarilyinconjunctionwithGoogleScholar,buthasrecentlyimplementedexperimentalMicrosoftAcademicsupport.Italsooffersextensivedataimportfacilities,providingtheabilitytoimportamongstothersScopusandWebofSciencedata.SearchesforScopusandtheWebofSciencewerethusconductedintheirnativeinterfaces,exportedandsubsequentlyimportedintoPublishorPerishtoallowforcalculationofthevariouscitationmetrics.ResultsforallfourdatabasesweresubsequentlyexportedtoExcel,allowingforone-on-onematchingofpublicationsandcomparisonofcitationscounts.

  • 4

    Onlypublicationswithsubstantiveacademiccontentwereincludedinourcomparison.Thismeansthatweexcludedbookreviews,errataandcorrigendaforallfourdatabases.Straypublications,i.e.publicationsreferringtothesamemasterrecordwithslightlydifferentbibliographicdetailsweremergedintotheirmasterrecordforbothGoogleScholarandMicrosoftAcademic.Obviousparsingerrors,suchaslistsofreviewers,werealsoexcluded,aswerepublicationsbyotherauthorsinmyeditedtextbook.TherewerefarmorestraypublicationsandparsingerrorsforGoogleScholarthanforMicrosoftAcademic.

    MicrosoftAcademiconlydisplayedtwoclearparsingerrors.Inbothcasesauthorsofonepublicationwerecombinedwithpublicationdetailsofanother.Inaddition,therewereabouttenincongruousstraypublicationscreatedbypickinguppre-publicationversionswithadifferenttitleorpublicationsfromtwodifferentsources;noneofthesehadanycitations.AspecialcategoryofstraypublicationsinMicrosoftAcademicconcernedfivearticleswherecitationsweresplitbetweenaversionwiththemaintitleonly,andaversionwithboththemaintitleandasub-title.Inaddition,thereweretwoarticleswherecitationsweresplitbetweentwoversionsofthedocument,becausetheseparatorbetweenmainandsub-titlewasprocessedindifferentways.Forinstanceaquestionmarkwasvariouslyreplacedby|[quest]|andasingleletterq.

    Finally,wediscoveredoneotherproblemwithMicrosoftAcademicthatwouldneedtobefixedbeforeanymetricsbasedontheyearofpublicationcouldbeused:thefactthatthedatabaseindicatedthewrongpublicationyearforsomepapers,eventhoughthecorrectjournalvolumewaslisted.IncorrectyearallocationsarebynomeansuncommoninGoogleScholareither.Infact,sevenofmy124publicationswereallocatedthewrongpublicationyearinGoogleScholar,twobecauseofinexplicableparsingerrors(thesourcedocumentdisplayedthecorrectyear)andfivebecauseGoogleScholarusedapre-publicationversionasitsmasterrecord.However,astheseincorrectyearallocationswereonlyyearoneyear“out”,thisisnotgenerallyamajorproblem.

    However,incorrectyearallocationsweremorefrequentinMicrosoftAcademic:nolessthaneighteenoutofmy89publicationshadthewrongpublicationyear.Outofthese,onewasaninexplicableparsingerrorofafairlyobscurebookchapterandoneoccurredasMicrosoftAcademicuseda2012reprintinaRomanianjournalasthesourcefora2008whitepaper.JustlikeGoogleScholar,MicrosoftAcademicincorrectyearallocationsalsooccurredbecauseofusingthepre-publicationoronlinefirstversionasasourcerecord(sevenoccurrencesintotal).Inallthesecasesthepublicationyearwasonlyoneyearout,whichisunlikelytocausemajorproblems.Amoredisturbingproblemwasthefactthatnineofmypublicationscarriedthewrongyearinspiteofreferringtoasourcedocumentwiththecorrectyear.Inthiscase,thepublicationyearwasoften“wayout”(tenyearsormoreinthreecases).AllninerecordsconcernedjournalspublishedbyeitherEmeraldorTaylor&Francis,withthefiveEmeraldrecordsallbeingallocateda2013publicationyear(withactualpublicationyearsvaryingbetween2001and2012).Hence,itwouldappearthatthereisaparsingproblemwiththesetwospecificpublishers’websites,whichwillhopefullyberesolvedsoon.

    ResultsFigures1and2visuallydisplaythecomparativecoverageofthefourdatabaseswithregardtopublicationsandcitations.Forbothcases,wewillfirstdiscusstheoverlapincoverageacrossthedatabasesandthenlookatthepublicationsandcitationsuniquetoeachofthefourdatabases.AsourinterestinthisarticleisinMicrosoftAcademiccoverage,wedonotprovideacomparisonbetweenGoogleScholarontheonehandand

  • 5

    theWebofScienceandScopusontheotherhand,orbetweentheWebofScienceandScopus.Therearemanypublicationsthathavealreadydonesointhepast,includingmostrecentlyHarzing&Alakangas(2016).Figure1:Comparingpublicationcoverageacrossfourdata-bases

    Publications:overlapbetweenthefourdatabasesAsindicatedabove,Ihave124uniquepublicationsinGoogleScholar.Ofthese,89werealsopresentinMicrosoftAcademicSearch;thisincludedall69ofmyjournalpublications;allthreebooks,sevenofthefifteenconferencepapers,sevenofthetwentybookchapters,oneofthewhitepapersandbothofthenewsletterarticles. Ofthe89publicationslistedinMicrosoftAcademic,only46werelistedintheWebofScience.Allofthesewerejournalarticles.Thisincluded40ofthe47publicationsinmainstreamjournals,butonly6ofthe22publicationsinsecondaryjournals. Ofthe89publicationslistedinMicrosoftAcademic,only59werelistedinScopus.Allbutthreeofthesewerejournalarticles.Thisincluded44ofthe47publicationsinmainstreamjournals(includingtwoin-pressarticles)and12ofthe22publicationsinsecondaryjournals.ScopusalsocoveredtwoconferencepaperspublishedintheAcademyofManagement’sbestpapersproceedingsandonebookchapterintheseriesProgressinInternationalBusinessResearch(Emeraldpublishers),ayearlyresearchannualwithselectedpaperspresentedattheEuropeanInternationalBusinessAcademyconference.

    GoogleScholar WebofScienceMicrosoftAcademic

    Scopus

    89

    59

    46

    B1:0

    B2:43

    B3:30

    A1:35 A2:1

    A3:2

  • 6

    ConclusionIncomparisontotheWebofScienceandScopus,MicrosoftAcademiccoversafarlargernumberofpublicationsthatarelistedinGoogleScholarand–importantly–coversalljournalpublicationsandbooksthatarealsocoveredinGoogleScholar.ThissuggeststhatMicrosoftAcademichasexcellentcoverageofwhatareusuallyconsideredtobethemostimportantacademicoutputs:journalarticlesandbooks.

    Publications:uniquecoverageinthefourdatabases

    MicrosoftAcademiccomparedwithGoogleScholarTherearenopublicationscoveredinMicrosoftAcademicthatarenotcoveredinGoogleScholar(B1=0).GoogleScholarincluded35publicationsthatwerenotincludedinMicrosoftAcademic(A1=35).Asindicatedabove,MicrosoftAcademicincludedalljournalsarticlesandbooksinourcasestudy.Hencethe35publicationsuniquetoGoogleScholarwerebookchapters(13),whitepapers(11),conferencepapers(8),aweb-basedjournalranking(theJournalQualityList),asoftwareproduct(PublishorPerish),andacompanyreport. Fornearlyhalfofthesepublications(17publications),GoogleScholarrecordsareofthe“[citation]”type,indicatingthatalthoughGoogleScholarfoundcitationstothesepublications,itwasnotabletofindtheoriginalpublication.Elevenpublicationswerefoundontheauthor’spersonalacademicwebsite,threeonGoogleBooks,threeinonlineconferenceproceedings,andoneonthewebsiteofEmeraldpublishing. AsMicrosoftAcademicdidfindsevenofthebookchapters,sevenoftheconferencepapersandoneofthewhitepapers,wetriedtoestablishwhethertheydifferedinanywayfromtheonesthatwereonlylistedinGoogleScholar.ThiswaseasyforthesolewhitepaperasMicrosoftAcademicactuallyfoundareprintofthiswhitepaperinaRomanianjournal.Ofthesevenbookchapters,fourweresourcedfrompre-publicationversionsattheauthor’swebsite,onefromResearchgate,onefromaninstitutionalrepository,andonedidn’thaveasourceitem.Ofthesevenconferencepapers,fourweresourcedfromtheAcademyofManagementproceedings,onefromapre-publicationversionattheauthor’swebsiteandtwofromuniversityrepositories.Itisunclearwhysomebookchaptersandconferencepapersavailableaspre-publicationontheauthor’swebsiteweresourcedbyMicrosoftAcademicandotherswerenot.

    MicrosoftAcademiccomparedwithWebofScienceIntotal,thereare43publicationscoveredinMicrosoftAcademicthatarenotcoveredinISI(B2=43).MicrosoftAcademiccoveredtwentynon-journalpublications(books,bookchapters,conferencepapers,whitepapers,andnewsletterarticles)thatwerenotincludedintheWebofScience.

    However,sevenofthearticlespublishedinmainstreamjournalsincludedinMicrosoftAcademicwerenotincludedintheWebofScienceeither.Forthreeofthose,thiswascausedbythefactthatthepublicationswereeitheravailableonlyinonlinefirst(two)orwererecentlypublished,butnotyetenteredintotheWebofSciencedatabase.TheremainingfourjournalarticlesuniquetoMicrosoftAcademicconcernedpublicationsin1995,1996,1997and2003injournalsthatwerenotISIlistedatthetime,butareincludedintheWebofSciencenow.

    Ofthetwenty-twopublicationsinsecondaryjournalsthatarecoveredinbothGoogleScholarandMicrosoftAcademicSearch,sixteenwerenotlistedintheWebofScienceatthetimethepublicationsappeared.Thesepublicationsrepresenteleven

  • 7

    differentjournalsandallbutoneofthepublicationsoccurredbetween2001and2008.Oftheseelevenjournals,allbutoneiarenowincludedintheWebofScience.

    Incontrast,thereisonlyonepublicationlistedinISIthatisnotlistedinMicrosoftAcademic(A2=1).Thisconcernsabookchapterinaneditedbook,publishedbyRoutledgein2011.

    MicrosoftAcademiccomparedwithScopusIntotalthereare30publicationscoveredinMicrosoftAcademicthatarenotcoveredinScopus(B3=30).ThecomparisonbetweenMicrosoftAcademicandScopusfornon-journalpublicationsissimilarinnaturetothatbetweenMicrosoftAcademicandtheWebofScienceinthatMicrosoftAcademicincludedseventeennon-journalpublicationsthatScopusdidnotcover.

    ThethreeuniquepublicationsinmainstreamjournalsinMicrosoftAcademicincludedtwoarticlespublishedin1995and1996beforetheoriginalstartofScopuscoveragein1996ii.Afinalpublicationin2003waspublishedinajournalthatwasnotlistedinScopusuntil2005.

    Ofthe22publicationsinsecondaryjournalsthatarecoveredinbothGoogleScholarandMicrosoftAcademicSearch,tenwerenotlistedinScopusatthetimethepublicationsappeared.Thesepublicationsrepresenteightdifferentjournalsandallbutoneofthepublicationsoccurredbetween2001and2008.AlleightjournalsarenowincludedinScopus,withScopusadoptionnearlyalwaysoccurringonlyoneortwoyearsaftertherelevantpublicationswerepublished.

    Incontrast,thereareonlytwopublicationslistedinScopusthatarenotlistedinMicrosoftAcademic(A3=2).ThisconcernsthesamebookchapteraslistedintheWebofScience,plusanotherbookchapterinaresearchannualAdvancesinInternationalManagement,publishedbyEmeraldpublishersin2003.

    ConclusionMicrosoftAcademicperformsverywellinourcomparisonofuniquecoverageinthefourdatabases.Ontheonehand,itdoesnotdisplayanyuniquecoveragevis-à-visGoogleScholar,whereasGoogleScholarhas35additionalpublicationsnotcoveredbyMicrosoftAcademic.Ontheotherhand,itdoesdisplayasubstantialuniquecoveragevis-à-visboththeWebofScience(43publications)andScopus(30publications).UniquecoveragefortheWebofScienceandScopusvis-à-visMicrosoftAcademicisminiscule:onebookchapterfortheWebofScienceandtwobookchaptersforScopus.

    Inadditiontomanynon-journalpublications,theuniquecoverageforMicrosoftAcademicincludes23journalarticleswhencomparedtotheWebofScienceand13uniquearticleswhencomparedtoScopus.ItmustbeacknowledgedthatallbutoneoftherelevantjournalsarenowcoveredinboththeWebofScienceandScopus,thusindicatingthattheywerebynomeansobscurejournals.Hence,forveryrecentjournalpublicationstheremightbelittle,ifany,differencebetweenthecoverageofGoogleScholar,MicrosoftAcademic,theWebofScienceandScopus.Thisisoflittlesolace,however,foracademicswith(aninterestin)publicationsthatstretchbackintime.InthosesituationsonlyGoogleScholarandMicrosoftAcademicwillprovidesufficientcoverage.

    Citations:overlapbetweenthefourdatabasesFigure2providesavisualillustrationofboththeoverlapandtheuniquecoverageofthefourdatabasesintermsofthecitationsassociatedwiththerelevantpublications.For

  • 8

    those89publicationsthatoverlapbetweenMicrosoftAcademicandGoogleScholar,GoogleScholarhasmorethan2.5timesasmanycitationsasMicrosoftAcademic.

    PartofthereasonforthisisthatMicrosoftAcademiccitationcountsfornon-journalpublicationsinparticularwerequitemodest.With97citations,onlytheManagementtheMultinationalsbookhadasubstantivenumberofcitations,althoughthiswasstillconsiderablylowerthaninGoogleScholar(433citations).However,forthetwootherbooks,thecomparisonwithGoogleScholarwasevenmoreunfavourable:20vs.203citationsforThePublishorPerishBookand14vs.392citationsfortheInternationalHRMtextbook.Mostoftheseventeenconferencepapers,bookchapters,andnon-refereedpublicationshadeitherzerooronecitationinMicrosoftAcademic.InfactthetotalnumberofcitationsfortheseseventeenpublicationsinMicrosoftAcademicwasonly26.GoogleScholar’scitationslevelfortheseseventeenpublicationswasnotveryhigheither,butat187wasstillseventimesashigh.Figure2:Comparingcitationcoverageacrossfourdata-bases

    Whencomparingcitationsforthe46publicationsthatarelistedinboth

    MicrosoftAcademicandtheWebofScience,wefindthatMicrosoftAcademichasapproximately20%highercitationslevelsoverall.Thisdoesn’tmeanthateveryindividualpublicationshowsthesamepattern.Morethanonethirdofthepublications(17outof46)hasatleast20%morecitationsinMicrosoftAcademic,goingupto94%and170%fortwospecificjournalarticles.Anotherthirdofthepublications(16outof46)hasbetween3%and19%morecitationsorcitationlevelsequaltotheWebof

    Scopus

    MAS:3424

    GS:9099

    MAS:2826Scopus:2861

    MAS:2212

    WoS:1844

    B1:0B2:1210

    B3:596

    A1:1310 A2:0

    A3:85

    GoogleScholarWebofScience

    MicrosoftAcademic

    WebofScience

  • 9

    Science.ThirteenarticleshadfewercitationsinMicrosoftAcademicthanintheWebofScience,butthedifferenceinallcaseswasmarginal,1-3citationsforelevenarticlesand4or5fortheremainingtwo. Whencomparingcitationsforthe59publicationsthatarelistedinbothMicrosoftAcademicandScopus,wefindthatoverallcitationlevelsareverysimilarindeed,withcitationsinMicrosoftAcademicbeinglessthan1%lowerthaninScopus.Thisisreflectedinthearticle-by-articlecomparisonwhereroughlyhalfofthearticleshadmorecitationsinMicrosoftAcademicandhalfhadmorecitationsinScopus.Absolutedifferences,however,werefairlysmall;onlyeightarticlesdifferedbymorethan10citationseitherway,andmorethanhalfofthearticlesdifferedby3citationsatmost.

    ConclusionMicrosoftAcademicperformsverywellintermsofcitationcountsforarticlesthatoverlapwithotherdatabases.ItoutperformstheWebofSciencefornearlyallarticlesandisanequaltoScopus.OnlyGoogleScholarstilloutperformsMicrosoftAcademicinthisrespect.

    Citations:uniquecoverageinthefourdatabasesInadditiontocomparingcitationsforarticlesthatcanbematchedacrossdatabases,itisimportanttoassesstowhatextentuniquearticlesineachdatabasecontributetotheoverallcitationcount.

    MicrosoftAcademiccomparedwithGoogleScholarAstherearenopublicationsuniquetoMicrosoftAcademic,therearenouniquecitationsforMicrosoftAcademicwhencomparedtoGoogleScholar(B1=0).Thereare,however,35uniquepublicationsinGoogleScholarthathaveaccumulated1310citationsintotal(A1=1310).MostofthesecitationscamefromPublishorPerish(521citations)andtwobookchapterspublishedinresearchannuals(189and101)thatwerenotcoveredMicrosoftAcademic.FourfurtherpublicationsuniquetoGoogleScholarwithsignificantcitationlevelsweretheJournalQualitylist(79),threechaptersoninternationalassignmentsinthreedifferenteditionsofmyInternationalHumanResourceManagementbook(67,51and48citations)andaconferencepapercomparingGoogleScholarwiththeWebofScience(46citations).Hence,84%oftheuniquecitationsinGoogleScholarcamefromlessthanaquarteroftheuniquepublications.

    MicrosoftAcademiccomparedwithWebofScienceThereare43uniquepublicationsinMicrosoftAcademicwhencomparedtotheWebofScience,whichhaveaccumulated1210uniquecitations(B2=1210).Mostoftheseuniquecitationscamefromjournalpublications,includingfourfairlyhighlycitedpublications(63-207citations)insecondaryjournals.Morethanathird–generallyeitherconferencepapersorveryrecentlypublishedjournalarticles–ofthe43uniquepublicationshadeithernoorjust1citation.Hence,threequartersoftheuniquecitationsinMicrosoftAcademiccamefromjust16%oftheuniquepublications.

    TheonlyuniquepublicationlistedinISI(abookchapterinaneditedbook)didn’thaveasinglecitation.HencetherearenouniquecitationsinWebofSciencewhencomparedtoMicrosoftAcademic.

  • 10

    MicrosoftAcademiccomparedwithScopusThereare30uniquepublicationsinMicrosoftAcademicwhencomparedtoScopus,whichhaveaccumulated596uniquecitations(B3=596).Mostoftheseuniquecitationscamefromjournalpublications,includingfourfairlyhighlycitedpublications(37-71citations)insecondaryjournals.Athirdofthe30uniquepublications–generallyeitherconferencepapersorbookchapters–hadnocitations.Hence,morethanthreequartersoftheuniquecitationsinMicrosoftAcademiccamefromlessthanaquarteroftheuniquepublications.

    OnlyoneofthetwouniquepublicationslistedinScopus(abookchapterinaresearchannual)hadcitations.Asthisbookchapterwasfairlyhighlycited(A3=85citations),incontrasttotheWebofScience,Scopusdidhaveanon-negligiblenumberofuniquecitationswhencomparedtoMicrosoftAcademic.

    ConclusionMicrosoftAcademicperformsverywellinourcomparisonofuniquecitationsinthefourdatabases.Ontheonehand,itdoesnotdisplayanyuniquecitationsvis-à-visGoogleScholar,whereasGoogleScholarhas1310additionalcitationsnotcoveredbyMicrosoftAcademic.Ontheotherhand,itdoesdisplayasubstantialnumberofuniquecitationsvis-à-visboththeWebofScience(1210citations)andScopus(596citations).UniquecitationsfortheWebofScienceandScopusareeithernon-existent(WebofScience)orrelativelymodest(Scopus).

    MostoftheuniquecitationsinMicrosoftAcademicrelatetojournalarticlesanditmustbeacknowledgedthatuniquecitationsareconcentratedinafairlysmallnumberofuniquepublications.However,theconclusionthatMicrosoftAcademicperformswellincomparisontotheWebofScienceandScopusincitationcoverageaswellaspublicationcoverageisinescapable.

    ConclusionOurdetailedcomparedofcoverageacrossfourdatabasesshowedthatMicrosoftAcademicsignificantlyoutperformstheWebofScienceintermsofbothpublicationandcitationcoverage.MicrosoftAcademiccanalsobeconsideredtobeatleastanequaltoScopusonbothcounts.OnlyGoogleScholaroutperformsMicrosoftAcademicintermsofbothpublicationsandcitations.

    ThebiggestdifferencebetweenGoogleScholarandMicrosoftAcademicliesintwoareas.First,GoogleScholarincludescoverageofnon-standardresearchoutputs,suchasthePublishorPerishsoftware,thusprovidingadditionalcitationsforuniquepublications.Second,GoogleScholarhasmorecitationsforalloftheoverlappingpublications,andsubstantiallymoreinsomecases. WedidfindthattheadditionaljournalcoverageofbothGoogleScholarandMicrosoftAcademicconcernedjournalsthatcurrentlyareincludedinboththeWebofScienceandScopus.Thusdifferencesbetweendatabasesmightbecomesmallerovertime.However,forthoseinterestedinacross-sectionofyoungerandolderpublications,bothGoogleScholarandMicrosoftAcademicappeartobeabetterchoicethantheWebofScienceorScopus. Sowhatdoesthismeanforanindividualacademic?Acomparisonofmyh-indexacrossdatabasesshowsittobemorethantwiceashighinGoogleScholar(46)thanintheWebofScience(22).MicrosoftAcademic(30)andScopus(27)providevaluesinbetweenthesetwoextremes.IntermsofthehIa–anindividualannualizedh-index(seeHarzing,Alakangas&Adams,2014),differencesaresmallerasbothScopusandtheWeb

  • 11

    ofSciencemisscoverageofarangeofolderarticles,thusreducingthenumberofyearssincemyfirstpublication.Asaresult,thevaluesofthehIaforScopus(1.11),MicrosoftAcademic(1.10)andtheWebofScience(1.06)areveryclosetogether.At1.81,thehIainGoogleissubstantiallyhigher. Overall,thisfirstsmall-scalecasestudysuggestthat–providedsometeethingproblemswithregardtopublicationduplicatesandwrongyearallocationscanberesolved–thenewincarnationofMicrosoftAcademicpresentsuswithanexcellentalternativeforcitationanalysis,especiallyifcoverageforbooksandnon-traditionalresearchoutputscouldbefurtherimproved.Ifourfindingscanbeconfirmedbylarger-scalestudies,MicrosoftAcademicmightwellturnouttocombinetheadvantagesofbroadercoverage,asdisplayedbyGoogleScholar,withtheadvantageofamorestructuredapproachtodatapresentationtypicalofScopusandtheWebofScience.Ifso,thenewMicrosoftAcademicservicewouldtrulybeaPhoenixarisenfromtheashes.

    ReferencesButler,D.(2011).Computinggiantslaunchfreesciencemetrics.Nature,476(7358),18-18.Chrysomallis,M.(2014,December8).Scopuscontinuestoaddpre-1996citations[Weblogpost].http://blog.scopus.com/posts/scopus-continues-to-add-pre-1996-citations,accessed21May2016.

    Delgado-López-Cózar,E.,&Repiso-Caballero,R.(2013).Elimpactodelasrevistasdecomunicación:comparandoGoogleScholarMetrics,WebofScienceyScopus.Comunicar:RevistaCientíficadeComunicaciónyEducación,21(41),45-52.Haley,M.R.(2014).RankingtopeconomicsandfinancejournalsusingMicrosoftacademicsearchversusGooglescholar:howdoesthenewpublishorperishoptioncompare?.JournaloftheAssociationforInformationScienceandTechnology,65(5),1079-1084.

    Harzing,A.W.;Alakangas,S.(2016)GoogleScholar,ScopusandtheWebofScience:Alongitudinalandcross-disciplinarycomparison,Scientometrics,106(2):787-804.

    Harzing,A.W.;Alakangas,S.;Adams,D.(2014)hIa:Anindividualannualh-indextoaccommodatedisciplinaryandcareerlengthdifferences,Scientometrics,99(3):811-821.

    Jacsó,P.(2011).TheprosandconsofMicrosoftAcademicSearchfromabibliometricperspective.OnlineInformationReview,35(6),983-997.

    Orduña-Malea,E.,Martín-Martín,A.,M.Ayllon,J.,&DelgadoLopez-Cozar,E.(2014).Thesilentfadingofanacademicsearchengine:thecaseofMicrosoftAcademicSearch.OnlineInformationReview,38(7),936-953.

    Ortega,J.L.(2014).Influenceofco-authorshipnetworksintheresearchimpact:EgonetworkanalysesfromMicrosoftAcademicSearch.JournalofInformetrics,8(3),728-737.

    Ortega,J.L.,&Aguillo,I.F.(2014).MicrosoftacademicsearchandGooglescholarcitations:Comparativeanalysisofauthorprofiles.JournaloftheAssociationforInformationScienceandTechnology,65(6),1149-1156.

    VanNoorden,R.(2014)ThedeclineandfallofMicrosoftAcademicSearch[Weblogpost].http://blogs.nature.com/news/2014/05/the-decline-and-fall-of-microsoft-academic-search.html,accessed21May2016.

    Wildgaard,L.(2015).Acomparisonof17author-levelbibliometricindicatorsforresearchersinAstronomy,EnvironmentalScience,PhilosophyandPublicHealthinWebofScienceandGoogleScholar.Scientometrics,1-34.

    iInterestingly,thisjournalhasbeenlistedinScopussinceitsfirstissuein2001.iiThisislikelytochangeasScopushasrecentlymadeafirmcommitmenttofurtherexpanditscoverageofpre-1996publicationsandcitations(Chrysomallis2014).


Recommended