Upload
gali-halevi
View
936
Download
0
Embed Size (px)
Citation preview
CiteSearch: Multi-faceted Fusion Approach to Citation Analysis
Kiduk Yang and Lokman Meho
Web Information Discovery Integrated Tool LaboratoryKeimyung University, Korea
American University of Beirut, LebanonOctober 27, 2010
2
CiteSearch: What, Why, & How
Goal• Quality Assessment of Scholarly Publications
Motivation• Lack of comprehensive citation database• Limitations of conventional citation analysis
One-dimensional assessment Misleading evaluation
Approach• Multi-faceted, Fusion-based Citation Analysis
Combine data from multiple citation databases Assess quality using various quality evaluation measures
3
CiteSearch Study: Overview Objectives
• Investigate current citation analysis environment • Test the viability of CiteSearch system
Method• Search citation databases and compare the results
Setup• Study sample
Publications of 15 SLIS faculty members (approx. 1,100 publications)
• Databases used Google Scholar, Scopus, Web of Science
• Citation sources Journals and conference papers in 1996-2005
44
Citation Databases
Web of Science Scopus Google Scholar
Breadth of coverage
36M records
8,700 titlesJournals (240 open access) & conference papers
28M records
15,000 titles Journals (500 open access) & conference papers
500M records
Unknown30+ document types
Coverage years A&HCI: 1975-
SCI: 1900-
SSCI: 1956-
1996-present (with cited references)
1966-present (without cited references)
Unknown
Subject area All All All
• Data collection- WoS: 100 hours- Scopus: 200 hours- GS: over 3,000 hours
Data as of 2006
5
Scopus and WoS: Citation Count Scopus vs. WoS
• 14.0% (278) more citations by Scopus More comprehensive coverage by Scopus (15,000 vs. 8,700 periodicals)
Scopus + WoS• Scopus increases WoS citations by 35% (710)• WoS increases Scopus citations by 19.0% (432)• Relatively low overlap (58%) and high uniqueness (42%)
Scopus(2,301)
Web of Science(2,023)
58%(1,591)
26%(710)
16%(432)
Scopus WoS(2,733)
6
Impact of Scopus By Research Area- varies significantly between research areas- varies significantly between research areas
7
Impact of Scopus on Faculty Members Relative Ranking
Scopus significantly alters the relative ranking of those faculty members that appear in the middle of the rankings
8
Scopus + WoS: Citation Count By Document Type
Scopus(359)
WoS(229)
18%(92)
54%(267)
Scopus WoS(496)
28%(137)
Conference Papers Only
9
Scopus + WoS: Summary of Results Coverage
• Varies greatly between research areas Increase in citations ranges from 5% to 99% by combining results from
both databases • Scopus has a much better coverage of conference proceedings
Overlap: 18% Scopus only: 54% WoS only: 28%
Ranking by citation count• Relative ranking of faculty members changes significantly for those in
the middle
10
Google Scholar Citations By Document Type
11
Citations By Language
12
Impact of GS By Research Area
13
Impact of GS on Faculty Members Relative Ranking
GS does not significantly alter the rankings of faculty members
14
GS vs. ScopusWoS GS increases WoSScopus citations by 93% (2,552) ScopusWoS increases GS citations by 26% (1,104) GS identifies 53% (or 1,448) more citations than WoSScopus GS has much better coverage of conference proceedings
• (1,849 by GS vs. 496 by ScopusWoS) GS has over twice as many unique citations as ScopusWoS
• (2,552 vs. 1,104, respectively)
Google Scholar(4,181)
ScopusWoS(2,733)
31%(1,629)
48%(2,552)
21%(1,104)
GS ScopusWoS(5,285)
15
CiteSearch Study: GS + Scopus + WoS
Google Scholar(4203)
4.3%(230)
18.3%(970)
48.3%(2561)
GS Scopus WoS(5307)
Scopus(2308)
WoS(2025)
11.7%(617)
8.2%(435)
3.8%(204)
5.3%(282)
16
GS + ScopusWoS: Summary of Results Coverage
• Varies greatly between research areas 23% to 144% increase by combining GS & ScopusWoS 5% to 98% increase by combining Scopus & WoS
• GS has strong coverage in CS & IS HCI, IR, computational linguistics, social informatics
• ScopusWoS has stronger coverage in LS Bibliometrics, collection development, information policy
• GS provides significantly better coverage of non-English materials GS (7%); Scopus (1%); WoS (1%)
Ranking• No significant changes in relative ranking of faculty members
17
Findings Scopus, WoS, and GS complement rather than replace each other
GS can be useful in showing evidence of broader international impact than could possibly be done through Scopus and WoS
GS can be very useful for citation searching purposes; however, it is not conducive for large-scale comparative citation analyses
Scopus significantly alters the relative citation ranking of scholars as measured by Web of Science. GS does not
18
Conclusions Multiple sources of citations should be used to generate accurate
citation counts and rankings• Citation databases complement one another • Small overlap between sources may significantly influence relative ranking
Multi-faceted citation analysis is needed
• citation coverage varies by research area, document type, language
CiteSearch can greatly facilitate citation analysis• Enormous effort is required to
Refine search strategy Parse search results Eliminate noise (duplicate citations) Extract & normalize citation metadata
19
CiteSearch System: Work-in-Progress Federated Citation Search
• To compile comprehensive & usable citation data
1. Query multiple citation databases2. Filter out noise
• e.g., invalid, duplicate citations3. Extract & normalize metadata
• bibliographical metadata (e.g., title, author, year, source, etc.)• citation metadata (e.g., doctype, subject, language, etc.)
Multi-faceted Citation Analysis• To produce multi-faceted quality/impact assessment measures that
account for variance in citation quality (e.g., Weighted citation counts, CiteRank) consider various facets of evaluation metric (e.g., Document type, language) accommodate diffent aspects of quality assessment (e.g., H-Index, Mentor-Index)
1. Compute citation-based quality scores (CQS) for each publication2. Compute CQS for authors, schools, publishers using publication CQS3. Compute CQS for each publication weighted by author/school/publisher scores4. Compute CQS for authors, schools, publishers using weighted publication CQS5. Repeat steps 3 and 4 until convergence
20
CiteSearch System: Architecture
21
End
22
23