Embed Size (px)
Citation preview
Scopus Content Curation:Re-evaluation
Shareef Bhailal
Product Manager Scopus Title Evaluation Platform
March 9th, 12th Symposium on Thai Scholarly Journals
Ambassador Hotel, Bangkok, Thailand
| 2| 2| 2
New title suggestions per month
0
100
200
300
400
500
600
700
800
Jan
-10
Mar
-10
May
-10
Jul-
10
Sep
-10
No
v-1
0
Jan
-11
Mar
-11
May
-11
Jul-
11
Sep
-11
No
v-1
1
Jan
-12
Mar
-12
May
-12
Jul-
12
Sep
-12
No
v-1
2
Jan
-13
Mar
-13
May
-13
Jul-
13
Sep
-13
No
v-1
3
Jan
-14
Mar
-14
May
-14
Jul-
14
Sep
-14
No
v-1
4
Jan
-15
Mar
-15
May
-15
Jul-
15
Sep
-15
No
v-1
5
Jan
-16
Mar
-16
May
-16
Jul-
16
Sep
-16
No
v-1
6
Jan
-17
Mar
-17
May
-17
Jul-
17
Sep
-17
Undetermined
Not for review
For review
2011: 3,884 titles suggested, of which 1,394 for review (36% success rate, 116 eligible titles per month)2012: 3,291 titles suggested, of which 1,311 for review (40% success rate, 109 eligible titles per month)2013: 3,094 titles suggested, of which 963 for review (31% success rate, 81 eligible titles per month)2014: 2,755 titles suggested, of which 1,157 for review (42% success rate, 97 eligible titles per month)2015: 2,929 titles suggested, of which 1,544 for freview (52% sucess rate, 129 eligible titles per month)2016: 3,426 titles suggested
2017YTD: 2,748 titles suggested
(Jan 2010 – Oct 2017)
| 3| 3| 3
Scopus Title Evaluation Process
| 4| 4| 4
Scopus Title Evaluation Process: Suggestion
1
2
| 5| 5| 5
Scopus Title Evaluation Process: Validation & Enrichment
| 6| 6| 6
Scopus Title Evaluation Process: Review & Decision
| 7| 7| 7
Involvement of the Local Board*
Pre-selection
Local board review
| 8
Less than half of the reviewed titles are selected for
Scopus coverage
604 90156
156 194150 69 365 50 52 105
4085 55 129 41 75 27 59
96
127 2154
104 131124 58 319 46 52 120
61162 107 251 84
187 68 158
456
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
RejectedAccepted
In total 7,229 titles reviewed (2011 –2016) of which 3,310 (45%) accepted for Scopus
Title review results from top 20 countries with most titles reviewed in the last 6 years:
.
Thailand 23%
| 9| 9| 9
Why: need for content curation is noticed in the market
& by Scopus
Source: https://scholarlykitchen.sspnet.org/2015/06/09/data-curation-the-new-killer-app/
Find more information on Elsevier.com ‘Discontinued Sources List’: https://www.elsevier.com/solutions/scopus/content
| 10| 10| 10
Full transparency
| 11| 11| 11
What: Ongoing content curation of the Scopus base to
ensure continuous high quality content
Identification of poor
performing journals
using metrics and
benchmarks
“Radar” to predict
journals with outlier
performance
Direct feedback from
users and
stakeholders on poor
performing journals
Re-evaluation by the Content Selection & Advisory Board (CSAB)
Content Curation
Curation of the full journal base is essential and expected by our
customers and users.
Review:
Curate:
1 32
| 12| 12| 12
1. How: by using metrics and benchmarks to identify
poor-performing journals for re-evaluation
Metric Benchmark Explanation
Self-citation rate
≥ 200%The journal has a self-citation rate two times higher, or more, when compared to peer journals in its subject field.
Total citation rate
≤ 50%The journal received half the number of citations, when compared to peer journals in its subject field.
CiteScore ≤ 50%The journal has a CiteScore value that is half or less than the average CiteScore value, when compared to peer journals in its subject field.
Number of articles
≤ 50%The journal produced half, or less, the number of articles, when compared to peer journals in its subject field.
Number of full-text clicks on Scopus.com
≤ 50%The journal's abstract are used half as much, or less, when compared to peer journals in its subject field.
Abstract usage on Scopus.com
≤ 50%The journal's full text are used half as much, or less, when compared to peer journals in its subject field.
Learn more on this topic via the Scopus blog: http://blog.scopus.com/posts/re-evaluation-maintaining-high-quality-content-in-scopus
| 13| 13| 13
Transparent, annual re-evaluation process to ensure
titles continue to meet high quality standards
Learn more on this topic via the Scopus blog: http://blog.scopus.com/posts/scopus-launches-annual-journal-re-evaluation-process-to-maintain-content-quality
or Elsevier.com: http://www.elsevier.com/solutions/scopus/content/content-policy-and-selection#title_re-evaluation
Full Scopus Journal base
Year 1
Year 2
Analyze full Scopus journal corpus performance based on
set metrics & benchmarks
Flag underperforming journals & inform journal publishers
If a journal underperforms for 2 consecutive years, CSAB
will re-evaluate the title based on Scopus selection criteria
Continue forward flow Discontinue forward flow or
Analyze full Scopus journal corpus performance based on
set metrics & benchmarks
Flag underperforming journals & inform journal publishers
Flagged journals for which concerns are raised, CSAB will
re-evaluate the title based on Scopus selection criteria
CSAB review
CSAB decision
| 14| 14| 14
>22,000 serial titles, 97K Conferences, 135K Books & growing every single day with ±7K items = >2.7M new records a year
RADAR: Scopus is a massive, 65M items database
| 15| 15| 15
RADAR: We’re critical of what enters the database
Scopus Content evaluation TEAM
+
=Only the highest and most relevant
content is indexed in Scopus
But once content is in Scopus, we need a
way to monitor the title it’s performance
on an ongoing basis too, and that is
where Radar comes in!
Independent Content Selection
& Advisory Board (CSAB)
| 16| 16| 16
2. RADAR: Identify and filter out outlier journals
Outlier journals are journals indexed in Scopus which
show outlier behavior and rapid, unexplainable changes
in behavior.
Journals showing that behavior are red flags as this may
point to (the beginning of) malpractice.
| 17| 17| 17
RADAR: Outlier performance as an indication for
publication practice concerns
• Examples of predicting behavior for outlier performance:
• Total article output and sudden article output growth
• Shift in geographical diversity among authors and editors
• Shift in received citations and percentage of self-citations
• A “Radar” is being developed to flag outlier journals on a regular basis
• Flagged journals will be reviewed for continuation of Scopus coverage
Examples of outlier performance:
0
100
200
300
400
500
600
700
800
900
2011 2012 2013 2014
Country 2
Other
Country 1
Big increase in article output after debut in Scopus
Shift in geographical diversity
| 18| 18| 18
Example of outlier behavior and changes in trendsSource: a biotech journal
Indian
institutesChinese institutes
Rapid
output
surge
Unexplainable change in publishing
country, its an Indian journal
Chinese authors overtaking Indian authors
| 19| 19| 19
The journal just mentioned is also present in Scopus
| 20| 20| 20
Radar launched in 2017 and is operational!
Source: https://www.elsevier.com/solutions/scopus/content
Journals flagged by the Radar tool will be added to the Re-evaluation
program and will be re-evaluated by the CSAB.
| 21| 21| 21
Re-evaluation: the results
• All journal publishers were informed by Scopus of the Re-evaluation
outcome of their journal in If discontinued = Journal forward flow
discontinued per January 1, next calendar year.
For more information: http://www.elsevier.com/solutions/scopus/content/content-policy-and-selection#title_re-evaluation
>22,000 Journals in
Scopus database
500+ Journals
underperformed
for 2 sequential
years, or
concerns were
raised
100% Re-
evaluated
by CSAB61%
Discontin
ued
| 22| 22| 22
Re-evaluation 2016/2017 results broken down
509re-evaluation
titlesMetrics Publication standards
232
241
17665
117115
51%
24%
RADAR
30
921
Out of the 509 titles selected for re-evaluation, 312 were
discontinued from Scopus (=61% discontinuation rate)
30%
| 23| 23| 23
Discontinued titles in Scopus
Source: https://www.elsevier.com/solutions/scopus/content
To keep track of which titles have been discontinued from Scopus coverage due
to underperformance, check Scopus Discontinued Sources list (300 titles):
| 24| 24| 24
Why is it important to keep track of Scopus
Discontinued Sources list?
• National Research Assessments and University Rankings must be aware as
coverage changes will alter their output.
• SciVal outputs will be impacted due to the discontinued sources.
• Authors to not publish their articles in underperforming journals.
Source: https://www.elsevier.com/solutions/scopus/content
| 27| 27| 27
www.elsevier.com/research-intelligence
Thanks all!
Scopus info site: https://www.elsevier.com/solutions/scopus
Scopus blog: http://blog.scopus.com
Webinar series: http://blog.scopus.com/webinars
Twitter: www.twitter.com/scopus
Facebook: www.facebook.com/elsevierscopus
LinkedIn: https://www.linkedin.com/company/scopus-an-eye-on-global-research
YouTube: https://www.youtube.com/c/ScopusDotCom
