Embed Size (px)
Citation preview
OPPORTUNITIES FOR IMPROVED QUALITY ASSESSMENT OF
RESEARCH AND JOURNALS IN AN OPEN ACCESS WORLD
SINO -FOREIGN JOINT VENTURE UNIVERSITY LIBRARY MEETING SUZHOU 2017
MARCH 30-31 , [email protected]
EDITOR-IN-CHIEF DOAJ
You are free to:
Copy, share, adapt, or re-mix;
Blog, live-blog, or post video of;
This presentation. Provided that:
You attribute the work to its author and respect the rights and licenses associated with its components.
Slide Concept by Cameron Neylon, who has waived all copyright and related or neighbouring rights. This slide only ccZero.Social Media Icons adapted with permission from originals by Christopher Ross. Original images are available under GPL at;http://w w w.thisism yurl.com /free-dow nloads/15-free-speech-bubble-icons-for-popular-w ebsites
• Q u a l i t y c o n t r o l : S C O P U S, Wo S , D O A J• W h a t i s t h e D O A J ?• W h a t i s o p e n a c c e s s ?• Q u a l i t y o f S c i e n c e• C a n s c i e n t i fi c q u a l i t y b e m e a s u r e d r e l i a b l y ?• C i t a t i o n a n a l y s i s : a r t i c l e l e v e l , j o u r n a l l e v e l• A r t i c l e l e v e l : t h e h - i n d e x • d r a w b a c k s o f c i t a t i o n a n a l y s i s• J o u r n a l l e v e l : T h e J o u r n a l I m p a c t Fa c t o r ( J I F ) a n d C i t e s c o r e• R e a s o n s f o r t h e d e m i s e o f t h e J I F• A l t m e t r i c s• s u p p o r t f o r a l t m e t r i c s : D O R A , L e i d e n M a n i f e s t o• A d v a n t a g e s a n d o p p o r t u n i t i e s o f O p e n A c c e s s• H o w D O A J d e a l s w i t h q u e s t i o n a b l e p u b l i s h e r s
Presentation overview
• DOAJ Directory o f Open Access JournalsO P E N A C C E S S J O U R N A L S O N LY
• SCOPUSO P E N A C C E S S & S U B S C R I P T I O N J O U R N A L S
• WEB OF SCIENCE O P E N A C C E S S & S U B S C R I P T I O N J O U R N A L S
• NATIONAL LISTS O P E N A C C E S S & S U B S C R I P T I O N J O U R N A L S
INDEXING SERVICES数据库收录for quality control
• QUALITY OF PUBLISHING （出版的质量）• QUALITY OF THE SCIENCE （科学的质量）
Assessing the Quality of a Journal核期刊的 量审 质
WHAT IS QUALITY?
Centrally, publicly and internationally available community-curated list of high quality open access journal titles across all disciplines and languages
The starting point for all information searches for quality, peer-reviewed open access material
A hub for metadata for Open Access articles and journals
What is the DOAJ?DOAJ是什么？
DOAJ IDRC Funded
Ambassador Programme
• DOAJ CRITERIA FOR QUALITY OPEN ACCESS PUBLISHING
•Principles of transparency and Best Practice in Scholarly Publishing
•(DOAJ, OASPA, COPE, WAME)
Assessing the Quality of Publishing（衡量出版 量）质
• A r t i c l e s o n l i n e f r e e t o r e a d = O p e n A c c e s s ?
A r t i c l e s o n l i n e f r e e t o d o w n l o a d = O p e n A c c e s s ?
• A u t h o r s c a n o n l y u s e a c c e p t e d o p e n a c c e s s a r t i c l e s f o r c l a s s r o o m u s e , p u t t h e m o n t h e i r o w n o r i n s t i t u t e w e b s i t e , m a k e l i m i t e d n r o f p r i n t s i s o p e n a c c e s s ?
• A u t h o r s n e e d t o r e t a i n t h e i r c o p y r i g h t i n O p e n A c c e s s ?
NO
• O p e n A c c e s s n e e d s B O A I s t a t e m e n t o n j o u r n a l w e b s i t e ,
s a y i n g t h e a r t i c l e w i l l b e i m m e d i a t e o p e n a c c e s s , f r e e t o r e a d , d o w n l o a d , d i s t r i b u t e , p r i n t o r u s e f o r w h a t e v e r o t h e r l e g a l p u r p o s e
A N D O p e n A c c e s s n e e d s l i c e n s i n g c o n d i t i o n s t e l l i n g w h o m a y u s e a r t i c l e c o n t e n t a n d f o r w h a t p u r p o s e s f o r i n s t a n c e C C - B Y, C C - B Y- N C , C C - B Y- S A•
Common Misunderstandings on Open Access开放 取常 的 解对 获 见 误
WHAT IS QUALITY OPEN
ACCESS?
• Free and (almost) unrestricted access
• Adherence to the BOAI definition
• No Embargoes
• No Hybrid Journals
• Copyright ownership clearly described
• Use of open access licensing like Creative Commons or equivalent
The BOAI Definition
Open Access is:
a publishing system where all content is freely available
without charge to the user or his/her institution. Users
are allowed to read, download, copy, distribute, print,
search, or link to the full texts of the articles, or use
them for any other lawful purpose, without asking prior
permission from the publisher or the author.
WHAT IS QUALITY OPEN
ACCESS?
Copyrights 权利• Clearly described on web site*• Recommend: author retains copyright• Recommend: no exclusive publishing rights• Recommend: no transfer of commercial
rightshttps://doajournals.wordpress.com/2015/06/02/copyright-and-licensing-part-2/
Content Licensing 内容许可• Clearly described on web site*• Recommend:
• licensing terms on all articles,
all versions (html, pdf, xml etc.)
• embedded in article level metadata
• Creative Commons licensing
What is a license?• Licensing means to grant a third party
(anyone else except the right holder) the right to use a copyright-protected work
• A license is a permission to use a work in specific ways
• Licenses can only be granted by copyright holder
• Copyright holder can be author or publisher
COPYRIGHT AND
LICENCING
CREATIVE COMMONS
The six variations of the CC Licences
Can we measure the Quality of Research?(in the current system)
For almost everything we eat we can find scientific evidence for causing or preventing cancer
Can we measure the Quality of Research?(in the current system)
All the studies were carefully done and scientifically sound
Is everything we eat associated with cancer? A systematic cookbook reviewJonathan D Schoenfeld and John PA Ioannidis. Am J Clin Nutrition 2013;97:127–34
oTHE BASIC S :
o PEER-REVI EWo C I TAT I ON ANALYSIS
Assessing Quality of Research核科学的 量审 质
o REASON FOR B AD QUALI TY OF SCI ENCE
o Scientific Method is flawedo Peer-review is flawed o Citation analysis has many drawbacks
o
Assessing Quality of Research核科学的 量审 质
Peer-review & the Quality of Research
Eisen JA, MacCallum CJ, Neylon C (2013) Expert Failure: Re-evaluatingResearch Assessment. PLoS Biol 11(10): e1001677. doi:10.1371/journal.pbio.1001677
o T H E S C I E N T I F I C Q UA L I T Y O F A J O U R N A L C A N N O T B E A SS E SS E D BY T H E AV E R A G E C I TAT I O N S C O R E S O F A RT I C L E S( T H E J O U R N A L I M PA C T FA C T O R J I F )
o T H E S C I E N T I F I C Q UA L I T Y O F A RT I C L E S C A N N O T B E A SS E SS E D BY W H E R E T H E Y A R E PU B L I S H E D
o T H E S C I E N T I F I C Q UA L I T Y O F S C I E N T I S T C A N N O T B E A SS E SS E D BY W H E R E O R H O W M U C H T H E Y PU B L I S H
o S C I E N T I F I C Q UA L I T Y C A N O N LY B E A SS E SS E D O N A RT I C L E L E V E L
Citations & Quality of Research核科学的 量审 质
o C I TAT I ON RATES ARE FOR ARTI CLES NOT JOURNALS
o C I T I NG DOES NOT NECESSARI LY MEAN READING
o MORE I S NOT B ET TER
Citation Based Metrics: SCI
Wageningen UR Library
oI N D I V I D UA L C I TAT I O N S C O R E : T H E H - I N D E X H I G H E S T N U M B E R ( N ) O F A RT I CL E S W I T H ( N ) C I TAT I O N S B E C O M E S H I G H E R W H E N N U M B E R O F PU B L I C AT I O N S I N CR E A S E So T H E H - I N D E X D O E S N O T C O U N T T H E A U T H O R P L A C E M E N T I N T H E A U T H O R S ' L I S ToT H E H - I N D E X D O E S N O T A CC O U N T F O R T H E T Y P I CA L N U M B E R O F C I TAT I O N S I N D I F F E R E N T F I E L D S
Citation Based Metrics: h-Index
oTHE J IF IS HIGHER W HEN ONLY A FEW ARTI CLES HAVE HI GH C ITATI ON RATES
o AVERAGE CI TATI ON SC ORES I N JOURNALS SAY NOTHING AB OUT I NDI VID UAL ARTI CLES
o THE J IF I S HI GHLY B I ASED TOWARDS ENGLISH LANGUAGE PUBLIC ATI ONS
o H IGH J I F ’S C ORRELATE W ITH MORE RETRACTI ONS
Citation Based Metrics: JIF
H E A D L I N E S I N , S C I E N C E , N A T U R E , T H E S C I E N T I S T, P L O S I N 2 0 1 6
Table 2: Percentage of papers published in 2013-2014 with
number of citations below the value of the 2015 JIF.
Larivière et al. (2016)
highly skewed distribution of citations高度不平衡的文献引用分布
doi: http://dx.doi.org/10.1101/062109
Replacing the Impact Factor取代影响因子
this analysis helps to expose the exaggerated value attributed to the JIF and strengthens the contention that it is an inappropriate indicator for the evaluation of research or researchers," the team concludes in a preprint paper posted to bioRxiv.
doi: http://dx.doi.org/10.1101/062109
FOR PUBLISHERS:DO NOT USE IF FOR THE PROMOTION OF YOUR
JOURNAL
FOR AUTHORS:IT IS NOT IMPORTANT WHERE YOU PUBLISH BUT
WHAT YOU PUBLISH
FOR FUNDERS:DO NOT JUDGE SCIENTISTS ON WHERE THEY
PUBLISH OR HOW MUCH THEY PUBLISH
Too much is wrong with the JIF期刊影 因子有太多的响 错误
Journal CiteScore
Journal CiteScore
• I O A N N I D I S : M O S T P U B L I S H E D M E D I C A L R E S E A R C H I S FA L S E
• B E T T E R E X P E R I M E N TA L S E T- U P, C O N T R O L P O S S I B I L I T I E S• O P E N A C C E S S ( A R T I C L E S A N D D ATA ) O F F E R S B E T T E R C O N T R O L P O S S I B I L I T I E S• N E E D T O P U B L I S H A L L , N E G AT I V E A N D P O S I T I V E F I N D I N G S• M O R E AT T E N T I O N F O R R E P R O D U C I B I L I T Y
Improving the Quality of Research?
* Ioannidis JPA (2005) Why most publishedresearch fi ndings are false. PLoS Med 2(8): e124
San Francisco Declaration on Research Impact
关于科研影 力的旧金山声明响
http://www.ascb.org/dora/
It doen’t matter where you publish or how much you publish, research assessment should not be based on journal citation analysis like JIF or number of articles published
San Francisco Declaration on Research Impact
关于科研影 力的旧金山声明响
http://www.ascb.org/dora/
Leiden Manifesto for Research Metrics
TEN PRINCIPLESAmong which
6. Account for variation by field in publication and citation practices8. the journal impact factor is published to three decimal places to avoid ties. it makes no sense to distinguishbetween journals on the basis of very small impact factor differences. Avoid false precision
ALTMETRICSTAKING INTO ACCOUNT CITATIONS, SOCIAL MEDIA,
CONVENTIAL MEDIA (NEWSPAPERS, TV, VIDEO)
RELATIVE CITATION SCORESCITATION INDICES ADJUSTED FOR SCIENTIFIC FIELD
Article Level Impact Assessment基于文献的影 力 估响 评
LEIDEN MANIFESTOdoi:http://dx.doi.org/10.1101/062109
DORAThe San Francisco Declaration on Research Assessmentdoi: 10.1242/dmm.012955
Relative Citation Ratios
https://icite.od.nih.gov/
•
Relative Citation Ratios
OPEN ACCESS.....• A RT I C L E S R E C E I V E M O R E C I TAT I O N S• A RT I C L E S A R E M O R E R E A D• R E A C H E S A B I G G E R A U D I E N C E• I S L E SS P R O N E T O B I A S• AVO I D S D U P L I C AT E S T U D I E S• A RT I C L E S C A N B E M O R E E A S I LY C H E C K E D• D ATA L E SS P R O N E T O M A N I PU L AT I O N• I S B E T T E R F O R A DVA N C I N G S C I E N C E• L E A D S T O M O R E I N N OVAT I O N• L E A D S T O B E T T E R E D U C AT I O N
Quality in Open Access真正的开放 取获
Quality in Open Accessrole of peer review
https://opennessinitiative.org/
Quality in Open Accesspost publication peer review
How DOAJ detects questionable journals
• Low publishing qualityJournal name, website, fees, peer review, publisher, ownership,
volume of articles, advertisements, prominent soliciting for editors
• Low scientific quality focus, format, self-citations, plagiarism
• Malpracticefalse claims, hidden costs, spamming authors, wrong information
Questionable Publishing in Perspective
• Not limited to open accessaccording to a study* by Walt Crawford the number
questionable publishers in 2014 was about 3275 publishing about 121,000 articles
• Proportion of low quality journals is comparable between open access and subscription publishing
FACTS• Not all subscription journals are in Scopus : only 10-20,000 of 100,000 (data Ulrich’s
Web)• Not all open access journals are in DOAJ : only 8 -10,000 of 30,000 **
CONCLUSION Percentage of Quality Journals is comparable
* Source : http://walt.lishost.org/2015/11/ppppredatory-article-counts-an-investigation-part-1/
** Source: Walt Crawfordhttp://citesandinsights.info/civ17i1.pdf
IF OPEN ACCESS WERE REALLY SUCH LOW QUALITY.... .
W H Y W O U L D B I G PU B L I S H E R S S W I T C H T O O P E N A C C E SS ? W H Y W O U L D 3 0 % O F A L L S T M PA P E R S T O D AT E B E PU B L I S H E D U S I N G O P E N A C C E S S ? W H Y W O U L D O P E N A C C E SS G R O W AT T W I C E T H E R AT E O F T O L L A C C E S S ?•
The Reality of Open Access真正的开放 取获
Thanks to :
Xin Bi and the Xi'an Jiaotong-Liverpool University , all the Library Consortia, Universities and Publishers and our Sponsors for the financial support to DOAJ!感谢我们所有的图书馆联盟，大学，出版商和赞助商对 DOAJ的财政支持！
DOAJ depends entirely on donations
https://doaj.org/membership
