Upload
jessie
View
26
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Competências Básicas de Investigação Científica e de Publicação. Lecture 8: Global science and the need for metrics. Brasil’s economic growth creates a demand for scientists and technologists. S cience drives the economy. - PowerPoint PPT Presentation
Citation preview
Ganesha Associates 2013
Competências Básicas de Investigação Científica e de Publicação
Lecture 8: Global science and the need for metrics
05/06/2013
Ganesha Associates 2013
Brasil’s economic growth creates a demand for scientists and technologists
05/06/2013
Ganesha Associates 2013
Science drives the economy• “The ability to create, distribute and exploit scientific and technological
knowledge has become a major source of competitive advantage, wealth creation and improvements in the quality of life.
• Some of the main features of this transformation are;– the growing impact of information and communications technologies (ICT) on the
economy and on society – the rapid application of recent scientific advances in new products and processes– a high rate of innovation across economically developed countries– a shift to more knowledge-intensive industries and services– rising skill requirements.”
• Source: Organisation for Economic Co-operation and Development ( OECD) Policy document 2000
05/06/2013
Ganesha Associates 201305/06/2013
Global research investment
28 August 2012 Ganesha Associates 5
Ganesha Associates 201305/06/2013
Ganesha Associates 201305/06/2013
Ganesha Associates 2013
Document output in 2011
05/06/2013
Ganesha Associates 2013
Brasil:
05/06/2013
Ganesha Associates 2013Source: International Comparative Activity Performance – Elsevier 2011
BRIC output: documents
05/06/2013
Ganesha Associates 2013
Citation quality is a problem
05/06/2013
Ganesha Associates 2013
International collaboration is low
05/06/2013
Ganesha Associates 2013
Share of articles by co-authorship type
05/06/2013
EPI : English Proficiency Index by age
Ganesha Associates 201305/06/2013
Ganesha Associates 2013
Institutional ranking
05/06/2013
Ganesha Associates 2013
• O Qualis é um levantamento realizado pela Capes com o intuito de mensurar a qualidade da produção científica dos programas de pós-graduação.
• A classificação de periódicos é realizada pelas áreas de
avaliação e passa por um processo anual e atualização. Esses veículos são enquadrados em estratos indicativos de qualidade; A1, o mais elevado; A2; B1; B2; B3; B4; B5; C, com peso zero.
• O seu programa de pós-graduação será avaliado segundo essa escala.
05/06/2013
Brasil Qualis and the importance of Indexação
UK’s Research Excellence Framework 2014
• Outputs: The panels will assess the quality of submitted research outputs in terms of their ‘originality, significance and rigour’, with reference to international research quality standards. This element will carry a weighting of 65 per cent.
• Impact: The sub-panels will assess the ‘reach and significance’ of impacts on the economy, society and/or culture that were underpinned by excellent research conducted in the submitted unit, as well as the submitted unit’s approach to enabling impact from its research. This element will carry a weighting of 20 per cent.
• Environment: The sub-panels will assess the research environment in terms of its ‘vitality and sustainability’, including its contribution to the vitality and sustainability of the wider discipline or research base. This element will carry a weighting of 15 per cent.
Global metrics summary
• Science and technology are too politically important for them to be left un-managed
• The outputs of science and technology are too numerous and complex for them to be managed without reference to metrics of some form
• Metrics will become more important as better standards are defined
Break
Portfolio analysis at NIH
• What is the state of the science? • Are there gaps in our research portfolio?• How do the gaps compare to research carried
out in other agencies?• How much do we spend on mitochondrial
research?• What is the average number of scientific
papers generated per $100K spent per grant?
28 August 2012 Ganesha Associates 27
Define: indexação
• A indexação de uma revista é o processo que confere o indicador de qualidade necessário para que esta possa pertencer a um ou mais bancos de dados de renome internacional, em uma área específica do conhecimento, como por exemplo, MEDLINE, ISI, LILACS, etc., segundo critérios preestabelecidos por essas instituições.
• É um fator importante para que a produção científica nela apresentada se torne conhecida e reconhecida internacionalmente.
• O periódico necessita, para isto, estar de acordo com os padrões básicos de apresentação formal e excelência de conteúdo científico, determinados internacionalmente.
28 August 2012 Ganesha Associates 28
28 August 2012 Ganesha Associates 29
28 August 2012 Ganesha Associates 30
As fontes de‘indexação’ em ordem de ‘qualidade’
• Thomson Web of Science• PubMed/Medline• Scopus• ScieLo• Google• Scirus• LILACS
28 August 2012 Ganesha Associates 31
Journal selection process - ISI
• To properly reflect the global context in which scientific research takes place, and to provide balanced coverage in each category, ISI seeks to cover the best regional journals as well.
• However, rather than compare a regional journal with all journals in a particular subject category, the ISI editor considers it in terms of those journals in the category from the same geographic area.
• High journal publishing standards, especially timeliness, and English language bibliographic elements remain essential.
28 August 2012 Ganesha Associates 32
Impact Factor defined
“The Impact Factor of a journal is probably the only quantitative way of assessing its worth and relevance to the academic community it serves.” Editorial, Journal of Computer-assisted Learning, 2001
Impact Factor 2011 =# citations received in 2011 to articles published in 2010/2009______________________________________________________
# number of source items published 2010 and 2009
Journal of Cell Science - IF
Nature citations per article
Citation statistics – PLoS Biology
5 July 2012 Ganesha Associates 35
Ganesha Associates 3614 August 2013
28 August 2012 Ganesha Associates 37
Impact Factors over time
28 August 2012 Ganesha Associates 38
Impact Factor and area
28 August 2012 Ganesha Associates 39
Impact Factors for Microbiology
28 August 2012 Ganesha Associates 40
Impact Factors for Neuroscience
28 August 2012 Ganesha Associates 41
Impact Factors and article type
28 August 2012 Ganesha Associates 42
Impact Factors and Authors
Correlation between article citation rate and journal impact for four authors
28 August 2012 Ganesha Associates 43
Impact Factors and Journals
Citation rates in 1986 or 1987 of articles published in three biochemical journals in 1983 or 1984, respectively. From Seglen, BMJ 1997;314:497
28 August 2012 Ganesha Associates 44
Alternatives ?• The Hirsch Index is rapidly gaining favor as a
performance measurement tool for scientific authors.
• A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np - h) papers have no more than h citations
• Thus, the h-index reflects both the number of publications and the number of citations per publication. The index is designed to improve upon simpler measures such as the total number of citations or publications.
• Like the IF, the index works properly only for comparing scientists working in the same field.
• For more information see Scopus or Wikipedia
28 August 2012 Ganesha Associates 45
H-index - weaknesses• The h-index is bounded by the total number of
publications. • This means that scientists with a short career are at
an inherent disadvantage, regardless of the importance of their discoveries.
• For example, had Albert Einstein died in early 1906, his h-index would be stuck at 4 or 5, despite his being widely acknowledged as one of the most important physicists, even considering only his publications to that date.
Declaration on Research Assessment
• Made during the Annual Meeting of The American Society for Cell Biology (ASCB) in San Francisco, CA, on December 16, 2012.
• Main recommendation: Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.
• Alternatives?
New Usage Metrics• Impact Factor
Well-established, easily understood and acceptedEndorsed by funding agencies and researchersSeveral competitors: H-index, Eigenfactor, SJR (Google Page Rank)
• COUNTER Usage FactorJournal/book usage-based alternative perspectiveReflects value of journals to all categories of userEasy to understand
• PIRUS (Publisher and Institutional Repository Usage Statistics) COUNTER-compliant usage reports at the article levelWill enable the usage of individual research outputs to be measured
• ORCID (Open Researcher and Contributor ID)Central registry of identifiers from individual researchersWill enable aggregation of multiple forms of research output
FundRef
Altmetrics, an alternative?
5 July 2012 Ganesha Associates 49
Ganesha Associates 50
Many metrics, many uses…• Development of national-level social contracts for science and technological investment• Does investment lead to the achievement of national/international objectives for
increased employment, economic growth, improved healthcare? • At a national/institutional/departmental/individual level how do we compare with the
competition?• Management of human resources and tacit intellectual property in a multinational
organisation• Do I get tenure/promotion?• How often has my article been downloaded/cited• How well has my paper contributed to building the knowledge network in my field?• Which journal should I publish in?• Which articles should I read?• Has my article been picked-up by the media?
• Standards: CrossRef, ORCID, DataCite, FundRef
14 August 2013
Snowball metrics
Global metrics summary
• Science and technology are too politically important for them to be left un-managed
• The outputs of science and technology are too numerous and complex for them to be managed without reference to metrics of some form
• Metrics will become more important as better standards are defined