SCIENTIFIC SOLUTIONS Searching for Value in a Changing Research Environment Patricia Brennan Academic & Government Product Development Thomson Scientific

SCIENTIFIC SOLUTIONS

Searching for Value in a Changing Research Environment

Patricia BrennanAcademic & Government Product DevelopmentThomson Scientific

NISO Usage Data ForumNovember 2, 2007Dallas, TX

Copyright 2007 Thomson


Searching for value…

How do we assess value?

The changing research environment – what does it look like?

What data best serve the needs of the key stakeholders?

Researcher

Librarian

Publisher

Viable metrics and consistent data for work-flow decisions

Where do standards fit?




How do we assess value?

– Role of citations, citation metrics– Possibilities with new metrics – Usage Factor, H Index, – Recognize the changing nature of the research environment




Science has grown faster than our ability to keep up

Estimated 20,000 papers published daily

Today’s scientific research is evolving in a global and multidisciplinary context

Selectivity is a must:

Which articles should a researcher read?

Which journals should a library subscribe to?

Which projects and researchers should be funded?

Data based decision making increasingly important



Scholarship is Global

0

200

400

600

800

1000

1200

1400

1600

1800

1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001 2003

Year

Papers with N Countries

> 5 countries > 10 countries > 15 countries

>5 countries

Source: Web of Science®



Mean Authors per Paper

0

0.5

1

1.5

2

2.5

3

3.5

4

81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 0 1 2 3 4 5 6

year

author/paper

…and Collaborative




Percent papers with 5 or more authors

0

5

10

15

20

25

30

35

81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 0 1 2 3 4 5 6

year

Percent

Collaboration is Getting Broader




…even Enormous!

Largest Collaboration in 2006: 2512 authors, a “collaboration of collaborations”




Research is also…

Communicative at ever higher velocities

Collaborative, as a fundamental research practice

Competitive, at many levels, and

Creative in new ways --multiple formats and venues (objects, databases, software)








The researcher/research group needs to:– Keep up to date on new and developing science

• Access to reliable and timely sources

– Nurture new talent through grants and funding• Identify rising stars

• Build your brand

– Provide timely, objective and informed synopses for decision makers

• Output reports

– Measure the impact of scientific efforts



Field strengths: Most cited paperCollaborative effort on the hot topic of Obesity & Disease.

The top 5 cited papers focus on Medical topics.

Citation performance graph



Where are the thought leaders at your institution?

Top 10 in Plant ScienceTop 10

Many top cited authors are focusing in a mix of medical areas. Where does management want to focus the traditional agricultural-related research?

Top 10 in Biochemistry & Mol. Biology



Individual strengths: Most cited Author

The most cited authors in the papers are involved in homocysteine research

Total Papers = 196, Total Cites = 7,392 Average times Cited = 37.71, Median times Cited = 11

H-index = 44. Selhub has 44 papers with at least 44 citations.C-index = 2.01. The baseline is 1. This value is the total cites divided by the total expected cites.Average Percentile = 20.55. On average, Selhub’s papers were in the top 20% of papers.Disciplinarity = 0.07. Ranging from 0 to 1, the lower the number, the greater the multidisciplinarity. Selhub’s work is very interdisciplinary.



Individual strengths: Most prolific Author

Dr. Dubey has worked on Toxoplasma gondii and Neospora caninum.

Collaborates with Virginia Tech, USDA colleagues, Auburn as well as Brazil, and France

Total Papers = 368, Total Cites = 5,435 Average times Cited = 14.77, Median times Cited = 6

H-index = 32. Dubey has 32 papers with at least 32 citations.C-index = 1.71. The baseline is 1. This value is the total cites divided by the total expected cites.Average Percentile = 35.39. On average, Dubey’s papers were in the top 1/3.Disciplinarity = 0.41. Ranging from 0 to 1, the lower the number, the greater the multidisciplinarity



Collaborating countries

Canada and other G-8 countries are the main collaborating countries.

But it is the collaborations in Northern Europe that have the highest impact.

0 200 400 600 800 1000 1200 1400

Finland

Scotland

Denmark

Switzerland

Belgium

Sweden

New Zealand

Israel

Netherlands

Italy

Mexico

Spain

Peoples R China*

Brazil

Japan

France*

Australia*

England

Germany

Canada

0 5 10 15 20 25 30

Avg. cite

Netherlands

Scotland

Iceland

Finland

North Ireland

Ireland

Wales

Countries with highest average cite rates

Wales

Ireland

North Ireland

Finland

Iceland

Scotland

Netherlands



Citing Organizations

These are the top foreign citing organizations, by citation are:• INRA, France• Wageningen University, the Netherlands• Chinese Academy of Science, PRC• Agriculture & Agricultural Food, Canada• CSIC, Spain• CSIRO, Australia

Top 5 INRA papers citing USDA research

What countries or institutions would USDA want to collaboratewith in the future?



Recently cited Work and specific topic areas

This is a portion of research written in 2005-2007. Here the fields are Agronomy and Plant Science, not medical fields. These could be papers and projects to watch.

Topic areas: Biofuels

175 organizations cite this biofuels research . Could there be opportunities for new research collaborations?

Of a sample of 100 biofuel papers, there were 26 collaborating organizations on these papers.




Decision Support in the Library Community

– Meeting Researcher Needs– Building collections




What librarians need:

– Systems that enable collection management librarians to work with and validate journal usage and about what journals our academics are publishing in.”

– Confidence in demonstrating that the subscribed journal packages are giving value for money.

– library is fully integrated in evaluation of University's publication output

– collection management in multi-tiered acquisition structures (consortia, partnerships, special libraries)

– collection development (relation between library holdings and researcher's publication activity)

– “we need to demonstrate we are truly supporting the current research focus of the institution.”




What tools do librarians have?

• Input from Faculty• Understanding of the Discipline • Understanding the needs of the Community• Journal Citation Reports (IFs)• Citation Analyses• Usage Data• Cost/Funds (special $$ allocations)• Pertinence to Local Focus • Reviews• Overlap Analyses Tools






• Within the new world of 21st century scholarship, new sets of questions about value emerge:

– Who of the many researchers listed on this paper is next “hot shot”?

– Isn’t the person who built the database underlying the research as deserving of credit as the person who wrote up the research?

– Isn’t this database a valuable research product in its own right, and perhaps more valuable in the real network of scholarly communications today than the article that incarnates the conclusions of a point-in-time discovery made using it?

– What portion of the scholarly network of value has actually been captured by the journal, and what is thus subject to any “journal-level” metric?



• Consider whether available data can address the question

• Choose publication types, field definitions, and years of data

• Decide on whole or fractional counting

• Judge whether data require editing to remove “artifacts”

• Compare like with like

• Use relative measures, not just absolute counts

• Obtain multiple measures

• Recognize the skewed nature of citation data

• Confirm data collected are relevant to question

• Ask whether the results are reasonable

Following Best Practices




• Usage Applications• Characteristics of community• Understanding the community of users

– Who are they ?– Where are they likely to come from?

• Systems and processes to support ongoing data gathering, analyses, interpretation, and application

• Collaborative efforts




Opportunities for standards?– Item Definitions– Performance Measurement– Global Repositories– Standard Applications– Data Transfer Protocols




Thank You!

Documents

SCIENTIFIC SOLUTIONS Searching for Value in a Changing Research Environment Patricia Brennan Academic & Government Product Development Thomson Scientific