Embed Size (px)
Citation preview
Institutional Repository Usage Statistics
IRUS-UK: the story so far and what’s next
17 July 2013
Balviar Notay, Jisc
Ross Macintyre, Mimas
Paul Needham, Cranfield University
Angela Conyers, Evidence Base, BCU
irus.mimas.ac.uk
The Jisc view (1)
Jisc began work in repository usage statistics area in
2009 – with requirements gathering and feasibility
testing (PIRUS)
Now we are building and in process of delivering a
national shared service for usage stats.
Recognised a growing need to measure usage - as
the repository infrastructure grew. (now 200 UK
repositories approx)
Parallel work with OpenAIRE. Also library usage
statistics– under the Jisc Activity Data programme).
irus.mimas.ac.uk
The Jisc view (2)
Usage is seen as important factor in capturing impact. Usage part of
the growing metrics infrastructure - which includes citation and
Altmetrics.
Usage statistics supports – management reporting and wider
business intelligence gathering. National aggregation of usage
statistics for allows for functions such as benchmarking
IRUS-UK gives JISC, other infrastructure providers and funders, a
nation-wide picture of the overall use of UK repositories
(demonstrate their value and impact).
IRUS-UK could also potentially act as an intermediary between UK
repositories and other agencies. E.g. The EU funded OpenAIRE
project are interested in an API to the service to monitor usage of UK
FP7 funded research.
irus.mimas.ac.uk
The Jisc view (3)
Jisc is planning the sustainability of IRUS-UK in the context of a
number of co-ordinated repository shared services.
–The UK RepNet initiative (scoping, co-ordinating and
developing technical infrastructure) coming to an end in July
2013.
–From August 2013 (for 2 years) Jisc will be building on the work
of UK RepNet to support sustainable repository shared
services.
–New name and branding will be developed for infrastructure.
– Jisc coordination and management of services (transition to
service)
–Developing the appropriate business models
–Build scalable infrastructure - interoperate with RIM and RDM
irus.mimas.ac.uk
Jisc services (1)
irus.mimas.ac.uk
Jisc services (2)
irus.mimas.ac.uk
Jisc services (3)
irus.mimas.ac.uk
Jisc programmes Academy/JISC Open Educational
Resources Programme Phase 3
Assessment & Feedback Programme
Business Intelligence
Content Programme 2011-2013
Developing Digital Literacies
Digital Infrastructure: Directions
Digital Infrastructure: Information and
Library Infrastructure Programme
Digitisation and Content
Directions: Strategic Directions
e-Learning Programme
Enhancing DMPonline Projects
Greening ICT Programme
Information and Library Infrastructure:
Emerging Opportunities
Information and Library Infrastructure:
Resource Discovery
Managing Research Data Programme 2011-
13
Relationship Management
Research Data Management Infrastructure
Projects
Research Data Management Planning
Projects
Research Management: Repositories and
Curation Shared Infrastructure
Strategic Content Alliance
Transformations Programme
UMF Shared Services and the Cloud
Programme
World War One Commemoration
irus.mimas.ac.uk
IRUS-UK: the story so far...
irus.mimas.ac.uk
IRUS-UK
Funded by Jisc as part of UK RepositoryNet+
Project Team Members:
Mimas
Cranfield University
EvidenceBase, BCU
IRUS-UK: ‘Institutional Repository Usage Statistics – UK’
Enable UK IRs to share/expose usage statistics based on a global
standard – COUNTER
Emerged from work done in ‘PIRUS2’ Project
Publisher and Institution Repository Usage Statistics project
http://www.cranfieldlibrary.cranfield.ac.uk/pirus2/
irus.mimas.ac.uk
IRUS-UK: Current status
Production-strength service infrastructure
Tracker code: DSpace & Eprints , Functional specification for Fedora.
Collecting raw usage data from UK IRs for all item types within (33)
repositories
Downloads not record views
Processing those raw data into COUNTER-compliant statistics
Making available to the originating repositories for their own use
Providing an aggregated picture of the use of items in UK repositories
Published ‘Item Types’ Report classifying types of items downloaded
COUNTER PIRUS Code of Practice published
Evaluation, dissemination and community engagement
irus.mimas.ac.uk
IRUS-UK: Repository Totals
irus.mimas.ac.uk
IRUS-UK: Item Types Totals
irus.mimas.ac.uk
IRUS-UK: Item Type <->IR: Item Type
irus.mimas.ac.uk
IRUS-UK: DOI Summary Stats
irus.mimas.ac.uk
IRUS-UK: Title/Author Search
irus.mimas.ac.uk
IRUS-UK: Ingest Summary Stats
irus.mimas.ac.uk
IRUS-UK: IR1 Report LSE Jan-Feb 2013
irus.mimas.ac.uk
IRUS-UK: CAR1 Report Jan-Feb 2013
irus.mimas.ac.uk
IRUS-UK: what’s next...
irus.mimas.ac.uk
IRUS-UK: the old ingest process
The existing ingest process has been described in detail in previous webinars and presentations - http://www.irus.mimas.ac.uk/news/
The key point is to apply the COUNTER Code of Practice to filter out robots and double clicks
However the COUNTER Robot Exclusion list is specified only as a minimum requirement – more can be done
We’ve added additional filters to
Remove more user agents
Apply a simple threshold for ‘overactive’ IP addresses
Substantially better, but we’re still not satisfied - we need a more sophisticated filtering system!
irus.mimas.ac.uk
IRUS-UK: the new ingest process (1)
We commissioned Information Power to:
Analyse raw data we’ve collected since July 2012
Test the feasibility of devising a set of algorithms that would ‘dynamically’ identify and filter out unusual usage/robot activity
A report on that work is available from http://www.irus.mimas.ac.uk/news/
Key findings from the work are
Suspicious behaviour can’t necessarily be judged on the basis of one day’s usage records or a month’s.
At certain levels of activity machine/non-genuine usage is practically indistinguishable from genuine human activity.
Going forward, we will test out and experiment with the new dynamic filtering
irus.mimas.ac.uk
IRUS-UK: the new ingest process (2)
As a service, we have to be pragmatic so we will go for a ‘best result
for least effort’ approach.
In each calendar month we will process logs daily
eliminate as much as we can with a quick, minimalist approach
insert statistics into a ‘Provisional Daily Stats’ table
At the end of each month we will reprocess those provisional stats
Apply more comprehensive, sophisticated filtering
load the restated stats into the permanent daily stats table
empty the provisional table ready for the next month
We can’t ever get to perfection in open web environment but, by the
time we’re done, we will be producing ‘the best wrong stats in town’
irus.mimas.ac.uk
IRUS-UK: Tracker patches and add-ons
Available for DSpace and Eprints
Eprints
Add-on for 3.2.x and 3.3.x
Not feasible to back port to earlier versions
DSpace
Patches for 1.8.x and 3.x
But there are a lot of older DSpace instances out there
We’ll commission 1.6.x and 1.7.x versions
irus.mimas.ac.uk
IRUS-UK: Tracker for other IR software
We will have to look at other repository software platforms on a case by case basis
Fedora Every Fedora repository is a one-off. - but some general guidelines are
available in Appendix O in the PIRUS2 Final Report, http://www.projectcounter.org/News/Pirus2_oct2011.pdf
University of Hull
PURE Portals We’ve opened discussions with Atira and we’re hopeful that IRUS-UK
Tracker functionality will be available for PURE portals …
Other platforms We would welcome dialogue with interested vendors & developers -
Contact us!
irus.mimas.ac.uk
IRUS-UK: Exposing statistics
We will be expanding The Portal
Adding new views and reports
Delving deeper into individual repository statistics
Improving DOI based views and reporting
Incorporating more metadata – Funder and Grant number
SUSHI Server
New SUSHI Service to meet COUNTER Release 4 requirements
API/Web Service
Expanding and enhancing the existing version
Usage statistics for incorporation into Repositories
Determining further requirements
irus.mimas.ac.uk
IRUS-UK: Community engagement
Growing number of repositories sending data to IRUS-UK
Currently 33 participants:
Eprints: Bath Spa, Bournemouth, City, UEA, Glasgow School of
Art, Goldsmiths. Greenwich, Huddersfield, Kent, Kingston,
Lancaster, LSE, Middlesex, NERC, Northampton, Northumbria,
Open, Reading, Salford, Sussex, UEL, UWE, Warwick
DSpace: Aberdeen, Abertay, Aberystwyth, Cranfield, Edinburgh,
Exeter, Heriot-Watt Imperial, RGU, St Andrews
Others in the pipeline
irus.mimas.ac.uk
IRUS-UK new users’ survey
Survey sent to new users 2-3 weeks
after joining – 19 replies so far
Aim to get initial impressions from users:
Best features of IRUS-UK
Benefits of using IRUS-UK
Ways in which IRUS-UK might be used
Challenges to using IRUS-UK.
irus.mimas.ac.uk
Best feature of IRUS-UK
Reliable, authoritative statistics
Ability to benchmark against others
Demonstrating value
Speed and ease of set-up
irus.mimas.ac.uk
Institutional benefits
Reliable figures to demonstrate
usage and for benchmarking
Improving repository by increasing
deposit and demonstrating usage to
researchers
Enhancing work flow and saving
time
irus.mimas.ac.uk
Improving statistical reporting
Respondents described their current methods of
collecting and using repository statistics and any
challenges they faced.
16 out of 19 (84%) said they expected IRUS-
UK to improve their statistical reporting (3 said
don’t know/too early to tell)
irus.mimas.ac.uk
Open data
During this initial period all data in IRUS-UK are
open to all UK HEIs via Shibboleth/Open Access
Asked if they were happy with data being
open:
16 out of 19 (84%) said yes, 3 not sure
Asked if their institution would be happy:
11 out of 19 (58%) said yes, 8 not sure
irus.mimas.ac.uk
Some views from the IRUS-UK
community
I do really look forward to using the download figures from your
portal. They are reliable and very useful.
It is a helpful service so far - and we are interested to follow
future developments
Very useful service, would like to see further long term
development.
Impressed so far, and looking forward to investigating further!
Great work and hope that other IRs join up asap!
irus.mimas.ac.uk
IRUS-UK: how to join
If you are a UK repository:
Contact us at irus.mimas.ac.uk to register your interest
Answer a few questions on the type of repository you have and
the version you are running
Get advice from us on what work will be involved depending on
your repository type and version
Implement any changes advised and then see your usage data
instantly in IRUS-UK with no more work from you
“The set up was quick and painless, which is always a delight!”
“Consistent collection of statistics without me having to do it!”
irus.mimas.ac.uk
Contacts and information
If you are a UK repository wishing to participate in
IRUS-UK, please contact
Project web site:
http://www.irus.mimas.ac.uk/
Thank you!
