NISO Webinar: Making Better Decisions with Usage Statistics

Making Better Decisionswith Usage Statistics

June 13, 2012

Speakers: Peter Shepherd, Oliver Pesch, and Amy Lynn Fry

http://www.niso.org/news/events/2012/nisowebinars/usage_statistics/

Release 4 of the COUNTER Code of Practice for e-Resources

Peter Shepherd

COUNTER

June 2012

COUNTER Release 4 - objectives

A single, unified Code covering all e-resources, including journals, databases, books, reference works, multimedia content, etc.

Improve the database reports Improve the reporting of archive usage Enable the reporting of mobile usage separately Expand the categories of ‘Access Denied’ covered Improve the application of XML and SUSHI in the

design of the usage reports Collect metadata that facilitates the linking of usage

statistics to other datasets, such as subscription information

Release 4: main features

A single, integrated Code of Practice covering journals, databases, books, reference works and multimedia content

An expanded list of Definitions, including terms such as ‘Gold Open Access’, ‘Multimedia Full Content Unit’, ‘Record View’, ‘Result Click’, as well as different categories of ‘Access Denied’, etc. that are used for the first time in Release 4

Enhancements of the SUSHI (Standardised Usage Statistics Harvesting Initiative) protocol designed to facilitate its implementation by vendors and its use by librarians


A requirement that Institutional Identifiers, Journal DOI and Book DOI be included in the usage reports, to facilitate not only the management of usage data, but also the linking of usage data to other data relevant to collections of online content.

A requirement that usage of Gold Open Access articles within journals be reported separately in a new report: Journal Report 1 GOA: Number of Successful Gold Open Access Full-text Article Requests by Month and Journal.

A requirement that Journal Report 5 must be provided


Modified Database Reports, in which the previous requirement to report Session counts has been dropped, and new requirements, to report Record Views and Result Clicks, have been added. (Database Report 3 has also been renamed Platform Report 1).A new report, Multimedia Report 1, which covers the usage of non-textual multimedia resources, such as audio, video and images, by reporting the number of successful requests for multimedia full content unitsNew optional reports covering usage on mobile devicesA description of the relative advantages of logfiles and page tags as the basis for counting online usage Flexibility in the usage reporting period that allows customers to specify a date range for their usage reports

Release 4: Standard Usage Reports

Journal Report 1: Number of Successful Full-Text Article Requests by Month and Journal

Journal Report 1 GOA: Number of Successful Gold Open Access Full-Text Article Requests by Month and Journal

Journal Report 2: Access Denied to Full-Text Articles by Month, Journal and Category

Journal Report 5: Number of Successful Full-Text Article Requests by Year-of-Publication (YOP) and Journal

Database Report 1: Total Searches, Result Clicks and Record Views by Month and Database

Database Report 2: Access Denied by Month, Database and Category Platform Report 1: Total Searches, Result Clicks and Record Views by Month and

Platform Book Report 1: Number of Successful Requests by Month and Title Book Report 2: Number of Successful Section Requests by Month and Title Book Report 3: Access Denied to Content Items by Month, Title and Category Book Report 4: Access Denied to Content Items by Month, Platform and Category Book Report 5: Total Searches by Month and Title Multimedia Report 1: Number of Successful Full Multimedia Content Units

Requests by Month and Collection

Release 4: Journal Report 5

Release 4: Database Report 1

Release 4: Multimedia Report 1

Release 4: recording and reporting usage on mobile devices

The following optional additional reports enable usage on mobile devices to be reported separately:

Journal Report 3 Mobile: Number of Successful Item Requests by Month, Journal and Page Type for usage on a Mobile Device

Title Report 1 Mobile: Number of Successful Requests for Journal Full-text Articles and Book Sections by Month and Title ( formatted for normal browsers/delivered to mobile devices AND formatted for mobile devices/delivered to mobile devices)

Title Report 3 Mobile: Number of Successful Requests by Month, Title and Page Type (formatted for normal browsers/delivered to mobile devices AND formatted for mobile devices/delivered to mobile devices)

COUNTER will recognize as usage on a mobile device, which may be reported in the above reports, any usage that meets one of the following criteria:

useragents that are included in the WURFL list. WURFL is the Wireless Universal Resource FiLe, a database containing the profile of mobile devices; this database may be found at: http://wurfl.sourceforge.net/

usage via a proprietary mobile App provided by the publisher/content provider

http://wurfl.sourceforge.net/

Release 4: timetable for implementation

Deadline date for implementation of Release 4:

31 December 2013

-after this date only vendors compliant with Release 4 will be COUNTER compliant

Between now and 31 December 2013, Release 4 and the existing Releases of the Codes of Practice are valid

Release 4: compliance process

1.Agree Release 4 usage reports that are relevant to publisher content

2.Review of reports by a COUNTER library test site

3.Vendor signs Declaration of COUNTER compliance

4.Vendor added to the Register of COUNTER-compliant vendors

5.Independent audit must be carried out and passed within 6 months of being added to the Register

Release 4: independent audit

Three aspects of the audit Check report formats Check data integrity Check delivery process

Three possible audit outcomesA Pass, in which case no further action is required by the publisher as a result of the audit. In some cases the auditor may add Observations to the audit report, which are designed to help the vendor improve its COUNTER usage reports, but which are outside the scope of the audit itself.A Qualified Pass, in which the auditor deems the publisher to have passed the audit, but where the auditor raises a Minor Issue requiring further action to maintain COUNTER-compliant status. A Minor Issue does not affect the reported figures, but is one which should be resolved within 3 months of the audit to maintain COUNTER-compliant status. An example of a Minor Issue is where a report format does not conform to the COUNTER specifications.A Fail, where the auditor has identified an issue that must be resolved immediately for the vendor to maintain COUNTER -compliant status

Full details of Release 4 will be found on the COUNTER website at:

http://www.projectcounter.org/code_practice.html

COUNTER Code of Practice-Release 4

http://www.projectcounter.org/code_practice.html

Quality Content • Resource Management • Access • Integration • Consultation

Making Better Decisions with Usage StatisticsSUSHI: More relevant than ever

Oliver PeschChief Strategist, E-Resource

Access and Management Services

Overview

• Why SUSHI• COUNTER Release 4

• Changes to schemas• Procedural changes• Applying lessons learned

• Other tools and resources available

Why SUSHI…

• Librarians doing more with less rely on usage statistics as one measure of value of their purchases

• Usage consolidation applications and related services rely on COUNTER

• To be valuable, usage collection needs to be comprehensive

• Efficiency depends on automation • SUSHI is a very scalable standard for

harvesting COUNTER reports!

COUNTER 4: Schema Changes…

SUSHI Schema (the actual SUSHI standard)


SUSHI Schema (the actual SUSHI standard)

No changes were required!


COUNTER XML Schema for reports: changes were minor and geared towards better compliance, including:• ItemIdentifier, ItemPublisher elements now

optional• PubYr attributes validate as year• PubYrFrom, PubYrTo attributes added to

support Journal Report 5


COUNTER Elements XML Schema: lists valid values for certain data elements

• DataType: added Collection and Multimedia options

• Categories: added Access_denied and removed Turnaways.

• MetricType: added several new values to support reports for mobile use, the new database reports, multimedia reports and revised access denied reports.

COUNTER 4: Procedural Changes related to SUSHI

• To be COUNTER compliant, a content provider must provide a working SUSHI server

• Testing the SUSHI implementation will be part of the audit

Lessons Learned:Perspective as a Usage Consolidation Vendor

COUNTER…• Offers the promise of consistency necessary

for consolidated reporting• Combined with SUSHI is potentially a

significant time-saver• Covers a broad range of reports providing

usage for journals, books, databases

UsageConsolidation


Challenges with COUNTER…• “Almost” compliant reports require manual

intervention

UsageConsolidation


UsageConsolidation


intervention



intervention

UsageConsolidation



intervention

UsageConsolidation



intervention

UsageConsolidation



intervention

UsageConsolidation



intervention

UsageConsolidation



intervention• SUSHI implementations falling short

UsageConsolidation



intervention• SUSHI implementations falling short

UsageConsolidation

SUSHI Challenges

• Overly complex authentication methods

• Lack of understanding of market (most SUSHI clients are hosted -- one client will harvest usage for many customers)

• Support staff unfamiliar with set-up needs or operational details



intervention• SUSHI implementations falling short• Many content providers still not COUNTER

compliant

UsageConsolidation

NISO SUSHI Standing Committee: Release 4 of the COUNTER Code of Practice

Applying the lessons learned…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile

removes ambiguity• Free web-based client available for testing or

basis for development• Even more resources available at the NISO

SUSHI web site.

NISO

NISO SUSHI Web Site: Support for developers and librarians

Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile


basis for development• Many resources available at the NISO

SUSHI web site.





SUSHI web site.





SUSHI web site.





SUSHI web site.





SUSHI web site.





SUSHI web site.





SUSHI web site.





SUSHI web site.





SUSHI web site.





SUSHI web site.





SUSHI web site.





SUSHI web site.





SUSHI web site.

Conclusion

• SUSHI is both relevant and necessary, but we need…• more compliant content providers• with better interoperability

• COUNTER Release 4 more comprehensive audit coupled with the COUNTER SUSHI Implementation Profile are keys to progress

SUSHI: More relevant than ever

Thank You!

Visit the NISO SUSHI web site

http://www.niso.org/workrooms/sushi

Or email me at:

[email protected]

http://www.niso.org/workrooms/sushi

Amy Fry, Electronic Resources Coordinator, Bowling Green State Universityimages by Ken Fager, http://www.flickr.com/photos/kenfagerdotcom/

Streamlining Stats:Managing Usage Statistics Efficiently and Effectively

http://www.flickr.com/photos/kenfagerdotcom/

What do I get? How do I get it?

I need it now! How do I put it all together?

Usage statistics: what libraries are thinking

Stats work today is HYBRID:part manual, part automated

Look familiar?

How about this?

Just a little manual labor…

Maybe we should just cancel everything.

Data control Organization of work Distribution of labor

Everything in its (one and only) place, all linked together.

Stats are integrated into the resource’s lifecycle.

There are jobs for librarians, staff and students.

BGSU’s three principles to streamline stats

Data controlONE place for logins and passwords:

our ERM.


our ERM.


our ERM.

Data control ONE place for downloading instructions: our wikis

dbusestats.bgsu.wikispaces.net ejusestats.bgsu.wikispaces.net

Data controlONE place for the stats files:

our R drive.

Data controlONE file for each report type

Organization of workStats are integrated into the lifecycle of each

resource.

• Reports are saved as downloaded (by calendar year)• Stats are collected once a year

Organization of workCollecting stats has been simplified

Distribution of labor

Librarian jobs:• Manage logins and passwords• Manage reporting• Manage the SUSHI table in the ERM• Manage stats through each resource’s lifecycle

Staff jobs:• Write downloading instructions• Make changes as needed in the wiki (adding, removing, and changing resources)• Convert files XML; upload XML files into the ERM

Student jobs:• Collect and save stats according to wiki instructions

Other jobs:• Serials, Acquisitions & IT staff work with order records, do coverage load, and more.

Everyone gets to help!

ReportingOur medium

– For databases: spreadsheets, for now– For e-journals: our ERM

Our philosophy– Focus on standard measures– Try to paint a picture over time– Accuracy isn’t everything

Database usage reporting:Standard measurements

Database usage reporting:Use over time

Database usage reporting:Using estimates

The “EBSCO Bump”

E-journal usage reporting: Millennium ERM JR1 Summary Report

E-journal reporting using the Millennium ERM

– Create resource records for each journal package– Relate order records with correct from/to dates for

each subscription– Download COUNTER JR1 reports in CSV format– Convert to XML using the University of Nebraska-

Lincoln script (http://statsconverter4erm.unl.edu/) – Upload each XML file to the proper resource record

http://statsconverter4erm.unl.edu/

http://statsconverter4erm.unl.edu/

Detail 1 Report for ACS Journals

Resource-Level Statistics Export for ACS Journals

With a little editing…

Coming soon…

• Innovative is expanding the report types that can be uploaded to its ERM in 2013!

Scholarly Stats– Basic stats collection service, pay per platform– Basic collated reports

SwetsWise Selection Support– Stats collection service (pay per

platform)– Options to upload data manually

360 Counter (Serials Solutions)– Unlimited platforms with annual

subscription– Options to upload data manually

Feel like outsourcing?

Open-source ERMs

• ERMes (University of Wisconsin-La Crosse)– Incorporates DB1 reports

• CORAL (Notre Dame)– Incorporates JR1 reports

• E-Matrix (NCSU)– Incorporates JR1 reports

• CUFTS (Simon Fraser)– Incorporates JR1 reports;

uses SUSHI

Remember…

You’re never, ever going to get out of collecting data altogether unless you decide to stop looking at your stats.

Technology

NISO Webinar: Making Better Decisions with Usage Statistics