Karen CalhounVice President, Metadata

The Future of Library Metadata

A Presentation for Japanese Librarians

The Future of Library Metadata

A Presentation for Japanese Librarians

Waseda

University

2010

8 September

By: Chris Jan

http://www.flickr.com/photos/chrisjan99/4607644605

THE TRENDS:SUPPORTINGKNOWLEDGECREATION

THE TRENDS:SUPPORTINGKNOWLEDGECREATION

-- Changes in scholars and scholarship-- A new generation of students

By: Chris Jan

http://www.flickr.com/photos/chrisjan99/4607646173

The Changing Context for Research, Teaching, and LearningThe Changing Context for Research, Teaching, and Learning

Knowledge itself will be modified and research and

development transformed by the new capacities provided

by IT. Nothing will be left untouched. The liberal arts will be

revived and transfigured, liberated from their age-long

reliance on text alone. The silos of the departments will

topple as new approaches to bewildering issues are

pursued with new vigor by scholars in mind-boggling

combinations of once insular and isolated disciplines.

–Frank H. T. Rhodes, past President of Cornell University, in

The Creation of the Future (Ithaca: Cornell University Press,

2001).

The Larger Context: Knowledge CommunitiesThe Larger Context: Knowledge Communities

Knowledge communities “interpret information about

the environment in order to construct meaning …

create new knowledge by converting and combining

the expertise and know-how of their members …

[and] analyze information in order to select and commit

to appropriate courses of action.”—Chun Wei Choo,

professor of Information Studies, University of Toronto

The Knowing Organization: How Organizations Use Information to Construct

Meaning, Create Knowledge, and Make Decisions (New York: Oxford

University Press, 1998), xii.

Library: Purchasing Agent or Knowledge Community Partner?Library: Purchasing Agent or Knowledge Community Partner?

“The gateway, archive, and buyer functions are among the core traditional roles of the library.

But many believe that these historical roles will not be the main focus of libraries in the future,

and envision the transformation of the library from an institution focused on acquiring,

maintaining, and providing services centered on a local print collection into a more electronic hub

offering a variety of services to support campus needs for research, teaching, and learning.”—p. 9

ImplicationsImplications

Students and faculty engage in information network processes with or without libraries

Libraries have the opportunity to engage more proactively with teachers and learners

Librarians have natural partnerships with subject domain and IT experts

Libraries and librarians need to better understand how social networks and information seeking styles contribute to learning and teaching

The Toppling Silos of the Disciplines and Mind-Boggling New Forms of Scholarly Communication

The Toppling Silos of the Disciplines and Mind-Boggling New Forms of Scholarly Communication

Osamu Shimomura, 2008 Nobel Prize, Chemistry

Born: Kyoto

Fields: Organic chemistry, marine biology, medicine

-- How does the library help him create

new knowledge?

-- What are his information seeking/sharing

behaviors and preferences?

-- In what ways does the library

serve his colleagues and his graduate

and post-doctoral students?Wikimedia Commons

File: Osamu_Shimomura-press_conference_Dec_06th,_2008-2.jpg

And Then There’s Today’s (and Tomorrow’s) StudentAnd Then There’s Today’s (and Tomorrow’s) Student

Tech-savvy

Nimble

Enthusiastic

Achievement-oriented

“We’re special”Waseda studentsBy: Montauk Beach

http://www.flickr.com/photos/thomaslevinson/14100202801

A New Kind of LibraryA New Kind of Library

Engage with the knowledge community

Make library collections more visible where users are

Move to next generation systems and services

Users and Libraries:

United on the Network

THE COLLECTIONSTHE COLLECTIONS

To know the future of collections metadata, we need to know the future of collections

What Is “The Collection”? What Is “The Collection”?

“[T]he stuff of cultural heritage collections, digital assets, pre-print

services and the open Web, research labs, and learning management

systems remains for the most part outside the scope of the catalog.

Scholarly information objects now include digitized rare and

historical materials, textual primary source materials, graphical images,

materials described in institutional and disciplinary repositories,

conference Web sites, scholarly Web sites … data sets, software,

simulations, a rising array of multimedia resources, learning objects

and courses—the list goes on.”

Calhoun, Karen. The changing nature of the catalog and its integration with other discovery tools. Washington, DC: Library of Congress, 2006.http://www.loc.gov/catdir/calhoun-report-final.pdf

What is the first thing people think of when they think of a library?

What is the first thing people think of when they think of a library?

Source: Perceptions of Libraries and Information Resources, OCLC, 2005, question 807.Note: The percentage is based on the number of comments received divided by the number of respondents. Some respondents chose to provide more than one response, and all responses were included.

http://www.oclc.org/reports/pdfs/Percept_pt3.pdf

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Computer FilesScoresSound RecordingsMixed MaterialsMapsVisual materialsSerialsBooks

WorldCat by Type of Material Described, 1999-2008

Source: data from OCLC annual reports describing WorldCat bibliographic and holdings data

What Did Users Say They Want? (2002)What Did Users Say They Want? (2002)

• Faculty and students do more work and study away from campus

• Loyal to the library, but library is only one element in complex information structure

• Print still important, but almost half of undergraduates say they rely exclusively or almost exclusively on electronic materials

• Seamless linking from one information object to another is expected

• Fast forward to 2010: these trends many times stronger!

Do you use electronic sources all of the time, most of the time, some of the time, or none of the

time?

0%

10%20%

30%

40%50%

60%

All of thetime/most of

the time

Some of thetime

None of thetime

Responses

Per

cen

t

Faculty/Graduate

Undergrad

http://www.clir.org/PUBS/reports/pub110/contents.html

Expenditure on E-Resources: 2008 ARL AverageExpenditure on E-Resources: 2008 ARL Average

E-mtls51%

All other49%

ARL

Research into use and users of digital library collections

Research into use and users of digital library collections

“Digital libraries, far from being simple digital versions of

library holdings, are now attracting a new type of public,

bringing about new, unique and original ways for reading

and understanding texts.”—BibUsages Study 2002

“The availability of primary sources has been crucial for the success of my teaching in history. Students have remarked what a difference it has made, and I have noticed a big difference between this course with the availability of online primary resources to those I have taught before that were based on printed resources.” –History instructor, University of California [2]

Usage of University of Wisconsin Digital Collections 2001-2008 [1]

R2 = 0.9701

0

1000000

2000000

3000000

4000000

5000000

6000000

7000000

8000000

9000000

10000000

2001 2002 2003 2004 2005 2006 2007 2008

Mill

ion

s o

f S

essi

on

s/U

ses

“The function of searching across collections is a dream frequently discussed but seldom realized at a robust level. This paper … discusses how we might move from isolated digital collections to interoperable digital libraries.”

—Howard Besser

Digital library collectionsDigital library collections

Digitized text• Books, newspapers, journals …

Digitized images• Photos, drawings, posters …• Sheet music• Maps

Sound and moving images• Recordings, oral histories• Film

Born digital materialScholarly preprints, data sets, dissertationsArchived Web sites

Digitized ImagesDigitized Images

Set of three large-size (35.7×25.5cm) color woodblock prints (nishiki-e)

Artist: Toyohara Kunichika

Publisher: Fukuda Kumajiro

Published in 1890

Owned by the National Diet Library

Rising Interest in Digital Collections on the BnF and LC Web Sites

Rising Interest in Digital Collections on the BnF and LC Web Sites

Source: Alexa.com, 15 Nov 2009

Where do people go

on bnf.fr and

loc.gov?

BnF:

Expositions: 30%

Catalogue: 26%

Gallica: 26%

LC:

American Memory: 41%

Catalog: 17%

Legislative information

(THOMAS): 6%

Open Access JournalsOpen Access Journals

Japanese Journals in WorldCat.orgJapanese Journals in WorldCat.org

http://jstage.jst.go.jp

Linking Out to Full Text Journals and ArticlesLinking Out to Full Text Journals and Articles

“knowledge base”“knowledge base”

A knowledge base is a set of data about electronic journals and ebooks.

It is used primarily to direct users of a library to the places where they can read full text content.

It is typically used as a part of a link resolver, such as

• Serials Solutions 360Link

• Ex Libris’ SFX

• WorldCat Link Manager

Federated

Search

Link

ResolverERM A to Z list

KB KB KB KB

Federated

Search AdminLink Resolver

Admin

ERM

Admin

A to Z list

Admin

User 1User 2 User 3

Librarian

1

Librarian

2

Librarian

3

Traditional knowledge base management

Federated

Search

Link

ResolverERM A to Z list

WorldCat knowledge base data

WorldCat knowledge base Admin

Knowledge base API

Librarian

1

Librarian

2

Librarian

3

User 1 User 2 User 3

WorldCat knowledge base management

The WorldCat Knowledge Base: Coverage of Japanese JournalsThe WorldCat Knowledge Base: Coverage of Japanese Journals

Total: 23,335

Estimated number of unique journals across

all Japanese collections: 8,590

Institutional Repositories in JapanInstitutional Repositories in Japan

Open Access Repositories Gaining Visibility and ImpactOpen Access Repositories Gaining Visibility and Impact

Sources: Alexa.com 5 September 2010

and the Cybermetrics Lab’s ranking of the world’s top repositories

(disciplinary and institutional) at

http://repositories.webometrics.info/about.html

2009-2010 Traffic and

Rankings Compared:

*Kyoto University Research

Information Repository (#38)

*arXiv.org (#5)

*Social Science Research

Network (#4)

Japanese Institutional Repository Metadata in WorldCat: OAIsterJapanese Institutional Repository Metadata in WorldCat: OAIster

OAIster harvesting

OAIster and OCLC WorldCatOAIster and OCLC WorldCat

• Aggregation (union catalog) of harvested open access collections

• OAI-OMH

• 25 million records, 1100 contributing institutions

• Began at University of Michigan; now managed by OCLC

• Japanese contributors to OAIster (at least 8 institutions, including Waseda)

• Going to self-service contribution model (Digital Collections Gateway)

OCLC’s Digital Collections GatewayOCLC’s Digital Collections Gateway

Self-service tool for uploading to WorldCat

Freely available to members and non-members of OCLC

Makes your digital content more visible by end users who search WorldCat.org, Google, Yahoo, and other popular Web sites

Compatible with all OAI-compliant repositories

More information: http://www.oclc.org/gateway/about/default.htm

COLLECTIONS METADATACOLLECTIONS METADATA

Where metadata comes from (and will come from)

Metadata Is ChangingMetadata Is Changing

B.W. (Before the Web)

• For finding and managing library materials (mostly print)

• Catalog records (well-understood rules and encoding conventions)

• Shared cooperative cataloging systems

• Usually handcrafted, one at a time

A.W. (After the Web)

• For finding and managing many types of materials, for many user communities

• Many types of records, many sources

• Loosely coupled metadata management, reuse and exchange services among multiple repositories

• Mix of manual and automated creation and metadata extract, conversion, mapping, ingest and transfer services

Where Metadata Comes From (and will come from) Where Metadata Comes From (and will come from)

•Library cataloging

•Publishers, vendors, aggregators

•Publication supply chain data (ONIX)

•Abstracting and indexing services

•Authority, classification data, terminologies

Professionally produced

•Institutional repositories

•Scholarly portals (e.g., arXiv.org)

•Tags, reviews, lists, etc.

Author/User contributed

•Knowledge bases

•Algorithmically-created indexes

•Author identity pages

•Facets for topics, places, events

•FRBR Work Sets …

Mined

Algorithmically produced,

re-used, harvested

BEYOND THE RECORD: METADATA FOR PEOPLEBEYOND THE RECORD: METADATA FOR PEOPLE

Photo: Kenzaburo OeBy: Amao

Wikimedia Commons

WorldCat IdentitiesWorldCat Identities

Swimming (as opposed to drowning) in a sea of metadata

Advice for SwimmersAdvice for Swimmers

1. Cooperate (don’t go it alone)

2. Use a blend of metadata techniques to:

• Create many paths to your collections (print, licensed, digital)

SynchronizeSyndicate

• Call attention to a wide array of collections on behalf of your communities (not just your own holdings)

• Manage metadata at the collection level when feasible and appropriate

LIBRARY COOPERATIONLIBRARY COOPERATION

“Being where their eyes are”

• A global network for connecting people with libraries

• National library loading to WorldCat

Outward Integration of collectionsOutward Integration of collections

“Integration should be outward rather than inward, with libraries seeking to use their components in new ways”

--Calhoun, LC report, p. 37

300 of the most influential websites,

positioned on the greater Tokyo-area train map.

http://informationarchitects.jp/web-trend-map-2008-beta/

“Being Where Their Eyes Are”: Embedding the Library in the Web“Being Where Their Eyes Are”: Embedding the Library in the Web

Data Synchronization and Syndication Data Synchronization and Syndication

WorldCat &

WorldCat Partners…

Data synch

Other partners

Flickr Commons

Synchronizing “Group” or “National” and “Local” catalogsSynchronizing “Group” or “National” and “Local” catalogs

National or regional

union catalogs

Transfer protocol

Central Library

District Library

Tech School Library

Transfer protocol

Territory Library

Design School Library

WorldCat.org

and PartnersCompleting the cycle:

driving searches back to the libraries

An Example: Starting with GoogleAn Example: Starting with Google

Find in a library link

Driving the search through WorldCat.org … to a library near youDriving the search through WorldCat.org … to a library near you

When Began Participating Number of national libraries

1975-2006 19

Since 2007 13

Languages of Materials Represented in the WorldCat Bibliographic Database

Languages of Materials Represented in the WorldCat Bibliographic Database

46%

54%

2009

English Non-English

Records for resources published in Japan in the WorldCat bibliographic database as of 30 August 2010

Records for resources published in Japan in the WorldCat bibliographic database as of 30 August 2010

Books All other material types0

500000

1000000

1500000

2000000

2500000

3000000 2855616

173814

2357630

54596

Published in JapanWith CJK Script

All other material types = continuing resources, visual materials, maps, sound

recordings, scores, computer files, archival materials

Loading the Records of the National Diet Library (Project in Progress)Loading the Records of the National Diet Library (Project in Progress)

• Number of records to load: about 4.2 million

• Forecast of those new to WorldCat: At least half

• The records new to WorldCat will be added to the over 2 million records for Japanese publications already in WorldCat

• The process: Teamwork between NDL staff and OCLC staff in the Leiden (Netherlands) and Dublin (USA) offices

• Transliteration and normalization of the data

• Conversion to MARC21

• Completion of set-up and load to WorldCat

Test NDL Record in Connexion (OCLC cataloging interface)Test NDL Record in Connexion (OCLC cataloging interface)

IN CONCLUSIONIN CONCLUSION

Bringing writers, readers, and libraries togetherBringing writers, readers, and libraries together

• Switch users from where they find things on the Web to library-managed collections of all kinds

• Infrastructure to permit global, national or regional, and local discovery and delivery of information among open, loosely-coupled systems

• Strengthen national research information / knowledge creation infrastructures

• Local catalogs, repositorites, cultural heritage collections linked to a chain of services on the network

• Representation of ALL the collections of interest to the communities libraries serve: physical, electronic, digital

We Can Be Connected: With Our Communities and With Each OtherWe Can Be Connected: With Our Communities and With Each Other

National Diet Libraryhttp://www.flickr.com/photos/superciliousness/37590237

And more …

Thank You!Thank You!

Karen Calhoun

calhounk@oclc.org

Waseda bear mascot

By: yuttamichael

http://www.flickr.com/photos/yuttamichael/892061