1

An introduction to the NSDL

William Y. ArmsCornell University

2

The NSDL is a program of the National Science Foundation's Directorate for Education and Human Resources, Division of Undergraduate Education.

The NSDL Core Integration is a collaboration between the University Center for Atmospheric Research (Dave Fulker), Columbia University (Kate Wittenberg) and Cornell University (Bill Arms).

The ideas discussed in this talk do not represent the official views of the NSF or the Core Integration team.

Acknowledgement and Disclaimer

3

The NSDL project

1996 Vision articulated by NSF's Division of Undergraduate Education

1997 National Research Council workshop

1998 Preliminary grants through Digital Libraries Initiative 2

1998 SMETE-Lib workshop

1999 NSDL Solicitation

2000 6 Core Integration System projects + 23 others funded

2001 Further collection and service projects + 1 Large Core Integration System project (total about $25 million/year)

2002 Formal release

2006 End of formative phase

4

Continuing questions

(a) Science education: How broadly defined?

(b) Funding: How much with how few dollars?

(c) Education: How can the NSDL have an impact?

(d) Management: How can a diverse community provide shared services?

5

Scientific and technical information in digital form

Materials used in education

Science education: scope of a digital library

Materials tailored toeducation

6

The NSF's strategy

7

NSDL collections funded by the NSF (a) Focused collections

8

9

10

11

NSDL-collections funded by the NSF (b) Aggregates and federations

12

13

14

15

NSDL service projects funded by the NSF

16

17

18

19

Core Integration demonstrations (2000-2001)

20

21

22

The broad view of the NSDL

23

24

25

26

27

All branches of science, all levels of education, very broadly defined:

Five year targets

1,000,000 different users

10,000,000 digital objects

10,000 to 100,000 independent sites

How big might the NSDL be?

28

The NSF cannot fund all collections

29

... to provide a coherent set of services across great diversity.

The Core Integration task ...

30

Resources

Core Integration

Budget $4 million

Staff 25 - 30

Management Diffuse How can a small team, without direct management control, create a very large-scale digital library?

31

A spectrum of interoperability

32

Approaches to interoperability

The conventional approach

Wise people develop standards: protocols, formats, etc.

Everybody implements the standards.

This creates an integrated, distributed system.

Unfortunately ...

Standards are expensive to adopt.

Concepts are continually changing.

Systems are continually changing.

33

Interoperability is about agreements

Technical agreements cover formats, protocols, security systems so that messages can be exchanged, etc.  Content agreements cover the data and metadata, and include semantic agreements on the interpretation of the messages.  Organizational agreements cover the ground rules for access, for changing collections and services, payment, authentication, etc.

The challenge is to create incentives for independent digital libraries to adopt agreements

34

Function versus cost of acceptance

Function

Cost of acceptance

Many adopters

Few adopters

35

Example: textual mark-up

Function

Cost of acceptance

SGML

ASCII

HTML

XML

36

Levels of interoperability

Level Agreements Example

Federation Strict use of standards AACR, MARC(syntax, semantic, Z 39.50and business)

Harvesting Digital libraries expose Open Archivesmetadata; simple metadata harvesting

protocol and registry

Gathering Digital libraries do not Web crawlerscooperate; services must and search enginesseek out information

37

Metadata is expensive

The NSDL cannot afford to create it manually

38

Every collection is different

39

Users

Collections

Metadata repository

The metadata repository

Services

The metadata repository is a resource for service providers.

It holds information about every collection and item known to the NSDL.

40

Metadata strategy

• Support eight standard formats

• Collect all existing metadata in these formats

• Provide crosswalks to Dublin Core

• Expose records in the metadata repository for others to harvest

• Concentrate on collection-level metadata

• Use automatic generation to augment item-level metadata

41

NSDL metadata options

Eight standard formats

Dublin Core Dublin Core + DC-Ed extensionsLTSC (IMS)ADL (SCORM)MARC 21Content Standard for Digital Geospatial Metadata (FGDC)Global Information Locator Service (GILS)Encoded Archival Description (EAD)

For additional information on supported formats:

http://128.253.121.110/NSDLMetaWG/IntroPage.html

42

Services strategy

43

The metadata repository as a resource

• Records will be exposed through Open Archives Initiative harvesting protocol.

• Core Integration team will provide some services based on the metadata repository.

• The architecture encourages others to build services.

44

Information retrieval

Basic metadata search

Basic content search

Combining metadata and content

James Allan, Bruce Croft (University of Massachusetts, Amherst)

45

Portal

Portal

Portal

Search andDiscoveryServices

Content

SDLIP?OAI

http?

How search service fits into the NSDL

Provides search and discovery functionality to portals

Metadata repository

46

Extending the architecture to support federations

Extending the spectrum of search interoperability

collections with non-DC metadata schemasdistributed and heterogeneous collectionsricher search functionality

geospatial search, thesaurus/concept space search, ...

Supporting the creation of new and personalized collections

Providing access to thesaurus and gazetteer services

Terry Smith, Jim Frew (University of California, Santa Barbara)

47

The ADEPT approach to search interoperability

metadata

per collection provider

ADEPT

OAI metadatarepository

harvest

ADEPTcollectiondiscovery

portal

2. harvest &interpret

ADEPTclient

1. map

ADEPT

3. h & i

48

User profiles and authentication

User authentication

User registryAffiliationsPrivacy

User preferences

User Interfaces and portalsEnable customizable user interface

Rights management

Kate Wittenberg, David Millman (Columbia University)

49

Conclusion The NSDL cannot do everything

50

Opportunities for the NSDL

• Categories of material that have been given lower priority by libraries and publishers, e.g., datasets, software, and other dynamic content, ...

• Materials that are accessible for automatic processing, e.g., scientific web sites and databases, image collections, ...

• Materials designed for education, e.g.,learning objects, curricula, problem sets, ...

Less opportunity for the NSDL

• Conventional scientific literature with restricted access