Fedora Content Models for the National Science Digital Library Data Repository

Fedora Content Models for Fedora Content Models for the National Science Digital the National Science Digital

Library Data RepositoryLibrary Data RepositoryFedora User’s Group MeetingFedora User’s Group Meeting

Copenhagen, September 28, 2005Copenhagen, September 28, 2005Carl LagozeCarl Lagoze

Cornell Information ScienceCornell Information Science

NSDL ContextNSDL Context

A bit of NSDL backgroundA bit of NSDL background

Mission: “Improve Science, Math, Engineering Mission: “Improve Science, Math, Engineering education through digital libraries”education through digital libraries”

Original NSDL solicitation in 1999Original NSDL solicitation in 1999

Over 180 projects fundedOver 180 projects funded

Core integration (Columbia, Cornell, UCAR) Core integration (Columbia, Cornell, UCAR) charged with providing organizational, technical charged with providing organizational, technical infrastructureinfrastructure

CI (Cornell) funding through 2006CI (Cornell) funding through 2006

http://www.nsdl.orghttp://www.nsdl.org

STEMResource

…who used it

…how was it used

…how is it

described &

rated

…how is it

classified

…how does it related

to standard

s

…how has it been

aggregated

…what has it

been used with

Information in ContextInformation in Context

Information Network OverlayInformation Network Overlay

NSDL Data Repository (NDR)NSDL Data Repository (NDR)

Fedora-based implementation of Fedora-based implementation of information network overlayinformation network overlay

Content model to represent NSDL Content model to represent NSDL information entities and relationshipsinformation entities and relationships

Extensive use of resource index and new Extensive use of resource index and new oai service oai service

Fedora NDRFedora NDR

Objects: agents, metadata items, resources, Objects: agents, metadata items, resources, services (metadata providers), aggregationsservices (metadata providers), aggregations

Relationships: metadataFor, providedBy, Relationships: metadataFor, providedBy, memberOf, representedBy + ontology-specificmemberOf, representedBy + ontology-specific

Disseminations: metadata transformationsDisseminations: metadata transformations

OAI harvesting: both static and generated OAI harvesting: both static and generated metadata formatsmetadata formats

Authentication/Authorization: Collections and Authentication/Authorization: Collections and services manage their own repository content, services manage their own repository content, contribution of annotations, new contentcontribution of annotations, new content

NSDL RecommenderService

ExampleCollection

NSDL BigBang

NSDL Agent1000

MDP 3000

Aggr2002 M

4002

NSDL Collections

1002

Aggr2005

M4005

NSDLRecom-mended

1005

NSDL RSAgent 1004

MDP 3004

ExampleAgent 10010

MDP 10011

Aggr10012

Aggr2004

M10005

Example.org

10006

pBy

pBy

repBy

repBypBy

mOf

m4

m4

m4

agg4

mdp4

agg4mdp4

agg4

agg4

1st mOf

repBy

Types of Objects

Agents

Aggregators

Metadata Providers

Resources

Metadata

Types of Relationships

metadataProviderFor (mdp4)aggregatorFor (agg4)providedBy (pBy)metadataFor (m4)memberOf (mOf)· 1st. A recommended resource· 2nd. Makes it a “blessed” NSDL Collection

2nd mOf

M10007

m4

pBy

mdp4

NSDL FEDORA-BASED REPOSITORY

Live DemoLive Demo

http://fedora.nsdlib.org:8380/repository/bighttp://fedora.nsdlib.org:8380/repository/bigBang.htmBang.htm

Metadata in the NDRMetadata in the NDR

Multiple formatsMultiple formats static (ingested from provider)static (ingested from provider) generated/crosswalkedgenerated/crosswalked

Multi-sourcedMulti-sourced de-duppedde-dupped Retain branding of metadataRetain branding of metadata

OAI-PMH harvestingOAI-PMH harvesting

Resources, Metadata, Metadata Resources, Metadata, Metadata ProvidersProviders

Metadata Content ModelMetadata Content Model

format_native_dc

format_native_marc

format_native_dc_info

native_dc nsdlmetadata

bdef

getMetadata(mf)

getMetadataAbout(mf)

DC

native_marc

format_nsdl_dc

format_native_marc_info

Harvest/Ingest Representation

RELS-EXT

Dissemination Exposure

proa

i getRecord(mf)listRecords(mf)

listIdentifiers(mf)

mf values: native_dc, native_marc, nsdl_dc, oai_dc, nsdl_all

proai – Fedora 2.1 OAI Service proai – Fedora 2.1 OAI Service

Old OAI service – harvest only system DCOld OAI service – harvest only system DC

Support for arbitrary metadata formatsSupport for arbitrary metadata formats static data streams and disseminator static data streams and disseminator

generatedgenerated exploits queries to resource indexexploits queries to resource index

proai.properties configurationproai.properties configuration

proai configurationproai configuration

Collections and AggregationsCollections and Aggregations

Set basisSet basis

Semantic basisSemantic basis

Agent associatedAgent associated

Aggregation ModelAggregation Model

Resource

Metadata

Service

Agent

Aggregation

aggFor

repBy metadataFor

metadataFor

metadataFor

memberOf

memberOf

metadataFor

Annotation/ReviewsAnnotation/Reviews

Unstructured metadata about a resourceUnstructured metadata about a resource

Exists as resource and annotationExists as resource and annotation

Separate agent provenance from Separate agent provenance from annotated resourceannotated resource

Annotation ModelAnnotation Model

The SDSC ArchiveThe SDSC ArchiveUses Storage Resource Broker (SRB)Uses Storage Resource Broker (SRB)Monthly snapshots of crawlable contentMonthly snapshots of crawlable contentIdentifies resource as collection of related Identifies resource as collection of related web pagesweb pagesCan’t access protected content, robots.txt Can’t access protected content, robots.txt blocked, etc. – no requirement for NSDL blocked, etc. – no requirement for NSDL projects to participateprojects to participateREST interface for read access (but not REST interface for read access (but not submission – yet)submission – yet)

Integrating SDSC Archive into NDRIntegrating SDSC Archive into NDR

version 1

version 2

version 3

Version 4

SRBArchive

Web Site

getContent(version)