Upload
neil-george
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
The Digital Object ArchitectureThe Digital Object Architecture
A presentation atA presentation atLouisiana State UniversityLouisiana State UniversityBaton Rouge, LouisianaBaton Rouge, Louisiana
August 26, 2005August 26, 2005
Robert E. KahnCorporation for National Research InitiativesReston, Virginia
Selected Major Network Selected Major Network IssuesIssues
How to get affordable broadband access to How to get affordable broadband access to homes, businesses, government, etc.homes, businesses, government, etc.
How to add more dimensionality to the How to add more dimensionality to the mobile wireless experiencemobile wireless experience
How to take advantage of many How to take advantage of many devices/appliances being on the Internetdevices/appliances being on the Internet
Protecting critical elements (including Protecting critical elements (including infrastructure elements such as DNS)infrastructure elements such as DNS)
Stifling SPAM; detecting and fighting VirusesStifling SPAM; detecting and fighting Viruses
Selected Major Issues (con’t)Selected Major Issues (con’t)
Identity Management (w/o certificates)Identity Management (w/o certificates)Trust in the security mechanismsTrust in the security mechanismsManaging PrivacyManaging PrivacyHow to enable more widespread sharing How to enable more widespread sharing
of important information on the netof important information on the netTrusting your information to the NetTrusting your information to the NetManaging your information on the Net over Managing your information on the Net over
very long periods of timevery long periods of time
Infrastructure DevelopmentInfrastructure Development
What is so hard about it?What is so hard about it? Making it scalable over platforms, size and timeMaking it scalable over platforms, size and time Achieving Critical MassAchieving Critical Mass
Getting Buy inGetting Buy in Pleasing many essential participantsPleasing many essential participants Displacing prior capabilitiesDisplacing prior capabilities Structuring matters to deal with concerns about empire Structuring matters to deal with concerns about empire
buildingbuilding
It’s a lot easier to create brand new capabilities It’s a lot easier to create brand new capabilities than to affect existing means of operationthan to affect existing means of operation
Infrastructure Creation is a Infrastructure Creation is a Subtractive ProcessSubtractive Process
Infrastructure reduces a common, shared Infrastructure reduces a common, shared capability to its basic and essential attributescapability to its basic and essential attributes
These attributes are not always recognized or These attributes are not always recognized or understood up frontunderstood up front
Upon further scrutiny, capabilities are usually Upon further scrutiny, capabilities are usually deleted from a well-conceived architecture deleted from a well-conceived architecture over timeover time
Consensus develops when no more can be Consensus develops when no more can be removed without disabling the infrastructureremoved without disabling the infrastructure
What is the Problem?What is the Problem?
Managing information in the Net over very long Managing information in the Net over very long periods of time – e.g. centuries or moreperiods of time – e.g. centuries or more
Dealing with very large amounts of information in Dealing with very large amounts of information in the Net over timethe Net over time
When information, its location(s) and even the When information, its location(s) and even the underlying systems may change dramatically underlying systems may change dramatically over timeover time
Respecting and protecting rights, interests and Respecting and protecting rights, interests and valuevalue
A Meta-level ArchitectureA Meta-level Architecture
Allows for arbitrary types of information Allows for arbitrary types of information systemssystems
Allows for dynamic formatting and data Allows for dynamic formatting and data typingtyping
Can accommodate interoperability Can accommodate interoperability between multiple different information between multiple different information systemssystems
Allows metadata schema to be identified Allows metadata schema to be identified and typedand typed
Digital Object Architecture:Digital Object Architecture:MotivationMotivation
To To reformulate the Internet architecturereformulate the Internet architecture around the around the notion of uniquely identifiable data structuresnotion of uniquely identifiable data structures
Enabling existing and new types of information to be Enabling existing and new types of information to be reliably managed and accessed in the Internet reliably managed and accessed in the Internet environment environment over long periods of timeover long periods of time
Providing mechanisms to stimulate innovation, the Providing mechanisms to stimulate innovation, the creation of dynamic new forms of expression and to creation of dynamic new forms of expression and to manifest older formsmanifest older forms
While supporting intellectual property protection, fine-While supporting intellectual property protection, fine-grained access control, and enable well-formed grained access control, and enable well-formed business practices to emergebusiness practices to emerge
Objective of the FrameworkObjective of the Framework
Internet objectiveBest-effort Packet Delivery
Heterogeneous
Networks Information Systems
Seamless Interoperability
Networks InformationSystems
Organizing Heterogeneous Systems
Digital Object ArchitectureDigital Object Architecture Technical ComponentsTechnical Components
Digital Objects (DOs)Digital Objects (DOs) Structured data, independent of the platform on which it was Structured data, independent of the platform on which it was
createdcreated Consisting of “elements” of the form <type,value>Consisting of “elements” of the form <type,value> One of which is its unique, persistent identifierOne of which is its unique, persistent identifier
Resolution of Unique IdentifiersResolution of Unique Identifiers Maps an identifier into “state information” about the DOMaps an identifier into “state information” about the DO Handle System is a general purpose resolution systemHandle System is a general purpose resolution system
RepositoriesRepositories from which DOs may be accessed from which DOs may be accessed And into which they may be depositedAnd into which they may be deposited
Metadata RegistriesMetadata Registries Repositories that contain general information about DOsRepositories that contain general information about DOs Supports multiple metadata schemesSupports multiple metadata schemes Can map queries into unique DO specifications (via handles)Can map queries into unique DO specifications (via handles)
What is a Digital ObjectWhat is a Digital Object Defined data structureDefined data structure, machine independent, machine independent Consisting of a set of elementsConsisting of a set of elements
Each of the form Each of the form <type,value><type,value> One of which is the One of which is the unique identifierunique identifier
Identifiers are known as “Handles”Identifiers are known as “Handles” Format is Format is “prefix/suffix”“prefix/suffix” Prefix is unique to a naming authorityPrefix is unique to a naming authority Suffix can be any string of bits assigned by that authoritySuffix can be any string of bits assigned by that authority
Data structure can be parsed; types can be resolved within the Data structure can be parsed; types can be resolved within the architecturearchitecture
Associated Associated properties recordproperties record and and transaction record transaction record containing containing metadata and usage informationmetadata and usage information
Interoperability & Federated Interoperability & Federated RepositoriesRepositories
Create a cohesive interoperable collection Create a cohesive interoperable collection of repository-based systemsof repository-based systems Initially, perhaps, around a core set of Initially, perhaps, around a core set of
projects, content, applications and/or projects, content, applications and/or organizations as in ADL organizations as in ADL
Demonstrate interoperability between Demonstrate interoperability between different repository collectionsdifferent repository collections
Develop procedures to insure continued Develop procedures to insure continued accessibility to key archival information accessibility to key archival information
Repository NotionRepository Notion
Any Hardware & SoftwareConfiguration
Logical External Interface
RAP
RepositoryAccess Protocol
Repository
Digital Object RepositoryDigital Object Repository
RA
P
ClientClient
• Provides distributed Digital Object storage.
• May itself be a Digital Object.
• Provides a dynamic acquisition and execution mechanism for the mobile code that implements the content type operations.
• Exclusively accessed using the Repository Access Protocol (RAP).
DisseminateDeposit
Nesting of RepositoryNesting of Repository FunctionalityFunctionality
CoreStructureContentAggregation &De-aggregation
Core Interface must be present at each levelOther levels could be separately defined later
Repositories & Digital ObjectsRepositories & Digital Objects
REPOSITORY
IPv6
Each DigitalObject has itsown unique & persistent ID
Content Providerswant to assign Ids
Could be upwardsof trillions of DOsper Repository
Objects may beReplicated inMultiple Repositories
Handle SystemHandle System Distributed Identifier Service on the InternetDistributed Identifier Service on the Internet
First General Purpose Resolution systemFirst General Purpose Resolution system
Can be used to Can be used to locate repositorieslocate repositories that contain digital objects given that contain digital objects given their handlestheir handles - - and more!and more!
Other indirect referencesOther indirect references Public Keys, Authentication information for DosPublic Keys, Authentication information for Dos
Accommodates interoperability between many different information Accommodates interoperability between many different information systems; for examplesystems; for example DNS was demonstrated on the Handle System in preparation for Y2KDNS was demonstrated on the Handle System in preparation for Y2K Can support ENUM, RFID, and moreCan support ENUM, RFID, and more
Attributes of the Handle SystemAttributes of the Handle System
The basic Architecture of the Handle System The basic Architecture of the Handle System is is flat, scaleable, and extensibleflat, scaleable, and extensible
Logically central, but physically decentralizedLogically central, but physically decentralized Supports Supports Local Handle ServicesLocal Handle Services, if desired, if desired Handle resolutions return entire “Handle Handle resolutions return entire “Handle
Records” Records” or portions thereofor portions thereof Handle Records are alsoHandle Records are also
digital objectsdigital objectssigned by the serverssigned by the serversdoubly certificated by the systemdoubly certificated by the system
Resolution MechanismResolution Mechanism
Multiple SitesMultiple Servers
Handle System<www.handle.net>
Handle
HandleRecord
• System is non –nodal• Scaleable & Distributed• Supports global (and local) resolution• With backup for reliability, mirroring for efficiency
Type ResolutionType Resolution
Types are resolvable in the Handle SystemTypes are resolvable in the Handle System Types may be created dynamicallyTypes may be created dynamically Types may be locally named, mapped into Types may be locally named, mapped into
bit strings without semanticsbit strings without semantics Primary prefix zero “0” is used for system Primary prefix zero “0” is used for system
identifiersidentifiers 0.type/<type> is the system handle for type0.type/<type> is the system handle for type Other handles may cross reference this Other handles may cross reference this
handle (e.g. for international use)handle (e.g. for international use)
Handle FormatHandle Format
Prefix Authority
Item ID(any format)
Prefix Suffix
In use, a Handle is an opaque string.
2304.40/1234 Other examples ofHandles
2304/general info2304/12304. HQ/staff2304.1/memo1232304.22.Pub/2004
Direct Access and ProxiesDirect Access and Proxies
DirectAccess
One or moreProxy Servers
IndirectAccess
Redirection of Handle RequestsRedirection of Handle Requests
DirectAccess
DirectAccess
One or moreLocal Handle Services
General Registry of allNaming Authorities
RedirectionInformation
RedirectionInformation
Literary Music Video Financial Grid Enum RFID
“SimpleLookup URL IPaddresses “Unfederated Databases”
Digital ObjectDigital ObjectContent Type(s)Content Type(s) AccessRequestsAccess
Requests
InformationInformation
Digital Object OverviewDigital Object Overview
DisseminationsDisseminations
Unique IdentifierUnique Identifier
Handle
HamletHamlet
It’s a BookIt’s a Book Get Page(2)Get Page(2)
Digital Object OverviewDigital Object Overview
HamletHamlet
Hamlet
•Digital objects are uniquely identified in a given identifier space.
•Data elements reference sequences of typed data.
•A Digital Object can have zero or more Content Types to reflect intended uses by its creator.
•Content Type Operations are accessible as DOs
DataElement
DataElement
DataElement
DataElement
HamletContent Type
Operations
Content TypeOperations
Digital Object OverviewDigital Object Overview
The Digital Object Identifier The Digital Object Identifier (DOI(DOI®®))
Used by the International DOI Foundation Used by the International DOI Foundation (IDF) to reference high-quality materials of (IDF) to reference high-quality materials of publishers (and other owners of IP)publishers (and other owners of IP)
Major Commercial User of the Handle Major Commercial User of the Handle System at present with approximately 12 System at present with approximately 12 Million handlesMillion handles
Usage growing at about 4 Million per yearUsage growing at about 4 Million per year DNS domain names, by comparison, are DNS domain names, by comparison, are
relatively flat with perhaps 40% churn per relatively flat with perhaps 40% churn per year.year.
Setting up a Setting up a Local Handle Local Handle Service...Service...
Download the software from Download the software from http://www.handle.nethttp://www.handle.net
Follow the instructions in the installation script.Follow the instructions in the installation script.
Send your “site bundle”, containing the IP address of Send your “site bundle”, containing the IP address of
your server and your administrator information, to the your server and your administrator information, to the
Global Handle RegistryGlobal Handle Registry®® (GHR) administrator (GHR) administrator
Site is under re-development to accommodate Site is under re-development to accommodate
widespread use via automated meanswidespread use via automated means
Experimental Repository software also available on-Experimental Repository software also available on-
lineline
Managing Rights & InterestsManaging Rights & Interests Not just about copyrightNot just about copyright Terms and Conditions (T&Cs) for use may be Terms and Conditions (T&Cs) for use may be
contained within each DO; also information contained within each DO; also information about intrinsic value, such as monetary valueabout intrinsic value, such as monetary value
T&Cs are intended to indicate clearly what one T&Cs are intended to indicate clearly what one can and/or cannot docan and/or cannot do with a given DO, where with a given DO, where such clarity is intended by the owner of the DOsuch clarity is intended by the owner of the DO
Not an enforcement means, although it may be Not an enforcement means, although it may be used by an enforcement systemused by an enforcement system
Mobile programs that are Digital Objects may Mobile programs that are Digital Objects may apply such terms to themselves and to any apply such terms to themselves and to any digital objects they containdigital objects they contain
Handle-DNS IntegrationHandle-DNS IntegrationDeveloping EnvironmentDeveloping Environment
C/C++, Linux/Windows C/C++, Linux/Windows Additional ModulesAdditional Modules
DNS Interface integrated with handle serverDNS Interface integrated with handle serverCache/Preload ModuleCache/Preload ModuleDatabase Connection PoolsDatabase Connection PoolsC-Version Handle-DNS Admin ToolkitC-Version Handle-DNS Admin Toolkit
Performance ImprovementsPerformance ImprovementsExceptional ProcessingExceptional ProcessingMemory Leak ProtectionMemory Leak ProtectionThread Pool Management Thread Pool Management
Design & Design & ImplementationImplementation
Simple Handle Server Workflow (C-Simple Handle Server Workflow (C-Version)Version)
Storage Management
Interface
Handle Requests
Thread Pool
Listener
Handle Server
Client
Message
Processor
DBDatabase
Connection Pool
External Protocol ConverterExternal Protocol Converter
DNS ProtocolDNS Protocol
Converter
Handle Protocol
53
8000
2641
Handle
Process
Module
Handle S
erver
Latency
Plug & Play InterfacesPlug & Play Interfaces
Integrate DNS Interface with Handle ServerIntegrate DNS Interface with Handle Server
DNS ProtocolDNS Message
Processor
Handle Protocol
53
8000
2641
Handle Message
Processor
Handle S
erver
Cache & Storage ManagementCache & Storage Management Preload (Cache) ModulePreload (Cache) Module
Preload Handle Records from Preload Handle Records from Database into RAM Database into RAM
Reduce Database Access TimesReduce Database Access Times Improve Throughput of Handle ServerImprove Throughput of Handle Server
Storage Management APIStorage Management API User TransparentUser Transparent
RAM or DatabaseRAM or Database Combination of RAM and DatabaseCombination of RAM and Database
Multiple Database InterfacesMultiple Database Interfaces Mysql, PostgreSQL, etc.Mysql, PostgreSQL, etc.
Features of Cache ModuleFeatures of Cache Module Efficient Query PerformanceEfficient Query Performance
STL RBTree, Hash TableSTL RBTree, Hash Table Configurable size of RAM for each Configurable size of RAM for each
Handle Record, or total recordsHandle Record, or total recordsStorage Management API
Storage Management
Interface
RAM
Operations
Create
Modify
Delete
Data Base
Periodic
Update
BenchmarkBenchmarkUDP Interface for DNS ProtocolUDP Interface for DNS ProtocolCompared to BIND 9.3.0Compared to BIND 9.3.0
Handl e-DNS VS Bi nd
0
2000
4000
6000
8000
10000
12000
14000
16000
2 8 14 20 26 32 38 44 50 56 62 68 74 80 86 92 98
Number of Cl i ent Requests(103)
Responses per Second
Handl e-DNSBi nd
Selling infrastructure technologySelling infrastructure technology Providing identification, management and Providing identification, management and
Metadata servicesMetadata services Enabling third-party value-added capabilitiesEnabling third-party value-added capabilities Helping organizations manage their own Helping organizations manage their own
information better & offer new types of servicesinformation better & offer new types of services Stimulating access to “surface information” and Stimulating access to “surface information” and
“embedded information” with appropriate “embedded information” with appropriate access controls and conditions of useaccess controls and conditions of use
Business PotentialBusiness Potential
ConclusionsConclusions
Managing Digital Objects for long-term access is a Managing Digital Objects for long-term access is a key challengekey challenge
Initial Technology Components are available; Initial Technology Components are available; Industry is expected to generate more over timeIndustry is expected to generate more over time
Third-party value-added providers in the private Third-party value-added providers in the private sector will ultimately shape the long-term evolutionsector will ultimately shape the long-term evolution
Interoperability and reliable information access is a Interoperability and reliable information access is a critical objectivecritical objective
A diversity of applications (with user-friendly A diversity of applications (with user-friendly interfaces) need to be developed & deployedinterfaces) need to be developed & deployed
Application Projects have a central role to play in Application Projects have a central role to play in demonstrating the technology and using it effectivelydemonstrating the technology and using it effectively