Developing a Governmental Long-term Archive Management System on Semantic Grid

姜國輝 MIS/NCCU

Dr.-Ing. Johannes ChiangeDeveloping a Governmental Long-term

Archive Management Systemon Semantic Grid

Johannes K. Chiang

National Cheng-Chi University, Taipei

[email protected]

The project is partially funded by

National Archive Administration, TW

姜國輝 MIS/NCCU

Dr.-Ing. Johannes Chiange

A feel on long-term archives

姜國輝 MIS/NCCU


Background of the Development

• Phase 1: Digitalizing and reformatting of doc’s and Archives

• Phase 2: Text and Meta-Searching on the Web• Phase 3. SRB/SRM, Knowledge-based DataGrid* • Phase 4: Long-term archive on Semantic GRID

*R. Moore is the one who coined the term

姜國輝 MIS/NCCU


Objectives and Requirements• Objectives:

– to facilitate the long-term Preservation of various kinds of documents, viz. archive, independent from the evolution and changes of time, techniques and digital environments.

• Requirements:– Integration of Storage management and Information Ma

nagement– Secured Preservation of Data, Metadata, Indexes, etc. (v

alue-added Information)– Effective search to resources and efficient storage/acces

s on data– Consistent User-Interface– Recovery drawing on co-location back-up, dynamic regu

lation on authentication and security management.

姜國輝 MIS/NCCU


General Archive Management Process by Governments

Access

Disposal

Archiving

Security

Appraisal

姜國輝 MIS/NCCU


Access Manage

ingest

KM Process

Local Appraisal Local A

rchivin

g

Development/

InnovationAcquisitionIdentification

Utilization Deployment/

DistributionRetain

Cen

tral Arch

iving

Central Appraisal

Local Access

Central Access

Disposal/Transfer

Re-AppraisalUtilization

Records

Knowledge-based Management Spiral for e-Archives

姜國輝 MIS/NCCU


R e t r i e v a lI n t e r f a c e

T e x t R e t r i e v a l

T e x t D i s p l a y D i r e c t o r y L o o k u p

D i r e c t o r y F i l e s

I n d e x A c c e s s

I n d e x F i l e s

F u l l t e x tS c a n

P r i m a r y A c c e s s

M a r k u p T e x t F i l e sP a g eF i l e s

U s e rI n t e r f a c e

S e a r c hE n g i n e

D a t a b a s e

Source: Simon Lin, ASCC

Baseline 1:Architecture of a digital Archive System

(Multi-lingual, i.e. Kanri and Roman text)

姜國輝 MIS/NCCU


Registry/Repository

Network Transmission Protocol(TCP/IP)

Specific AP

Service

Information Model

CPP

Secu

rity

Message TRP

Ingestion Management Access

RD

FX

M L

X-

Qu

ery

AIP

/H

DF

Posix I/O

Rule-based Mgt.

Information–based Mgt.

Attribute-basedQuery

RelationsBetweenConcepts

Knowledge

Repository for

Rules

Knowledge or

Topic-basedQuery

Feature-basedQuery

Infor.

Repository Storage

(Replicas,

Persisten

t IDs)

FieldsContaine

rsFolders

Attributes

Semantics

Know-ledge

Information

Data

Secu

rity

Registry/Repository

Service

InformationModel

CPP

Message TRP

Specific AP

Baseline 2: Collaboration Framework Knowledge based DataGrid and SOA

OW

L

姜國輝 MIS/NCCU

Dr.-Ing. Johannes ChiangeSemantic GRID and Semantic Service Matching

SemanticWeb

Grid(Services)

Semantic Grid (Services)

Metadata Annotation

Metadata Annotation

ServiceConsumer

ServiceProvider

Services

Matching

Engine

Ontology

Social actions wh. In 1950s in TW.

Interior Records

Social actions, Living,.. etc are in Interior Records

姜國輝 MIS/NCCU


Methodology for SA/D: ZACHMAN Framework

姜國輝 MIS/NCCU


Results of Analysis based on Z. Framework Entities Activities Locations

What How Where

Planner

National e-Records

Local Long-term e-records

Ingestion Management Access

National Archives Administration

Local Archive Admin. Agencies, Counties

Scope

Owner

Records -Persistent Records -Short-term

Records Memo, Reports

Knowledge

Appraisal Archiving Access Disposal

Central Archives Central Supportive

Archives Deep Archives Local Archives Local Supportive

Archives (geo-locations)

Enterprise Model

Designer

Record Schema Service Specificat

ion Metadata -Structural -Descriptive -Operational

Capture Arrangement/ D

escription Preservation Disposal/Trans-f

er/Re-appraisal Query/Access/U

tilization

Core Systems Main Archive Systems Supportive Archive

System Deep Archive System Connection System with

local administrator. Interface with other

systems

System Model

姜國輝 MIS/NCCU


Results of Framework Analysis (con’t)Entities Activities Locations

What How Where

Builder

Data ModelInformation ModelsMessage ProtocolsAuthorization Records

Data/Information/Knowledge captureRelative interactions between semantic piecesProcess InteractionConsistency proceduresAuthorization procedures

Logical Deployment of Components

Technical Model

Sub-contractor

Document FormatsData TypesInterfaceEncoding RulesMessage Package FormatsEtc.

Metadata creation and managementDocument format transformationTag mappingPhysical address allocationStorage resource mgt.Message package and transmissionData and information flow between functional blocks

Physical component topologyReal-life data

Components

Data Functions Network

姜國輝 MIS/NCCU


Paradigm Shift to SOA Methods to define Processes and Services

姜國輝 MIS/NCCU


Authentication for Resource Sharing

User Roles:1. System Administrators2. Validation Authorities3. Schema Generators4. Document Generators5. Official Reader6. Reader in Public (Citizen)

User Right to Resources=:Documents+ Services

姜國輝 MIS/NCCU


Our Archive GRID Security Infrastructure

PKI(Taiwan’s

e-Gov. Project)

SSL/TLS

Proxies and Delegation

姜國輝 MIS/NCCU


Federation / Collaboration/Synchron.

SRM-Client

SRM SRM

MassStorage

6.FTP(pull mode)

Replica Catalog

Network transferof DATA

1.DATA Creation 2. SRM-PUT

Network transfer

3. Register

Node 0

Replica Manager

Node 1archive files

4.SRM-COPYNode0 to Node1

5.SRM-GET

archive filesstage files

SRM

Node2

Network transfer

9.FTP (push mode)

8.SRM-PUT

7.SRM-COPYNode1 toNode2

SRM-Client

Retrieve data for application10.SRM-GET

UsersSRM-Client

Network transferof DATA

Backup Mechanisms

MassStorage

MassStorage

Backup Mechanisms

姜國輝 MIS/NCCU


Web ServerApplications

Core System

Registries/ Repositories

Info. Components

Service Profiles

BS

Tech. Components

Submitting organisation

Creation /submission

Connector

Submitter

Forms

BS

BrowserValidation

Validator

Group

Central Archive

Standard Reg/Rep

Candidate CCs

BrowserGeneration

Doc. (PDF)

XML Schema

UML/XMI(techno neutral)

Guest users

RetrievalBrowser

EDIFACT docs

ASN1 docs

Other docs

Validating organisationGuest users

(dev. team, IS designers…Standard Maker

Validation authority

Submitting Organisation

Overview: Operatiponal Architecture

Metadata Repositories

姜國輝 MIS/NCCU


Experience learned and future works• Digital archives on Semantic Grid can be stored and used in a distri

buted manner, managed in a centralised Catalog/Registry as a whole. Each agency has its own right to manage its storages.

• Maintaining the consistency (spatially, temporally) of the archiving storages is the most important task for the long-term preservation.

• Access on replicas leads to better efficiency and reliability.• Authentication (roles) management is most crucial for ingestion on t

his long-term Archive-system.• Metadata herewith embraces data and logic relations between docu

ments (content) but also descriptions of physical environments such as format, viewer services used, etc.

• Name space and naming conventions are of great importance for long-term archiving and have to be robust. Using the mapping between logical and physical file-names allows maintaining the dependency of archives, viz. files, and storage environments.

• Gov. personnel need training on new skills needed for this re-engineering of Document and Archive Management.

姜國輝 MIS/NCCU


Future works

• Compliant with the mutual PKI and digital signature techniques resulting from the e-Gov. Project of Taiwan.

• Survey on XMI for Metadata Interchange

• Consideration on using RSS and Atom progm’ing for syndicating news as well as the content of governmental sites

姜國輝 MIS/NCCU


Thank YouQuestions ? None or lessTill the next time

Hope in Maui