Web Pages Seamless Flow Analysis Phase Programme Board December 2004

Web Pages

Seamless FlowAnalysis PhaseProgramme Board

December 2004

Web Pages

Seamless FlowAnalysis Phase

Agenda

1. Introduction

2. State of play

3. The products

• Process model

• Data model

• Requirements catalogue

4. Summary and close

Web Pages


Terms of ReferenceObjectives

1. Provide the business scoping for the Seamless Flow constituent projects, in terms of high-level requirements.

2. Provide a starting point for projects to perform detailed analysis and design, through initial process and data modelling.

Deliverables

• High-level requirements catalogue.

• Detailed process model, using the high-level process model from the previous (initiation) phase as a starting point.

• Seamless data model, covering the scope as defined by the requirements and process model.

Web Pages


Initiation Kick-off FinaliseAnalysis Analysis

Collatematerial

RequirementsCatalogue

Process / DataModels

Sharewith

Board

PRODUCTS

Consolidate

ReviewWS

ReviewWS

ReviewWS

ReviewWS

ReviewWS

How the plan fits together…

ModellingWS

ModellingWS

ModellingWS

ModellingWS

ModellingWS

Reqt’sWS

Reqt’sWS

Reqt’sWS

Themes

Reqt’sWS

Reqt’sWS

Web Pages


(the life cycle of a record)

1. Record Usage at OGD

5. Technology Watch

4. Resource Discovery and Presentation

3. Preservation and

Maintenance2. Ingest record

TNA

Sources Paymentsystem

Recordcopying

FOITrackingsystem

Users

Customerregistration

‘PRONOM’suite Externals TNA staff

PaperprocessingTNA staff

Inventorysystem

ClosureControlsystem

Orderingsystem

OGDs

Papersystem

Process Model – Top level

Web Pages


1. Record Usage at OGD

1.1 Set disposal criteria

(OGD)

TNA1.6 Check for

and obtain FOI exemptions

(OGD)

1.4 Selection

1.3 Appraisal

1.5 Sustain record (OGD)

2.2 Physical transfer

1.2 Create new record (OGD)

1.8 TNA technology

watch advice

1.7 Apply FOI redactions where required (OGD)

1.9 Prepare for transfer (OGD)

‘Wild West’Starting point

RecordMetadata(‘WildWest’)

RecordMetadata(Non ‘Wild West’)

RecordMetadata(‘Wild West’)

Custody policyRecord freeze(Semi-Current – 10-30)

RecordMetadata(migrated / preserved)Record

Metadata Checkapplication(Close ifapplicable)

FinishedRecords(releasable orRedacted)

Biz class’n / file planStandards & guidanceOSPsCustody policyInventory

File planFiling guidance

Non ‘Wild West’Starting point

Appraisal criteriaOSPsTechnical report (DPD)

Appraisaldata

Enhanced technical reportMetadata

Advice Recordrepository /inventory

RecordMetadataAccession decision

RecordMetadata

(business usage of record prior to transfer to TNA)

Web Pages


2. Ingest Record

2.5 Record Salvage

2.3 Pre-Accession

loading

2.2 Physical transfer

1.9 Prepare

for transfer

3.1 Update archival

descriptions

2.1 Prepare for transfer

2.4 Reconciliation

OGD

OGD

Liaison(Agree date,Mechanism, media)

Metadata for preliminary lookNotification, incl.Security level info

Paper records(hybrid accession)

Papersystem

Invalid record

Notification of receipt(incl. ‘incomplete’,‘surprises’, etc.)

Approval to load

Record, metadataMediaSupporting material

Accession idConsignment list Approval to load

Record, metadata

5.1 Identify

technical charact’n

Alert (problems)

Archival descriptiontrigger

Notification of validity

Recommendation

3.2 Store record and metadata

RecordMetadata

1.9 Prepare

for transfer

Metadata(for closed records)

(making sure we have a record to work with)

Web Pages


3. Preservation and Maintenance

3.2 Maintenance

3.1 Accessioning

2. Ingest Record

5.Technology

Watch

RecordMetadata

4. ResourceDiscovery

andPresentation

RecordMetadata

Query, triggerMigrated records Record details

Records

Presentation copyPresentation copy

(working with the record)

Web Pages


3.1 Accessioning

3.1.2 Store recordand metadata

3.1.4 Transferpresentation

copies topresentation areas

3.1.1 Updatearchival

descriptions

Stage 1 notification

Orderingsystem

3.1.3 Identify / Createpresentation copies

Trigger back thatlodging ‘completed’

Mandatory metadatafields completed(Accession route)

OGDs

Base preservation copy+ metadata to manage /describe objects

Presentation copy(Accession route)

‘Available’

Release trigger to cataloguefor paper records

3.1.6 Completionof accessioning

2.3 Pre-Accession

loading

Archival description triggerMetadata

2.3 Pre-Accession

loading

2.4Reconciliation

RecordMetadataApproval

to proceed

Paperprocessing

Paper record details(part of hybrid)

Inventorysystem

Presentation copies

4.4 Supply TNA

record

Presentationcopy

Stage 2 notification

Trigger back that Paper lodging complete

3.2Maintenance

RecordMetadata

3.1.5 Controlclosure

Metadata, Record

Updatedmetadata

Closurecontrolsystem

Request

Closureinfo

Updates(prior to agreeingTransfer date)

Web Pages


3.2 Maintenance

TNA staff

3.2.7 Generatenew copies

3.2.3 Identify / Createpresentation copies

3.2.1 Updatearchival

descriptions

3.2.2 Store recordand metadata

OGDs

Migrated record

Base preservationcopy


copies topresentation areas

3.1Accessioning Technology watch driven

need for change (record request)

3.2.5 Controlclosure

Metadata, Record

Updatedmetadata

Closurecontrolsystem

Request

Closureinfo

3.2.8 Redactionupdates

Updates(prior to agreeingTransfer date)

Presentation copies

4.4 Supply TNA

record

Presentationcopy

Redaction

Redactiondetails

3.2.9 ControlledTNA access Viewable copy

OGDs

Request forclosed records

Closed records

Metadata

Update cataloguetrigger

Open recordtrigger

3.2.6 Extract copy

Record

5.5 (Auto)migration

Record

Metadata

5.1 Identify Technical

characterisation

RecordMetadata

Records(to migrate)

5.5 (Auto)migration

5.2 Identifychanges

and impact onrecords stored

Record query

List of recordsaffected (of type)

Newmanifestations(migrated records)

Web Pages


4. Resource Discovery and Presentation

4.3 Payment

4.4 SupplyTNA record

4.2 Requestrecord

4.1 Discovery and search

Agreed estimateMethod of payment

Distant Selling RegulationsContact details

SourcesOther records(Portal / link)

‘Contact us’

Recordcopying

Tools feedback

Paymentsystem

5.1 Identifytechnical

characterisation

4.5 Add resource discovery metadata

FOITrackingSystem

User

3.1.6Completion ofaccessioning


copies toPresentation

areas

ResultsRecord Ref

TNA staff

RecordMetadata

Method of accessChosen search result

Access request

Receipt

Record

Trigger productionof delivery copy

Financialinterface

Hyperlinks(from other web sites)

Updates

Presentationcopy

5.5 (Auto) migration

New versionavailable

5.4 Technology

refresh planning

“Cannot do”

Request

Request

Users

Customerregistration

User entrypoint

Request

Registration

(record delivery to the users)

Web Pages


5. Technology Watch

5.1 Identifytechnical

characterisation

Externals

5.5 (Auto) migration

5.4 Update technical

information

5.2 Identify changes and

impact on records stored

5.3 Technology refresh planning

4.4 Supply TNA

record

4.1 Discovery and search

3.2.2 Storerecord andmetadata

Technical info

Exceptionreport Technical

info

Problems(no ‘viable plan’)

AlertMetadata

Impact / Requirements

Request

“Cannot do”

Migration pathwaySelected recordsMetadata

Load new migrated Manifestations back(migrated records)

New versionavailable

RecordMetadata

2.3 Pre-Accession

loading

Updates totechnical repository

Alert(problems)

Enhanced metadata

Exceptionsfor resolution

‘PRONOM’suite

TNA staff

Migration QA


Recordquery

List of records affected(of type)


Recordrequest(trigger)

Records(to migrate)


(Ensuring future record accessibility)

4.4 SupplyTNA record

Tools feedback

Request

“Cannot do”

Web Pages


Data model

Metadata

OGDBusiness

Unit (Series)

Search Result

Customer

Order Holding Body Object

Record

Accession

Plan

Record Component

Medium / Delivery Channel

Deliverable / Presentation

Unit

Manifestation

Action Audit

Access Control

Policy

Selection Criteria

Division

SupportingDocumentation /

Resource

Record Breakdown

ReasonTechnical

Info

Pathway

Validation Action

Exemption Info

Access Info

Notification

Application System

TNA Staff

Record Migration

Action

Technical Info Source

Manifestation Type

2

4

5

3

1

Web Pages


TNA Objectives 11. To develop effective and efficient management of active records to

support government business operations

2. To improve the selection of public records for permanent preservation for the benefit of the public and of government

3. To ensure that our policies for records selection are socially inclusive

4. To make arrangements for the permanent preservation of government digital records of enduring historical value; and to ensure that archives on traditional media are kept in accordance with national and international standards

5. To extend the range and improve the quality of our online public services

6. To improve the quality and efficiency of our onsite public services

Process requirementscontribution to the objective

1, 4, 5

1, 4

1, 4, 5

3, 4, 5

1, 2, 3, 4, 5

3, 4, 5

Web Pages


TNA Objectives 27. To promote the public records as a national information and

learning resource to the widest possible audience

8. To raise an awareness of our holdings among under-represented groups in our user profile by means of a rigorous social inclusion programme

9. To promote awareness of, and access to, archival sources relating to British history

10. To develop standards and guidance for archive and records management professionals and to promote and monitor their use

11. To contribute to the implementation of the Government Archives Policy Action Plan and to monitor progress towards its objectives

12. To raise the public profile of The National Archives

Process requirementscontribution to the objective

1, 2, 3, 4, 5

3, 4

4

4

5

4, 5

?!

?!

Web Pages


Requirements Summary 1

Record Usage at OGD

• Disposal / selection criteria

• Partial transfers

• Advice and guidance

• Ongoing validation of disposal criteria

• Taking in redactions

• Secure storage for FOI

• Facilitate OGD migrations

Web Pages



Ingest Record

• Record reference

• Transfer mechanism / custody notifications

• Security classification aspects

• Check records and metadata (and quarantine)

• Air-gapped systems

• Progress / status reporting

Web Pages


Requirements Summary 3Preservation and Maintenance

• Archival description

• Closure checks

• Automated metadata loading

• Processing records and notifications

• Referential integrity of records and metadata

• Capacity planning

• QA and audit trail

• Management of multiple copies

• Back-up and recovery

Web Pages


Requirements Summary 4Resource Discovery and Presentation

• Resource discovery metadata

• Various access methods (e.g. browse and search)

• Metadata and full record text search

• Record ordering and copyright issues

• Chargeable vs. non-chargeable

• KIS

• Display, help, etc. features

• View on-line or at Kew only

• Controlled access to closed records

Web Pages



Technology Watch

• Formats guidance to OGDs

• Automated format identification and validation

• Technical object inter-relationships

• Generate obsolescence alerts

• Migration plan with various stages, or pathways

• Migration plans should support automated migrations

• Reporting and control of failed migrations

• Process controls

Web Pages


• TNA records management monitoring

• Custom requests and delivery options

• Pricing and payments• Login accounts• Added value services• One shopping basket

• Dummy presentation copy for testing

• Audit trails• Multiple migration pathways• External refereeing re

quality of data (archivists / specialists)

• Multiple formats• Manual checks

Requirements Out of Scope

Web Pages


More Requirements Out of Scope?

• Advice to OGDs on version control for redactions

• Encourage reduction in formats

• Accessibility and disability

• Backtrack manifestation migrations (dead-ends and

failures)

• Choice of delivery (display / download)

• Tools for OGDs (format id) and externals (format info)

• Separate preservation / presentation migration plans

Web Pages


Summary

• Process model– Understand detail (inputs, outputs, controls, resources), incl. threads– Descriptions make up the Seamless Flow process

• Data model– Record vs. metadata– Project specific vs. programme-wide– Attributes are a starter

• Requirements catalogue– Developed detail– Some outstanding questions– 443 (8% OOS – over half in Resource Disc & Presentation)

Web Pages


Process : Project basic mapping

1. Record usageat OGD

2. Ingest record

3. Preservationand maintenance

4. Resourcediscovery andpresentation

5. Technologywatch

Departmental FOI ActivitiesAppraisal and SelectionManagement of Semi-current Records

Ingest Process (Transfer to TNA)

PreservationCataloguing (Metadata, archival description)Management and Security

Cataloguing (Resource discovery)Presentation Portal

Technology Watch

Bus

ine

ss C

han

geT

rain

ing

Web Pages


Going forward

• Final feedback

• Project scopes / profiles

• Address outstanding issues

• Information Architect

• Programme (and project) planning

Web Pages

Thank You

Documents

Web Pages Seamless Flow Analysis Phase Programme Board December 2004