68

Implementing a media archive - lessons learned

  • Upload
    viaa

  • View
    31

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Implementing a media archive - lessons learned
Page 2: Implementing a media archive - lessons learned

FOUNDED

21 Dec. ‘12

Page 3: Implementing a media archive - lessons learned

Not (only) a Broadcast

Archive

Page 4: Implementing a media archive - lessons learned

CULTURAL HERITAGE

We don’t own

collections

We provide services towards

50+ org’s

BROADCAST CITY

ARCHIVES

SERVICE PROVIDER

Page 5: Implementing a media archive - lessons learned

STRONGER TOGETHER

Collectie Huis Van Alijn | © Huis Van Alijn

Page 6: Implementing a media archive - lessons learned

DIGITISATION ARCHIVING DISSEMINATION

Page 7: Implementing a media archive - lessons learned

DIGITISATION

Page 8: Implementing a media archive - lessons learned

HERITAGE + MEDIA

HERITAGE

MEDIA

Type materiaal/formaat

# DRAGERS

TOTAAL

UREN

TOTAAL

# dragers aandeel %

# dragers aandeel %

Film 74.173 26.709 11% 89%

Video analog 150.389 228.773 26% 74%

Video digital 225.578 156.189 5% 95%

Audio analogue 165.883 78.370 36% 64%

Audio digital 34.870 18.112 54% 46%

TOTAL 650.893 508.153 21% 79%

INVENTORY ANALOGUE CARRIERS

Page 9: Implementing a media archive - lessons learned

FIRST DIGITIZATION WAVE

Page 10: Implementing a media archive - lessons learned

ALL DIGITIZATION WAVES

UP  TO    450  TB/MONTH  

Page 11: Implementing a media archive - lessons learned

ALL DIGITIZATION WAVES

UP  TO    450  TB/MONTH  

Page 12: Implementing a media archive - lessons learned

WOI  

Evergem's Yzerblad: komt uit de loopgrachten als ’t past (S.l. 1917 – 1918) Collectie Erfgoed- bibliotheek Hendrik Conscience | © Vlaamse Erfgoedbibliotheek – Foto: Stefan Tavernier

WW I Newspaper

project

Page 13: Implementing a media archive - lessons learned

ARCHIVING

Page 14: Implementing a media archive - lessons learned

Slide 14

Scalable, redundant preservation system

Page 15: Implementing a media archive - lessons learned

Slide 15

MAM system

Page 16: Implementing a media archive - lessons learned

DISSEMINATION

Page 17: Implementing a media archive - lessons learned

QUID PRO QUO

Page 18: Implementing a media archive - lessons learned

CP’s

Education

Libraries

Research

TARGET AUDIENCE

Page 19: Implementing a media archive - lessons learned

EDUCATION

Page 20: Implementing a media archive - lessons learned

Slide 20

FROM ARCHIVE

TO CLASSROOM

Page 21: Implementing a media archive - lessons learned

ARCHIVE SYSTEM PROCUREMENT / INSTALLATION

Page 22: Implementing a media archive - lessons learned

BUDGET 2013-2014

11.8 mio EURO

TIME CONSTRAINT

Page 23: Implementing a media archive - lessons learned

TIME CONSTRAINT Evaluation

after 18 months

START JAN 1, 2013

(staff: 1)

Get and keep all customers

on board Start all

services within 1 year

Page 24: Implementing a media archive - lessons learned

VIAA NOW TEAM

ARCHIVE 4 people

14 FTE

Page 25: Implementing a media archive - lessons learned

March 2013 -

Greenfield!

Page 26: Implementing a media archive - lessons learned

lessons learned

Page 27: Implementing a media archive - lessons learned

May 2013 -

Greenfield!

Page 28: Implementing a media archive - lessons learned

Public Procurement

Page 29: Implementing a media archive - lessons learned

surveys towards users

UX Designed according

to their needs

Creates buy-in!

requirements gathering

workhops involving users

Users co-wrote our RFP

Page 30: Implementing a media archive - lessons learned

PRIOR NOTICE

Transparent Procurement Proces

DETAILED  ALLOTMENT  REPORT  (debrief)  

Page 31: Implementing a media archive - lessons learned

EVALUATION JURY

Page 32: Implementing a media archive - lessons learned

Working with a jury

•  Jury consisted of •  Stakeholders (future users)•  International experts•  VIAA staff

•  Why?•  A balanced answer / evaluation•  Jury members add weight & credibility•  Again: user buy-in!

Page 33: Implementing a media archive - lessons learned

3 EU tenders - Timeline of procurement•  June 2013 : prior notice•  August 2 : Candidates invited•  September 12 : Candidates selected•  September 13 : RFP published•  October 17 : Quotes received•  November, 6 : Allotment •  November 25 : Final allotment•  April 2014 : MAM in production

Page 34: Implementing a media archive - lessons learned

Slide 34

Three copies (2 MAM’s)

Page 35: Implementing a media archive - lessons learned

Archive system

Page 36: Implementing a media archive - lessons learned

Slide 36

Zeticon MediaHaven

Page 37: Implementing a media archive - lessons learned

Main MAM services

mul@-­‐tenant  MAM  SYSTEM  

1.  IMPORT  WORKFLOWS  &  INGEST  REPORTING  

3.  EXPORT  WORKFLOW  (OAI-­‐PMH)  ,  EXIT  PATH  

2.  MANAGEMENT      -­‐  Transcoding  (ffmpeg)      -­‐  Par@al  re@eve      -­‐  Metadata      -­‐  Time-­‐based  Annota@on      -­‐  Search  (~SOLR)      -­‐  Manual  QC      -­‐  Storage  Management    

Page 38: Implementing a media archive - lessons learned

Slide 38

Processing capacity & ramp-up

Processing capacity: => up until 13 TB / day (and still scaling)

Page 39: Implementing a media archive - lessons learned

WORKFLOWS

Page 40: Implementing a media archive - lessons learned

Practical MAM use

•  We work along the lines of OAIS for defining•  Ingest process (SIP definition)•  Long term preservation (AIP)•  Dissemination (DIP)

•  Definition in a service agreement•  Practical agreement between VIAA and CP•  Usually for one collection (e.g. audio digitization)

Page 41: Implementing a media archive - lessons learned

Ingest workflow from digitisation

•  We use PREMIS for provenance•  Every step in the process is recorded

digitization firm, CP, VIAA, …  

registration, carrier inspection, digitisation, encoding, …  

Each having a time and outcome + notes  

Analogue carrier or digital equivalent  

Page 42: Implementing a media archive - lessons learned

Complete flow => more than 1 MAM

Page 43: Implementing a media archive - lessons learned

1. Registration of the carrier / AMS

AMS  func@ons:  •  Analogue  carrier  

registra@on  •  PID  crea@on  •  Support  the  

logis@cs  process  •  Technical  

characteris@cs  •  Metadata  from  

digi@za@on  

Page 44: Implementing a media archive - lessons learned

Persistent identifier creation

Page 45: Implementing a media archive - lessons learned

Persistent identifier creation

1  Analogue  carrier  =  1  Intellectual  En@ty  =  1  PID  

Page 46: Implementing a media archive - lessons learned

Persistent identifier

The  PID  is  the  key  for  keeping  track  of  all  events  that  have  an  impact  on  the  digital  object.      We  monitor  the  complete    lifecycle  of  the  digital  object,  from  registra7on  to  dissemina7on  

Page 47: Implementing a media archive - lessons learned

2. Ingest validation

SIP  contains  •  Essence  (MXF)  •  Metadata  (XML)  •  QC  Report  (XML)  

Page 48: Implementing a media archive - lessons learned

2a. Transfer validation

Page 49: Implementing a media archive - lessons learned

2b. SIP validation

Page 50: Implementing a media archive - lessons learned

2c. Storage

Page 51: Implementing a media archive - lessons learned

All  metadata  generated  during  the  ingest  process  is  stored  as  PREMIS.  

Page 52: Implementing a media archive - lessons learned

MAM workflow-Ingest monitoring using PREMIS

Page 53: Implementing a media archive - lessons learned

Ingest monitoring using PREMIS

Page 54: Implementing a media archive - lessons learned

Error handling

Page 55: Implementing a media archive - lessons learned

Error handling (Tableau report)

Page 56: Implementing a media archive - lessons learned

Step 3: Archived

Page 57: Implementing a media archive - lessons learned

 

How  big  is  the  VIAA  archive?  Which  mime  types  do  we  have  in  the  archive?  How  many  items  do  we  archive  /  CP?  

     

   

 

Page 58: Implementing a media archive - lessons learned

grappige  foto  van  VIAA  team  en  dashboard?  

Page 59: Implementing a media archive - lessons learned

Items Archived

Page 60: Implementing a media archive - lessons learned

 

●  539,6  TB  ●  47.273  items  

     

   

 

Size of the VIAA archive

•  539  TB  •  50.000  items  

Page 61: Implementing a media archive - lessons learned

TB archived per CP

Page 62: Implementing a media archive - lessons learned

Step 4: Reuse / interaction

Page 63: Implementing a media archive - lessons learned

Collectie Huis Van Alijn © Huis Van Alijn

Page 64: Implementing a media archive - lessons learned

Road Ahead

Collectie Huis Van Alijn © Huis Van Alijn

Page 65: Implementing a media archive - lessons learned

Road ahead

•  Ingest of born digital material•  Complex and many (50+) data sources•  Need for a ‘SIP creator’•  Realisation through integration with an

enterprise service bus•  Should we be a TDR?

•  Builing along the lines, looking into certification

Page 66: Implementing a media archive - lessons learned

CONCLUSIONS

Collectie Huis Van Alijn © Huis Van Alijn

Page 67: Implementing a media archive - lessons learned

CONCLUSION•  MAM system

•  Took a while to understand business needs•  Found a very flexible partner in Zeticon

•  Perfect MAM system? •  Flexibility through well documented API’s•  Pluggable!•  Interface: Usability – HTML5•  Standards: support for and keep up with –

PREMIS, RDF, DC:TERMS, OAI-PMH, …

Page 68: Implementing a media archive - lessons learned

THANKS!

Collectie Huis Van Alijn © Huis Van Alijn