1 Ircam Artistic Testbed Jerome Barthelemy, Ircam

1

Ircam Artistic Testbed

Jerome Barthelemy, Ircam

2

Summary

• Artistic production context in Ircam– Music production with digital components since the 70ies– An example– Structure of our objects– Risks and Good practices

• The preservation issue– Definition of Representation Information– PDI– Assessing authenticity

• Authenticity in the CASPAR framework• Some examples of authenticity protocols

• Work done and future work– MustiCASPAR server– Scenario and implementation – demonstration and RepInfo validation– Repository current status– Future work

IRCAM context

• IRCAM : musical production since the 70es• Development of audio/music digital processors :

– 4A, 4X…. (Hardware)– Max/MSP (Software)– Audiosculpt, Modaly, OpenMusic… (software)

• Musical creation using audio digital processing– 450 works created since 77

• Problem of preservation recognized since the middle of the 80s, but no formal approach– Production of documentation (from 80es to 2000 - in paper)– From 2002 : digital storage of documentation and digital

objects : Mustica

3

IRCAM context

• GOAL of preservation in this context :– To be able to REPERFORM the works– Not simply record audio files !– To make possible interactions between human

performers and digital processes :• Preserve the processes themselves, not the results• What processes?

– Digital instruments (audio effects, like reverberation, harmonizers…)

– Encoded in the form of « software »

4

Example

• Diademes by M.A. Dalbavie– Creation in 1986, – Written for solo viola, acoustic orchestra and live electronic– Live-electronic part :

• 1 YAMAHA TX816 : FM-Synthesis for 2 Yamaha KX88 keyboards• 2 YAMAHA SPX1000 : Effects/Transformation of the viola:

Harmonizer & Panning• 2 YAMAHA REV5 : Effects/Transformation of the viola:

Reverberation & Echo

– Several new performances between 1986 and 1995– No new performance since 1995 (but several attempts), due to

issues related to FM synthesis component– A new performance on 2008 (December – New York)

5

Example

• Diademes by M.A. Dalbavie

6

•Live-electronic part : –1 YAMAHA TX816 : FM-Synthesis for 2 Yamaha KX88 keyboards–2 YAMAHA SPX1000 : Effects/Transformation of the viola: Harmonizer & Panning–2 YAMAHA REV5 : Effects/Transformation of the viola: Reverberation & Echo

Example

• Diademes by M.A. Dalbavie

7

•FM Synthesis : – Modulation of a waveform by another waveform (both in the audio range) – Original implementation by Yamaha (under patent)– Results in « complex » sounds, not produceable by another means

The general case

8

Preservation data

Data are composed of:- Data files (mainly audio)- Processes

(hardware or software based)Plus some information files:

- Sketch plans- Instruction files…

The whole set defines a « work » (in addition to the musical score)

The general case

9

MIDIMIDI

MaxPatch

MaxPatch

MIDIMIDI

MaxPatch

MaxPatch

instructions

equipments

information

A more specific case :

The general case

- We can consider the process as a kind of « musical digital instrument » that is to be preserved (as for acoustic instruments)- Obsolescence is very fast (for example, comparing to audio files)- New performances imply generally a migration (or a porting, or emulation) of the process- Most important question : authenticity

Migration

The general case

11

Good practices on Authenticity :- Record samples in input- Record audio output

In order to keep track of the intended use of the process.

CASPAR cope with current good practices !

The preservation issue

• Preservation – Preserve the process itself (not the recording)– Preserve the data files (audio files)– Preserve all documents that are related to the work – Preserve the logic (the relation ship between all the

elements)

12

The preservation issue

• About the process (the « patch »)– At the very core of the work– Encodes the logic of the relationship of the different

elements (HW, like microphons or speakers, and data files)

– Preservation :• Very difficult, since the process can be considered a

software• But fortunately, a specific kind of software, most of the time

encoded with specific software like Max/MSP, PureData, or Reaktor, all based on graphical programming

• In Ircam, generally based on Max/MSP

• The major risk

13

RepInfo for real-time process

14

• Representation Information :– Structure: Block-diagram flow– Semantics of elements : already existing in

documentation

• Methodology :– Reduce the block-diagram flow to an algebra

• Choice of existing « FAUST » language, concise and sufficiently expressive (developed by Grame, Lyon, France)

• Work in progress, to be developed in future project

– Store the semantics of the elements • Extract them from existing documentation

RepInfo for real-time process (and PDI)

15

DOC tool : extracts from existing documentation the semantics of elementsFUNC tool : parses the code of each single process in order to identify elements, verify existence of RepInfo (if RepInfo missing, a warning is generated), and provides PDI for the process according to patch ontologyFILE tool : analyze the global structure of all provided files, encodes PDI of work according to provided ontologyLANG tool : reencodes the syntax of the original process

Preservation Description Information

• Based onto CIDOC-CRM (ISO 21127:2006) (cidoc.ics.forth.gr)

• Ontologies templates expressed in RDF for each element we have to preserve :– Work– Real-time process :

• Patch• Library• Function

– Documents : Booklet (documentation), Hall program, Biography, Interview, audio sample, Video sample, Score, Recording…

16


• Ontology template for work:

17


• Ontology template for patch:

18

Authenticity

• Authenticity is a process (Mariella Guercio, University Urbino)

• CASPAR defines Authenticity Protocols (AP)– Each one composed of several steps (AS)

• Must be executed at different steps according to the lifecycle of the object (access, migration…)– Each execution is composed of different execution steps

(ASE)– The overall result should lead to an evaluation of the

authenticity

• Defined accordingly to a Designated Community

19

Authenticity

• Need to attach different AP to different steps of the lifecycle. Example:– AP1 <-> Migration of component– AP2 <-> Maintenance

• Different AP for context– Dependent on the work in which is used

(example of FMSynthesis)– Dependent from the point of view of community

• Developers• Musical assistant• Curators

20

Authenticity protocol #1

• Case : maintenance (audio file)– Compute fixity– Verify provenance

21


22

• Case : migration of audio file, from format f1 to format f2• AS1 - before migration : verify semantics (is it an audio file) ?

– Yes : continue– No : failure - apply another protocol suitable to the right semantics

• AS2 - verify file not compressed– Yes : continue– No : uncompress (or return to analog!)

• AS3 - before migration : verify availability of source sampling rate and sampling size in target format f2

– Yes : continue– No : failure - apply another protocol

• AS4 - after migration : compute fixity of sample size and sample rate


• AS5 - after migration : compute fixity of Pulse Code Modulation from audio files


Failure - apply another protocol

Compression?

Sample rate?

Sample size?

Migration

Fixity sample rate & sample

size

Fixity PCM

Decompress




Audio file?


• Case : migration of an audio effect (AFX1 -> AFX2)– Ingest phase (AP1):

• AS1 : Define a list of audio samples in input (using predefined audio samples 1khz sinusoid…)

• AS2 : Apply to them the audio effect AFX1, and store the output audio samples

• AS3 : Define the comparison features (with range) that are to be validated when executing AP2

– Migration phase (AP2) :• AS1 : Apply to input audio files the migrated audio effect AFX2 and store

the results• AS2 : Compare the audio samples resulting from AFX2 to those resulting

from AFX1, according to features defined in AP1

23

24

Ircam testbed scenario

Start point : « Max/MSP no longer available »

25

SUB-SCENARIO #1

• Ingest of a new WORK and components

STEPS

Collect DATA for the WORK

For each COMPONENT of the WORK {

Search the REGISTRY for existing CPID using features

Chose best compliant existing RepInfo CPID

If no one CPID is found {

Register / Log into RepInfo toolbox portal

Use Analysis tools to produce RepInfo for the COMPONENT

}

Else {

Retain and use CPID

}

Analyze DATA object and produce AP and PDI objects

Construct AIP

Store AIP

Notify the PRESERVATION ORCHESTRATION MANAGER (POM) for meta-data added

Store DATA into DATA STORE

}

Analyze WORK and produce AP and PDI objects

Construct AIP of the WORK

Store AIP of the WORK

Notify the PRESERVATION ORCHESTRATION MANAGER (POM) for meta-data added

26

SUB-SCENARIO #2

• Notification of loss of availability for a COMPONENT STEPS

Register / log into PRESERVATION ORCHESTRATION MANAGER (POM)

Search the REGISTRY for the old existing CPID (CPIDold)

If CPID is not found {

Register / Log into RepInfo toolbox portal

Use Analysis tools to produce RepInfo for old existing COMPONENT

Store RepInfo into REGISTRY

Notify the PRESERVATION ORCHESTRATION MANAGER (POM) for change of RepInfo

}

Notify the PRESERVATION ORCHESTRATION MANAGER (POM) for loss of availability of the COMPONENT

27

SUB-SCENARIO #3

• Search for equivalent COMPONENTs

Strategy 1 – Search on the basis of provenance (all porting of existing component) STEPS

Search the KM for existing CPID using provenance

If no one CPID is found {

Notify the PRESERVATION ORCHESTRATION MANAGER (POM) for loss of availability of the COMPONENT

}

Else {

Notify the PRESERVATION ORCHESTRATION MANAGER (POM) for availability of new version for the COMPONENT

}

Strateg y 2 – Equivale nt comp one nt on the basis of ontologica l Re pInfo STEPS

To be de fine d late r if ne ede d

28

SUB-SCENARIO #4

• Ingest of a new version of a COMPONENTSTEPS

Receive the ALERT from the PRESERVATION ORCHESTRATION MANAGER (POM) containing the CPID of existing COMPONENT (CPIDold), CPID of a new version of a COMPONENT (CPIDnew) and the AUTHENTICITY PROTOCOL (APnew).

Search the REGISTRY for the old existing CPID (CPIDold)

If CPID is found {

Retrieve the associated AUTHENTICITY PROTOCOL (APold)

If AP exists {

Verify existence, consistence and correct application of AUTHENTICITY PROTOCOL (APold)

If AP is “equivalent” {

Search all the WORKS, belonging to the managed DATA STOREs, using the COMPONENT (CPIDold).

For each WORK using COMPONENT (CPIDold) {

Retrieve existing ARCHIVAL INFORMATION PACKAGE (AIP)

Retrieve DATA of new COMPONENT (CPIDnew)

Ingest a new WORK after substituting the old COMPONENT (CPIDold) with new COMPONENT (CPIDnew)

}

Notify the PRESERVATION ORCHESTRATION MANAGER (POM) for availability of new version for the COMPONENT

}

Else {

Notify PRESERVATION ORCHESTRATION MANAGER (POM)

}

}

Else {


}

}

Else {


}

29

Demo movie

• Based onto the MustiCASPAR server

RepInfo validation

• Done in 3 steps :– 1 Completeness of information : reconstruction of

original process from extracted Representation Information

– 2 Usefulness : construction of an equivalent process, from extracted RepInfo , but executed from PureData (equivalent to a migration)

– 3 Authenticity : comparison of audio outputs, according to defined Authenticity protocol

30

Reconstruction of original process

31

RepInfo

Verification of completeness of information

Migration

• Usefulness of Representation Information :– Ability to reconstruct a new process under a different

environment (that is to say, PureData instead of Max/MSP)– Some possible loss of information and inconsistencies :

manual adjustments to be made

32

RepInfo

Authenticity

• Authenticity protocol #3

• At Ingest phase, an authenticity protocol was defined, with 3 steps :– Choose an input audio file (inputFile1)– Apply audio effect on it– Record output audio file (outputFile1)

33

Authenticity

• Authenticity protocol #3

• At Ingest phase, an authenticity protocol was defined, with 3 steps :– Choose an input audio file (inputFile1)– Apply audio effect on it– Record output audio file (outputFile1)

• Migration Authenticity protocol :– After migration, apply new audio effect on inputFile1– Record output audio file (outputFile2)– Compare outputFile1 and outputFile2 (by hearing – audio engineer, or any other method of comparison,

for example comparing spectrograms)

34

≠ ?migration

Authenticity

• Migration based on Representation Information was not sufficient.• Adjustments have to be made, particularly on the sliders in order to achieve authenticity

– Semantic RepInfo is not sufficiently defined on some parameters (the reason why there are sliders that have to be manually adjusted!)

35

36

Repository status

• From current repository Mustica to MustiCASPAR :– Preliminary analysis of the whole content of the repository

– Several common file formats have been found inside, including zip, dmg, rar and ISO CD.

– Reconstruction of PDI possible for works (but not for parts of works, as explained below)

• Migration of « Jupiter » by Philippe Manoury from Mustica to MustiCASPAR– Purpose of the test : to evaluate the amount of work of a complete

migration for the whole content

– Most important Issues :• Missing informations about provenance and context

– Need to identify persons able to provide this information

• Unexpected files– Representation Information missing

– PDI missing

Achievements

• MustiCASPAR server• Methodology for extraction of Representation Information• CIDOC-CRM based ontologies for expression of PDI• Scenario definition and implementation• Evaluation of current state of repository• Validation :

– Ability to build on Authenticity Protocol and Representation Information for migration

– adequation of CASPAR and OAIS model to current practices and current expression of documentation

– efficiency : Information about the current state of the repository (RepInfo, PDI missing…)

37

38

NEXT DEVELOPMENTS

• Middle term (until end of 2012):– Two research projects (funded by National Research Agency)

• First one focused on RepInfo production• Second one more focused on PDI (tracking provenance) – with

INA and UTC (CASPAR Partners), and EMI Music

– Development of MustiCASPAR server• Specialized towards specific User Communities

– ASTREE : Max/MSP developers – GAMELAN : audio production – …

– Integration MustiCASPAR with current repository• Production of adequate (missing) Representation Information• Production of adequate (missing) PDI