Real music libraries in the virtual future: for an integrated view of music and music information

Contactforum: Digitale bibliotheken voor muziek. 3/6/2005

Real music libraries in the virtual future: for an integrated view of music and music information

Michael FingerhutIrcam – Centre Pompidou

Paris, France

2

The new challenges facing librariesand their users

Need to organize and provide access to:• Collections and

archives

• Physical, ephemeral and virtual objects

• Local and remote

• Audiovisual, symbolic and textual data

• Data and metadata (and metametadata, metametametadata…)

Approach• Domain-specific

(music information)

• Share resources

• Provide structured and efficient search and navigation tools• In collections

• In objects

• Provide appropriation tools (remember, analyze, explain)

pattern extraction, form recognition…

synchronization, summarization

indexation

pitch, voice extraction

work

publicationspaper

symbolic

OCR

score

about

sound

audio programs

text

fingerprint

structure

taste, mood

user

musicology

data

multimediaaudio

data

rights owner

DR

M

annotation

performer

composer

performance (live, recorded)

info

rmat

ion

mu

sic

sto

red

dat

a

metadata

genre

automatic

playlists

librarian

lyrics

abstract

digital

symbolic

physical

semantic

author

interaction

cognitive, social

theory

feature extraction,

identification

motives

conc

ept

→ s

ign

→ s

igna

l →

si

gn

→

conc

ept

textual

features

characteristicsmusical

acoustical

interpretation similarity form orchestration

melody key harmonypolyphony tempo instrumentation

chordssegments dynamics rhythm

…durationpitch timbre intensity spatial room

analysis

A (simplified) map of music information

similarity

metadata

4

Library catalogue

Calendarof concerts

Common search point

OAI protocol

(name, title, instrumentation, date, place…)

Whe

re

Audio concertarchives

List

en

When,

where

Biographies,articles

Concertnotes

Re

ad

Read

Federating internal contents and information sources

Webreference

5

OAI protocol

(name, title, in

strumentation, date, place…)

Federating contents and information sources of distinct organizations

Organization1

Organization3

Common search point

Organization2

6

Technical information Characteristics

• Distributed catalog vs. directory

• Domain-specific search vs. googling

Sharing metadata• Distributed search

• Z39.50, ZING…• “Real-time”

• Harvest metadata• OAI (Open Archive

Initiative)• Light

How to share data (full-content search)• Share the indexes

Underlying software: SDX• Open source search engine and

publishing framework for XML documents

• Includes OAI• Browse, search, thesauri• http://adnx.org/sdx/

Current projects• Sharing contemporary music

resources within and across institutions

• Sharing ethnomusicological collections across institutions

7

Example: ethnomusicological portal

8

Displaying musical structure and navigating within a document

9

Displaying semantic structure of a sound recording and navigating within it

10

Annotating multimedia documents for analysis and comparison

Arbitrary multipage background

Arbitrary shapes, images and text

Layers (overlays) allowing selective display at play time

Timed events Automatic

synchronization with different audio files using cue points

Cursor (forward, backward)

Play same annotation with distinct audio (video) files

Multiwindow (Automatic tools)

11

Efficiency (and rights): produce music excerpts via automatic summarization

all 1:47:35 A small bit of each segment

each 0:14:82 One segment of each type

longest 0:03:25 Longest segment

most freq. 0:03:25 Most frequent segmentsimilarity matrixThe warmer the color (red = warmest), the more similar

1. Find which parts sound like other parts (timbre similarity, e.g.)

2. Extract segments

3. Produce summaries

6

5

4

3

2

1

Courtesy of Geoffroy Peeters, Ircam

0m0s

0m0s

5m25s

5m25s

12

The new librarian

Less hard-core cataloguing• Automatized

procedures

Not just “google” collecting• Collection building

• Selection of documents

• Validation

Valorization• Virtual works

• Virtual collections

Physicaland virtualmediation• VRML, etc.

• Informationspecialist