Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
11/1/11
1
CompMusic: Computational models for the discovery of the world’s music
Xavier Serra Music Technology Group
Universitat Pompeu Fabra, Barcelona (Spain)
ERC mission: support investigator-driven frontier research.
CompMusic is funded with an ERC Advanced Grant for a period of 5 years and with a budget of 2,5 million Euros.
Current IT problems
• IT research does not respond to the world's multi-cultural reality.
• Data models, cognition models, user models, interaction models, ontologies, … are culturally biased.
• Music information is not just CDs and metadata.
Taxonomy of musical information
(Lesaffre, 2005)
11/1/11
2
Computational music modeling
Sound and Music Computing
Music Information Processing
Computational Musicology
Hum
an-Computer
Interaction
Ontologies
Cognitive Musicology
Cognition models
Interaction models
Data models
CompMusic objectives
• Promote a multicultural approach to IT research.
• Advance in the description and formalization of music to make it accessible to computational approaches.
• Reduce the gap between audio signal descriptions and semantically meaningful concepts for music.
• Develop data modelling techniques for different music repertories.
• Develop computational models to represent culture specific musical contexts.
• Design culture driven music discovery systems.
Proposed approach • Combination of academic disciplines: Computational
Musicology, Cognitive Musicology, Music Information Processing, Music Interaction.
• Combination of methodologies: qualitative and quantitative; scientific and engineering.
• Combination of information sources: audio features, symbolic scores, text commentaries, user evaluations, etc…
• Combination of music repertoires: Indian (hindustani, carnatic), Turkish-Arab (turkish, andalusian), Chinese (han).
• Combination of cultural perspectives: Research teams and users immersed in the different music cultures.
Why these musical repertoires?
• Belong to formalized classical traditions with strong influence on current society.
• Musicological and cultural studies available.
• Alive performance practice traditions.
• Exists within active social/cultural contexts.
Possibility to challenge current western centred information paradigms.
11/1/11
3
Indian music Melodic structure: Raga
Rhythmic structure: Tala
Texture: monophonic
Style: pre-composed and improvisatory.
…
Hindustani: Ravi Shankar
Carnatic: Sudha Ragunathan
Turkish-Arab music Melodic structure: Maqam
Rhythmic structure: Wazn
Texture: Monophonic
Style: pre-composed and improvisatory.
Ottoman classical music
Andalusian classical music
Han Chinese music Melodic structure: heptatonic (not pentatonic!!)
Harmony: five harmonies
Rhythmic structure: duple
Texture: Polyphonic
Liu Ji Hong, Erhu concerto
CompMusic tasks
11/1/11
4
Task 1: Music repertoires
Gathering and organizing audio recordings, metadata, descriptions, scores, plus all the needed contextual information.
References: • MTG-DB framework (MTG-UPF)
• http://www.raaga.com; http://chinasite.com; http://www.listenarabic.com; http://www.turkishmusicportal.org; …
• Open data movement: Wikipedia, Musicbrainz, Wikibooks, Wordnet, DBLP Bibliography, DBTune, Geonames, …
• Resource Description Framework (http://www.w3.org/RDF/)
• Grid Computing
Task 2: Musicological framework
Musicological studies to understand the chosen repertories within their cultural context.
References: • Tonal pitch space theory (Lerdahl, 2001)
• Performance studies (Gabrielsson, 2003)
• Embodied cognition (Leman, 2008)
• Humdrum toolkit (http://humdrum.ccarh.org)
• Rasas in Indian art (Rangacharya, 2010)
Task 3: Music ontologies
Building the ontologies needed for annotating the gathered collections.
References: • The music ontology specification (http://musicontology.com)
(Raimond, 2007)
• Community-based ontologies (Mika, 2006)
• Knowledge management and metadata (Pachet, 2005)
• http://musicbrainz.org; http://wordnet.princeton.edu
Task 4: Audio description
Audio content analysis to describe the music collections chosen.
References: • Essentia & Gaia framework (MTG-UPF)
• Music transcription (Klapuri & Davy, 2006)
• Top-down and knowledge-based processing (in Klapuri & Davy, 2006)
• Computational auditory scene analysis (Wang & Brown, 2006)
11/1/11
5
Task 5: User profiling
Characterization of users and communities, modelling their musical preferences and behaviours.
References: • Social Computing (Chai et al., 2010)
• Theory of music preferences (Rentfrow & Gosling, 2003)
• http://www.last.fm; http://freesound.org
Task 6: Music interaction
Interaction models by studying user behaviour in musical tasks.
References: • Cultural Computing (Nakatsu et al., 2010)
• Information Foraging Theory (Pirolli, 2007)
• Interactive Information Retrieval (Cole et al. 2005)
• Table-top interfaces (Reactable)
Task 7: Music discovery
Active models and systems for culture-based music discovery.
References: • Collaborative creativity
• Online learning (Moh et al., 2008)
• Recommendation systems (Celma, 2009)
• http://freesound.org; http://last.fm
Conclusions
Big and challenging !!!!
But hopefully we can contribute with our music research to develop better
IT for our multicultural world.
11/1/11
6
References (1 of 3) • Celma, O. 2008. Music Recommendation and Discovery in the Long
Tail. PhD thesis.
• Cole, C., et al. 2005. "Interactive information retrieval: Bringing the user to a selection state". in A. Spink & C. Cole (Eds.), New directions in cognitive information retrieval. Springer.
• Chai, S., et al. (Eds.) 2010. Advances in Social Computing. Springer.
• Gabrielsson, A. 2003. “Music Performance Research at the Millennium”. Psychology of Music.
• Klapuri, A., Davy, M. (Eds.) 2006. Signal Processing Methods for Music Transcription. Springer.
• Leman, M. 2007. Embodied music cognition and mediation technology. The MIT Press.
• Lerdahl, F. 2001. Tonal Pitch Space. Oxford University
References (2 of 3) • Lesaffre, M. 2005. Music Information Retrieval: Conceptual framework,
Annotation and User Behaviour. PhD Thesis.
• Mika, P. 2006. “Ontologies are us: A unified model of social networks and semantics”. Web Semantics: Science, Services and Agents on the World Wide Web 5, no. 1 (March): 5-15.
• Moh, Y., Orbanz, P., Buhmann, J. M. 2008. "Music Preference Learning with Partial Information". ICASSP 2008.
• Nakatsu, R. et al. (Eds.) 2010. Cultural Computing. Springer.
• Pachet, F. 2005. “Knowledge Management and Musical Metadata”. Encyclopedia of Knowledge Management.
• Pirolli, P. 2007. Information Foraging Theory: Adaptive Interaction with Information. Oxford, Oxford University Press.
References (3 of 3) • Raimond, Y. 2008. A Distributed Music Information System. PhD
Thesis.
• Rangacharya, A. 2010. The Natyasastra. Munshiram Manoharlal Publishers.
• Rentfrow, P. J., Gosling, S. D. 2003. “The Do Re Mi's of Everyday Life: The Structure and Personality Correlates of Music Preferences”. Journal of Personality and Social Psychology 84, no. 6: 1236 -1256.
• Wang, D. L. and Brown, G. J. (Eds.). 2006. Computational auditory scene analysis: Principles, algorithms and applications. IEEE Press/Wiley-Interscience.