AV CLASH

Online Tool for Mixing and Visualizing Audio Retrieved from Freesound.org Database

Nuno N. Correia Aalto University, School of Art and Design, Media Lab

Helsinki, 11/2010

1. Introduction

•  Nuno N. Correia - Background: – electronic musician / programmer / designer (and teacher); – studies in international business / innovation management.

• Motivation for the research: – lack of visual feedback in electronic music performances; – lack of interactivity and visual feedback in Internet music.

2. Framework

2.1 Generic Title / Research Question

•  Title of the dissertation: Interactive Audiovisual Objects

•  How to provide an engaging visual representation of sound in an interactive Internet-based art project?

2.2 Main Research Objective

•  Build a series of engaging interactive audiovisual projects for the web, with: increasing integration of sound and graphics; variety of content (maintaining cohesiveness); manipulation capabilities; and customization.

2.3 Secondary Research Objectives

•  Produce •  Present •  Document •  Provide web access to •  Frame •  Explore interfaces of •  Map future uses of

•  interactive audiovisual projects

2.4 Research Methodology

•  Practice-based artistic research

2.5 Fields of Reference •  Media studies •  New media art / internet art •  Visual music / audiovisual art / music videos •  Experimental cinema / interactive narratives •  Film studies •  Sound and music computing •  Interaction design •  Internet and society •  Innovation management •  Synesthesia / perception •  Aesthetics / philosophy of art

Narra$ve   Abstract  

3.  Master  and  Margarita  (2009)  

2.  AVOL  (2007)  

4.  AV  Clash  (2010)  

1.  Heat  Seeker  (2006/8)  -­‐  Aud

io  re

ac$vity

 /  diversity  +  

0.  Idiot  Prince  (2006)    [founda$on  work]  

2.6 Projects Flow Chart / Sandbox

3. AV Clash

3.1 Framing AV Clash

•  AV Clash is the last project to be included in the scope of the research plan.

•  It reflects upon the main aspects of the research question: – integration of sound and graphics; –  variety of content (maintaining cohesiveness); – manipulation capabilities; – customization.

•  It can be considered the culmination of the research.

3.2 About AV Clash (1)

•  A Web-based project by Video Jack (Nuno Correia, André Carrilho and Gokce Taskan), which allows for the creation of audiovisual compositions, consisting of combinations of sound and animation loops.

•  Composed of five audiovisual units, which enable playback and manipulation of four main different loops of sound and audio-reactive visuals.

3.2 About AV Clash (2)

•  Uses sounds retrieved from Freesound.org, an online sound database (www.freesound.org).

•  Aims to develop upon a previous Video Jack project, AVOL (www.videojackstudios.com/avol), by allowing access to more sounds and more manipulation possibilities.

•  Allows for sound uploading, via Freesound.org, using the “avclash” tag.

•  URL : www.avclash.com

3.3 IAVOs

•  The five audiovisual units were named “Interactive AudioVisual Objects” (“IAVOs”)

•  They are composed of user interface (UI) elements that trigger and manipulate sounds, together with animations that react to those sounds.

3.4 Tag Structure

•  Each IAVO has a “tag” associated to it, which is retrieved from Freesound.org.

•  The tag is a filter to the possible sounds and animations it may contain.

•  There are eight animations and several dozens of sounds available per tag.

•  The user may change the four main sounds and animations, and even the tag, of each IAVO, in its “back”.

3.5 IAVO UI Ring (1)

•  The UI is arranged around and inside the IAVO’s “ring”, which allows for dragging and throwing the object around the stage.

•  The ring also incorporates two faders (for volume and audio effect intensity), shaped as semi circles on each of the sides.

3.5 IAVO UI Ring (2)

3.6 Additional Options (1)

• When the user presses the center button of the IAVO, the four selection buttons expand into four larger rectangles, allowing users to pick and adjust sounds and animations for the respective buttons, from drop-down menus.

•  Sliders allow for trimming the sound loop and adjusting the reactivity of the animation to the sounds.

•  The tag can be changed. •  A cycle functionality is also available.

3.6 Additional Options (2)

4. Conclusions

4.1 Conclusions

•  The structure behind AVOL has been developed into a more flexible and diverse project, in terms of : – choice of source sounds (imported from Freesound.org); – choice of animations (although less diverse than sounds); –  audiovisual manipulation (with effects); – playfulness (with the “throw” and “clash” behaviors); – randomness (reloading, “shuffle”); –  sound uploading (via the “avclash” tag).

4.2 Conclusions – future developments

•  However, AV Clash could benefit from: – access to even more content; – more content manipulation capabilities; – implementation of collaboration functionalities; – development of sharing and recording capabilities, allowing users to record, and then share, their sessions.

4.3 Conclusions - Questionnaire

•  Feedback gathered from users, through questionnaires (to be conducted soon), will enrich the conclusions.

5. Related links •  Practice: – www.videojackstudios.com – www.avclash.com

•  Theory: – www.nunocorreia.com

•  Technique: – mlab.taik.fi/mediacode

•  References: – www.zotero.org/ncorreia/items – www.mendeley.com/profiles/nuno-correia