Upload
yago-de-quay
View
223
Download
0
Embed Size (px)
Citation preview
8/8/2019 COST SID - Interactive Music 2.0 - Short Term Scientific Mission Work Plan
http://slidepdf.com/reader/full/cost-sid-interactive-music-20-short-term-scientific-mission-work-plan 1/11
Interactive Music 2.0 User-driven ad hoc dance clubs Short Term Scientific Mission: COST-IC0601 - WG3Art (Interactive Art and Music)
Candidate: Yago de Quay
Host: fourMs Laboratories, University of Oslo – Norway
Dates & Budget: 1st of August – 30th of September 2010; 2500€
Mission:
Investigate how gestural sonic design can be applied in multi-participatory dance
environments.
Introduction:
Much has been learned in the area of interactive music up to now, spawning new areas such as music
information retrieval, generative music, adaptive scores, virtual instruments and the like. However1,2
group movement descriptors and their music information remain un-investigated. This mission
proposes to fill in this research gap, disseminate the work, and promptly implement the results in
public locations.
Alexander Jensenius has been working in the field of musical gesture analyses for quite some time
and his department has materials to support this STSM. In my Home Institution, Carlos Guedes has
been working extensity with generative music, Fabien Gouyon with music information retrieval, and I
1Rolf Inge Godøy, Alexander Refsum Jensenius. Body Movement In Music Information Retrieval. ISMIR 2009
2Camurri, A., Canepa, C., and Volpe, G., Colet ta, P., Mazzarino, Ferrari, N. Social Active Listening and Making of Expressive Music:
The Interactive Piece The Bow is bent and drawn. DIMEA 2008
8/8/2019 COST SID - Interactive Music 2.0 - Short Term Scientific Mission Work Plan
http://slidepdf.com/reader/full/cost-sid-interactive-music-20-short-term-scientific-mission-work-plan 2/11
2
Interactive Music 2.0 Mission|Work Packages|Budget|Timeline
with sound and interactive systems design. Combined, these diverse backgrounds can help usher
interactive systems into popular public venues.
In contrast to existing work, I intend for the output of this mission to be made accessible, inexpensive,
and easy to implement in multi-participative environments. I plan for the final interactive system to
be scalable and permit interplay between users. Furthermore, using social web instruments, I hope to
raise awareness and help the exchange of knowledge this field of research.
Figure 1 - Mission Goals
The work plan will be divided into 5 work packages. Work Packages 2 & 3 will have routine qualitativeand quantitative testing's on available users, and will be inspired by online community feedback.
run
simulation assemble motion
capturing system developgesture
analysis
define
interactive
music
Mission Goals
GDIF generative
composition
music
information
retrieval
design
interaction
model
software
hardware
user –
generated
content web 2.0
approaches
and techniques
beta
testing alpha
testing
usability
testing define user
mappingtechniques
collaborators professors
sample
groups
draftquestionnaire
collect data
8/8/2019 COST SID - Interactive Music 2.0 - Short Term Scientific Mission Work Plan
http://slidepdf.com/reader/full/cost-sid-interactive-music-20-short-term-scientific-mission-work-plan 3/11
3
WorkPackage 1 initial dissemination
Wordpress blog
Facebookprofile
Microblogging
FriendFeed
Advisory Board
WorkPackage 2 theoretical &
methodological
framework
Mappingtechnique
Musiccomposition
strategy
User testingcommunity
feedback
WorkPackage 3motion capture
system
List of hard &software
Installation site
Motion capturesystem
Questionnaireresults
WorkPackage 4final dissemination
Articles
Videos
Images
Talks
Interactive Music 2.0 Mission|Work Packages|Budget|Timeline
WP 1 Where: Home Institution Funding: No
The first package will focus on preparing the dissemination so that we take full advantage of
techniques such as viral marketing and social learning to bring in individuals as well as communities to
influence and increase the reach of this mission’s results, tools, practices and insights.
WP 2 Where: Host Institution Funding: Yes, partial (1650€) Work package 2 will determine the theoretical and methodological modus operandi to be followed
throughout the mission. It will decide the gesture mapping and music composition strategy.
WP 3 Where: Host Institution Funding: Yes, remainder (850€)
This stage concerns the implementation of the motion capturing system, the simulation, and the
questionnaire.
WP 4 Where: Home Institution Funding: No
Work Package 4 is concerned with the analysis and dissemination of the results, the publication of a
paper, and exhibitions.
Figure 2 - Overview of internal and external deliverables
8/8/2019 COST SID - Interactive Music 2.0 - Short Term Scientific Mission Work Plan
http://slidepdf.com/reader/full/cost-sid-interactive-music-20-short-term-scientific-mission-work-plan 4/11
4
Interactive Music 2.0 Mission|Work Packages|Budget|Timeline - WP 1
Work Package 1: Initial Dissemination
_Week 1 _Home Institution _No Funding
“The term "Web 2.0" (2004–present) is commonly associated with web applications that facilitate
interactive information sharing, interoperability, user-centered design, and collaboration on the3World Wide Web.” This STSM will not build a web application, but will mobilize and interact with on-
line communities using Web 2.0 technologies to foster social interaction and exchange between
stakeholders both on-line and off-line.
My strategy is to empower researchers with the necessary knowledge and tools to effectively
implement the methodology themselves, and foster massive participation of user-generated content
from the scientific community. It will include Blogs, group-related or personal online journals, where
people can post ideas, images, and links to other web pages or sites related with their role in this
STSM as well as micro-blogging tools, like Twitter, in a stream of constantly-updated information. Andin addition, other tools include social bookmarking, shared scholarly references and groups, socia
news, and social publishing. This collaborative environment, where users are expected to co-create
the solutions, leads to a natural acceptance by the stakeholders who will feel empowered to test,
evaluate and report their own experience with the new solutions in a controlled social environment.
Centralized development approach
Technology-driven BroadcastTop-Down
Low communication effectiveness
High costs of content production
Low participation
Limited growth
Mission loosely-coupled integration
Demand-driven ParticipationBottom Up / Edge to Core
High communication effectiveness
Very low costs of content production
Exponentially high participation
Unlimited growth
3Wikipedia, http://en.wikipedia.org/wiki/Web_2.0
BroadcastUser Centric
8/8/2019 COST SID - Interactive Music 2.0 - Short Term Scientific Mission Work Plan
http://slidepdf.com/reader/full/cost-sid-interactive-music-20-short-term-scientific-mission-work-plan 5/11
5
Interactive Music 2.0 Mission|Work Packages|Budget|Timeline - WP 1
Figure 3 - Broadcast vs. User centric approaches
This mission is positioned closer to the demand of new content and services related with technology-
enhanced learning where a significant percentage of the content will be generated by intermediate
and end users/researchers. As an example, most of my projects are made publicly available and count
with a growing community of users and contributors.
Figure 4 - (Starting from top, clock-wise) Ethnomusicological Research (yagodequay.blogspot.com); Beatpick Social Tagging
(paginas.fe.up.pt/~mm09011/SDI/index.html); Reason Producers Community (reasonproducers.ning.com); Porto’s Home-lessStories (paginas.fe.up.pt/~mm09021/); Blog (yagodequay.com)
The Advisory Board is a project management instrument that provides an
additional layer of quality by having external experts to the consortium assess
the progress and research outputs of the project. However, the advisory board
has been carefully selected with individuals with strong networks that will assist the project in
fomenting the growth of communities and in identifying early adopters amongst universities and
enterprises.
Advisory Board
8/8/2019 COST SID - Interactive Music 2.0 - Short Term Scientific Mission Work Plan
http://slidepdf.com/reader/full/cost-sid-interactive-music-20-short-term-scientific-mission-work-plan 6/11
6
Interactive Music 2.0 Mission|Work Packages|Budget|Timeline - WP 2
Work Package 2: Theoretical and Methodological
Framework
_Week 2-5 _Host Institution _Partial Funding (1650€)
This initial research step will define the research strategy to be adopted. Considering the various
current forms of music interaction and motion capture, use cases and community feedback will help
steer the research practice. Work Package 2 will figure out the best way to merge the areas of gesture
analysis, music composition and dance. This is important not only for the purpose of this mission, but
for the scientific community at large, for there has been limited research on music information1,2
retrieval from single and, to a lesser extent, group gestures. Furthermore, the theoretical and
methodological model hopes to investigate the following issues: First, the limitation in the method of
music creation. Engineered music, contrary to interactive music, only allows the expert to have a say
on the overall creative direction of a song. Second, non-interactive systems miss out on the plethora
of knowledge and inputs that other users and experts can provide. Third, competition and innovation4,5
in this field barriers to entry imposed by high financial cost lack of an open knowledge community.
This research procures solving these problems by offering an interactive music model for multi-
participative environments. This interactive model is not intended to substitute performers, experts
or engineered content, but intended to add a tool for public interactive media, a tool that can have
benefits on various on education, art and the scientific community.
Below are photos of two interactive music systems that I presented at formal exhibits and danceclubs. They were designed to be inexpensive, intuitive and open for all. The result is a creative outlet
for users and accessibility for other researchers.
4Paul Chin. The Value of User-Generated Content. Intranet Journal. July 3, 2006.
5Tim O'Reilly. What Is Web 2.0, O'Reilly Network. http://www.oreillynet .com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-
20.html. August 2006.
8/8/2019 COST SID - Interactive Music 2.0 - Short Term Scientific Mission Work Plan
http://slidepdf.com/reader/full/cost-sid-interactive-music-20-short-term-scientific-mission-work-plan 7/11
7
Interactive Music 2.0 Mission|Work Packages|Budget|Timeline - WP 3
Figure 5 - (Left) Electro-Gymnasts, Mostra UP – Porto, Februray 2010; (Right) iDance System, Factor J – Vale de Cambra, March
2010.
Work Package 3: Motion Capture System
_Week 6-7 _Host Institution _Remainder Funding (850€)
Considering the formerly mentioned methodology, we will at this point assemble the motion capture
system. Dancers can choose how they interact with media by selecting between body-oriented and
environment-oriented, or a mixture of both input approaches. Currently, body-oriented systems
involve extracting data directly from the dancer such as brain waves, heart rate and skin6
conductivity. Environment-oriented methods position the dancers in a 3D movement sensitive grid
by using motion tracking devices such as video cameras, sonar, and laser beams. Because dance is
mostly involved with fluid body movement, the trend goes towards environment-oriented and non-
intrusive techniques that remove the burden of the dancers carrying wires and devices attached to
their bodies.
In the past I have used rather simple setups using infrared cameras, black light, wiis or simple black
and white cameras to capture motion. Below are pictures of two motion capture system that
designed which respond, with sound effects, to motion. These have systems were presented at
academic institutions and discothèques.
6Robert Wechsler http://www.palindrome.de/
8/8/2019 COST SID - Interactive Music 2.0 - Short Term Scientific Mission Work Plan
http://slidepdf.com/reader/full/cost-sid-interactive-music-20-short-term-scientific-mission-work-plan 8/11
8
Interactive Music 2.0 Mission|Work Packages|Budget|Timeline - WP 3
Figure 6 - (Left) iDance 2.0 – single user feature tracking; (Right) iDance 1.0 – multi-user feature tracking w/ black lights.
List of materials:
Hardware
CCD Camera
Frame Grabber
Laptop
Projector
IR Light
Software
Eyecon
Eyesweb
Max/MSP
Ableton Live
Processing
Data Collection & Analysis:
Questionnaires and opinion surveys will rate music appreciation as well as dance appreciation. The
nature of data will be both qualitative and quantitative.
Qualitative data
• Videos of users interacting with motion capturing system (MOCAP);• Recorded music material created by users;
• Performance tests will evaluate the degree of sensitivity between the subjects interact ion and the
_.affected music. This will fine tune parameters in the music software.
Quantitative data
• Behaviour observation checklists will be used to evaluate dancers' movements and provide
_.information on what movement to capture;
video capture
interaction design; audio procesing
image processing
8/8/2019 COST SID - Interactive Music 2.0 - Short Term Scientific Mission Work Plan
http://slidepdf.com/reader/full/cost-sid-interactive-music-20-short-term-scientific-mission-work-plan 9/11
9
Interactive Music 2.0 Mission|Work Packages|Budget|Timeline - WP 3
• Questionnaires and opinion surveys will rate music appreciation as well as dance appreciation;
• Likert scale will allow participants to self-rate how much control they had on the music.
After the motion capturing system has been implemented, the testing will proceed as follows:
Two random samples will be created from a population of club goers. Demographic questionnaires
will filter subjects to a sample representative of club goers. The sample would then be split in to two
conditions. Under these conditions participants will repeat various sets. Each set has a length of
10min and the music played will have one of three levels of interaction assigned randomly: Non-
interactive, interactive and very interactive.
Under condition A, participants will dance in various sets and rate the experience after every set. This
will be a blind study; they will not be told which sets are interactive.
Under condition B, different participants will dance in various sets and rate the experience after every
set. They will be told which sets are interactive.
The unit of observation will depend on the availability of participants. If there is no time to implement
the questionnaires, plan B would be to do this experiment in my home institution after the STMS.
Figure 7 - Visual representation of experiment sets
Furthermore, systematic usability tests through the mission and community feedback will help steer
the project during the implementation as well as assuring sustainability after its completion.
8/8/2019 COST SID - Interactive Music 2.0 - Short Term Scientific Mission Work Plan
http://slidepdf.com/reader/full/cost-sid-interactive-music-20-short-term-scientific-mission-work-plan 10/11
10
Interactive Music 2.0 Mission|Work Packages|Budget|Timeline - WP 4
Work Package 4: Final Dissemination
_Week 8-… _Home Institution _No Funding
Although dissemination will begin in the first week of the mission, along with the scientific report, this
final step will formalize the results of the mission.
Dates & Budget:
This STSM will start on the 1st
of August and end on the 30th
of September, 2010. The budget is of
2500€. The cheapest housing possible in Oslo is a youth hostel, and a private room costs 56 € per
night. As of right now, airfare from Porto to Oslo prices 300€ and daily predicted expenditures
amount to 30€ per day. The 2500€ financial support barely covers the housing expenses for the 6
weeks; therefore I will have to bear the costs of the travel and daily expenses.
Possible Candidates for Publications
• SMC 2011
• NIME 2011
• ICMC 2011
• Computer Music Journal
• Journal of New Music Research
Exhibitions
• Mostra da UP (March 2011)
• Maus habitos
Conferences and Workshops
• FEUP
FCT
• Further research funding (December)
• PhD funding (March 2011)
Web
• wordpress
• FriendFeed
• Scribd
• Slideshare
• Vimeo & Youtube
• Flickr
8/8/2019 COST SID - Interactive Music 2.0 - Short Term Scientific Mission Work Plan
http://slidepdf.com/reader/full/cost-sid-interactive-music-20-short-term-scientific-mission-work-plan 11/11
11
Interactive Music 2.0 Mission|Work Packages|Budget|Timeline
Figure 8 - Mission Timeline
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8
describe use
gesture taxonomy
assemble mocap
run experiment uestionnaire
dissemination dissemination
music com osition
esture ma in
data anal sis
beta testin
WP1 – initial
dissemination WP1 – theoretical & methodological framework WP2 – motion capture WP4 – final
dissemination
usability testing
al ha testin