Upload
brent-horton
View
223
Download
0
Tags:
Embed Size (px)
Citation preview
Motion in Sound: Motion in Sound: Designing Sound for Interactive Dance PerformanceDesigning Sound for Interactive Dance Performance
Dr. Dan HoskenDr. Dan Hosken
Associate Professor of MusicAssociate Professor of Music
California State University, NorthridgeCalifornia State University, Northridge
Presented at: Presented at:
ATMI 2006ATMI 2006
San Antonio, TXSan Antonio, TX
September 16, 2006September 16, 2006
PurposePurpose
Present a somewhat simplified and useful Present a somewhat simplified and useful approach to creating for the interactive approach to creating for the interactive dance medium dance medium
Facilitate collaboration between students Facilitate collaboration between students of dance and students of musicof dance and students of music
Objectives:Objectives:
Give an overview of the hardware and Give an overview of the hardware and software components of a camera-based software components of a camera-based interactive dance/music systeminteractive dance/music system
Present a loose taxonomy of motion Present a loose taxonomy of motion parameters and mapping typesparameters and mapping types
Suggest some useful mappings between Suggest some useful mappings between motion parameters and sound element motion parameters and sound element parametersparameters
Illustrate these mappings using examples of Illustrate these mappings using examples of my recent work with the Palindrome IMPGmy recent work with the Palindrome IMPG
General System OverviewGeneral System Overview
Camera trained on dancer(s) is connected Camera trained on dancer(s) is connected to computerto computer
Video Analysis Software abstracts motion Video Analysis Software abstracts motion data in realtimedata in realtime
Motion Data are passed to Sound SoftwareMotion Data are passed to Sound Software Sound Software maps incoming motion Sound Software maps incoming motion
data to sound element parameters in data to sound element parameters in realtimerealtime
Overview w/ bad clipartOverview w/ bad clipart
video computervideo computer audio computeraudio computer
ethernetethernet
Sound Generation SoftwareSound Generation Software
Max/MSPMax/MSP (Cycling ‘74) (Cycling ‘74) PDPD (Miller Puckette)—free! (Miller Puckette)—free! Supercollider (J. McCartney)—free!Supercollider (J. McCartney)—free! ReaktorReaktor (Native Instruments) (Native Instruments) ……and any software that can receive and any software that can receive
data and produce sound in realtimedata and produce sound in realtime
Video Analysis SoftwareVideo Analysis Software
EyeConEyeCon (Frieder Weiss) (Frieder Weiss) EyesWebEyesWeb (eyesweb.org)—free! (eyesweb.org)—free! JitterJitter (Cycling ‘74) (Cycling ‘74)
SoftVNS (David Rokeby)SoftVNS (David Rokeby) CyclopsCyclops (Eric Singer/Cycling ‘74) (Eric Singer/Cycling ‘74) TapTools (Electrotap)TapTools (Electrotap) cv.jit (Jean-Marc Pelletier)cv.jit (Jean-Marc Pelletier) Eyes (Rob Lovel)—free!Eyes (Rob Lovel)—free!
Objectives (redux):Objectives (redux):
Give an overview of the hardware and software Give an overview of the hardware and software components of a camera-based interactive components of a camera-based interactive dance/music systemdance/music system
Present a loose taxonomy of motion parameters and Present a loose taxonomy of motion parameters and mapping typesmapping types
Suggest some useful mappings between motion Suggest some useful mappings between motion parameters and sound element parametersparameters and sound element parameters
Illustrate these mappings using examples of my Illustrate these mappings using examples of my recent work with the Palindrome IMPGrecent work with the Palindrome IMPG
Definitions (1)Definitions (1)
Motion ParameterMotion Parameter: made up of specified data : made up of specified data abstracted from part or all of video, e.g., abstracted from part or all of video, e.g., HeightHeight WidthWidth DynamicDynamic
Sound ElementSound Element: a distinct, coherent sonic : a distinct, coherent sonic behavior created by one or more synthesis or behavior created by one or more synthesis or processing techniques, e.g., processing techniques, e.g., A Low Drone created by FM SynthesisA Low Drone created by FM Synthesis Time-stretched text created by GranulationTime-stretched text created by Granulation Percussive patterns created by Sample PlaybackPercussive patterns created by Sample Playback
Definitions (2)Definitions (2)
Sound Element ParameterSound Element Parameter: a parameter of a : a parameter of a synthesis/processing technique, e.g.,synthesis/processing technique, e.g., Modulation Frequency of a simple FM pairModulation Frequency of a simple FM pair Grain Size of a granulated sound fileGrain Size of a granulated sound file Ir/regularity of tempo in a rhythmic patternIr/regularity of tempo in a rhythmic pattern
MappingMapping: the connection between a motion : the connection between a motion parameter and a sound element parameter, parameter and a sound element parameter, e.g.,e.g., HeightHeightmodulation frequency of FMmodulation frequency of FM WidthWidthgrain size of granulated sound filegrain size of granulated sound file DynamicDynamicIrregularity of tempoIrregularity of tempo
Definitions (3)Definitions (3)
SceneScene: a group of motion parameters, : a group of motion parameters, sound elements, and mappings sound elements, and mappings between thembetween them
EyeCon Interface (1)EyeCon Interface (1)
Field: can measure height or width or dynamic or…
Touchlines: detect crossing and position on line
EyeCon Interface (2)EyeCon Interface (2)Fields and lines are mapped to MIDI data (or OSC)
Sequencer steps through “scenes”
Taxonomy of Motion ParametersTaxonomy of Motion Parameters
Body Parameters: position independent, Body Parameters: position independent, “attached” to body“attached” to body HeightHeight WidthWidth DynamicDynamic
Stage Parameters: position dependent, Stage Parameters: position dependent, “attached” to stage“attached” to stage Left-right positionLeft-right position TouchlinesTouchlines Extremely Narrow FieldsExtremely Narrow Fields
Parameter Type ExamplesParameter Type Examples
Stage Parameter (position): Scene 3 from Stage Parameter (position): Scene 3 from Brother-Sister SoloBrother-Sister Solo Julia Eisele, dancer/choregrapherJulia Eisele, dancer/choregrapher Stuttgart, June 2005Stuttgart, June 2005
Body Parameter (Dynamic): Body Parameter (Dynamic): ConversationConversation Robert Wechsler, dancer/choreographerRobert Wechsler, dancer/choreographer Julia Eisele, dancerJulia Eisele, dancer Stuttgart, June 2005Stuttgart, June 2005
Primary/Secondary MappingsPrimary/Secondary Mappings
Primary Mapping: controls dominant sonic Primary Mapping: controls dominant sonic featurefeature
Secondary Mapping: …is secondary…Secondary Mapping: …is secondary… Example: Scene 3 from Brother-Sister SoloExample: Scene 3 from Brother-Sister Solo
Primary mapping: positionPrimary mapping: positionposition in sound position in sound “landscape”“landscape”
Secondary mapping: dynamicSecondary mapping: dynamicdisturbance of disturbance of dronedrone
Secondary mapping: widthSecondary mapping: widthloop size/speed of loop size/speed of segment within sound filesegment within sound file
Sound Element mappings (1)Sound Element mappings (1)
A Human Conversation (in progress)A Human Conversation (in progress) Scene 7-8:Scene 7-8:
DynamicDynamicGranulated Text (playback rate)Granulated Text (playback rate) Scene 9:Scene 9:
Dynamic (left)Dynamic (left)Granulated Text (playback rate)Granulated Text (playback rate) Dynamic (right)Dynamic (right)Granulated Text (playback rate)Granulated Text (playback rate)
A Human ConversationA Human Conversation
Robert Wechsler (Palindrome), Robert Wechsler (Palindrome), choreographer/dancerchoreographer/dancer
J’aime Morrison (CSUN), J’aime Morrison (CSUN), choreographer/dancerchoreographer/dancer
Dan Hosken, composer and sound Dan Hosken, composer and sound programmerprogrammer
Work session, CSUN, June 23, 2006Work session, CSUN, June 23, 2006
Sound Element mappings (2)Sound Element mappings (2)
Perceivable Bodies (Emily Fernandez)Perceivable Bodies (Emily Fernandez) Scene 3a:Scene 3a:
PositionPositionGranulated Text (position in file) [Primary]Granulated Text (position in file) [Primary] WidthWidthGranulated Text (grain duration)Granulated Text (grain duration) DynamicDynamicLow FM Drone (mod frequency)Low FM Drone (mod frequency)
Scene 3b:Scene 3b: PositionPositionPhase Voc File (position in file) [Primary]Phase Voc File (position in file) [Primary] WidthWidthPhase Voc file (loop length/rate)Phase Voc file (loop length/rate) DynamicDynamicLow FM Drone (mod frequency)Low FM Drone (mod frequency)
Scene 4:Scene 4: DynamicDynamicGranulated Noise (density) [Primary]Granulated Noise (density) [Primary] DynamicDynamicGranulated Noise (position in file)Granulated Noise (position in file)
Perceivable BodiesPerceivable Bodies
Emily Fernandez, Emily Fernandez, choreographer/dancerchoreographer/dancer
Frieder Weiss, projections and Frieder Weiss, projections and interactive programminginteractive programming
Dan Hosken, composer and sound Dan Hosken, composer and sound programmerprogrammer
World Premiere at Connecticut College, World Premiere at Connecticut College, April 1, 2006April 1, 2006
[email protected]@csun.edu
Examples shown can be found:Examples shown can be found:http://www.http://www.csuncsun.edu/~dwh50750/Papers-Presentations/.edu/~dwh50750/Papers-Presentations/
Full Pieces can be found:http://www.http://www.csuncsun.edu/~dwh50750/Music/.edu/~dwh50750/Music/
Other Examples of Palindrome’s work:http://www.palindrome.de/
Max/MSP ScreenshotMax/MSP Screenshot
PD ScreenshotPD Screenshot
Reaktor ScreenshotsReaktor Screenshots
Eyecon ScreenshotEyecon Screenshot
EyesWeb ScreenshotEyesWeb Screenshot
Jitter ScreenshotJitter Screenshot
Cyclops ScreenshotCyclops Screenshot