Tricia Postle – Artistic Director, Majlis Art Garden { majlisarts.com }

Leif Bloomquist – Programmer & Composer { schemafactor.com }

New Adventures In Sound Art

TransX Transmission Art Symposium – Toronto, ON, Canada

May 19th, 2013

The Dancer From The Dance:Mapping Motion With Sound Via Radio Transmission

The Dancer From The Dance

O chestnut-tree, great-rooted blossomer,Are you the leaf, the blossom or the bole?O body swayed to music, O brightening glance,How can we know the dancer from the dance?

William Butler Yeats (from The Tower, 1928)

Source: folkadvance.org

The Dancer From The Dance

1. Introduction

2. The Past: Movement Mapping and Dance Notation

3. The Present: Arduinos, XBees and the MotionMIDI device

4. The Future: What’s next?

5. Other work in this area

6. Vignettes

7. Performance

Introduction

• How to add visual interest to a live electronic music performance?

• lights, lasers, strobes• projected visuals• cool outfits• etc

• What about dance?

• How to integrate the dancers with the performance?

Source: www.thissongissick.com Deadmau5

The Past: Movement Mapping and Dance Notation

Beauchamp-Feuillet Notation, c. 1680

Movement Mapping and Dance Notation

Friederich Zorn’s system from his “Grammatik der

Tanzkunst” (1887)

Movement Mapping and Dance Notation

Labanotation/Kinetography first developed in the 1920s

Source: Wikipedia

8

Flexibility within the piece + constraint within the form

8

flamenco: llamada kathak: hastakskokie public library

Movement Mapping and Dance Notation

Movement Mapping and Dance Notation

Motion capture... optical, inertial, mechanical, magnetic

The Present: Introducing Arduino

• Arduino is an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software. It's intended for artists, designers, hobbyists, and anyone interested in creating interactive objects or environments.

• Named for Arduin of Ivrea, King of Italy (1002-1014)

Source: www.arduino.cc

Arduino Continued

• Based on Atmel Microprocessors and the Processing language (a simplified version of C++)

• Designs are “Open” – free to use, modify, and create derivatives (hardware and software)

• Strong community focus

• Countless variations and sizes• Some (i.e. Arduino Lilypad) can be sewn into clothing!

• Countless add-ons, “shields”, kits, sensors, actuators, other devices…like radios! – ZigBee (XBee), 802.11 (Wifi), Bluetooth…

• SparkFun Electronics { www.sparkfun.com }• AdaFruit Industries { www.adafruit.com }• Creatron (right here in Toronto!) { www.creatroninc.com }• Many many more

The MotionMIDI Prototype

• Version 1 presented at Toronto Mini-Maker Faire in May 2011

• Version 2 presented at KwartzLab Makerspace in October 2012

Accelerometer

Arduino Uno

TransmitterXBee

ReceiverXBee

USB

9V Battery

The MotionMIDI System

XBee RadioReceiver

FTDI Serial-to-USB Adaptor

Digital Audio Workstation

FL Studio, Ableton, Cubase, Reason etc.

Virtual COM Port

Serial to MIDIAdaptor/Driver

) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) )

Accelerometer(ADXL335)

Arduino Uno

XBee RadioTransmitter

Analog Voltages(x, y, z)

MIDI Messages@ 38400 baud

Wireless Data

MIDI Messages@ 38400 baud

USB

MIDI Yoke

MIDI Messages@ 38400 baud

Movement andGravity

Or MIDI to Synths,

Keyboards, etc.

Some Vector Math

• Acceleration forces in 3D can be represented as 3 components: x, y, and z.

• This includes gravity!

• This gives you the direction and magnitude of the acceleration.

• Can determine orientation of the device (i.e. smartphones)

MIDI Assignments

Source: www.djtechtools.com

• We take the x, y, and z components from the acceleration, scale them, and map each into a unique MIDI “Continuous Controller” message.

• The force magnitude was added as its own MIDI CC.

• Ideal for filter sweeps (cutoff, resonance) and other continuous control (pitch, stereo, you name it!)

This is the *entire* sketch (program)!

#define CENTER 371 // Same for x, y, and z#define RANGE 100#define MAX (CENTER+RANGE)#define MIN (CENTER-RANGE)

#define LED 13

void setup(){ // Direct connect or XBee. Use 31250 for Raw MIDI  Serial.begin(38400); pinMode(LED, OUTPUT);}

void loop(){   int x, y, z;  byte x1, y1, z1;    digitalWrite(LED, HIGH); // set the LED on   x = analogRead(0); // read analog input pin 0  y = analogRead(1); // read analog input pin 1  z = analogRead(2); // read analog input pin 2

  x1 = AccelToCC(x);  y1 = AccelToCC(y);  z1 = AccelToCC(z);

  sendMIDI(0xB0,20,x1); // Channel 1, CC#20  sendMIDI(0xB0,21,y1); // Channel 1, CC#21  sendMIDI(0xB0,22,z1); // Channel 1, CC#22

  digitalWrite(LED, LOW); // set the LED off

  delay(100); // wait 100ms for next reading}

// Send a MIDI messagevoid sendMIDI(byte cmd, byte data1, byte data2){   Serial.write(cmd);   Serial.write(data1);   Serial.write(data2);}

// Map an input between MIN and MAX to 0 to 127byte AccelToCC(int in){     // Bound    if (in > MAX) in=MAX;    if (in < MIN) in=MIN;        // Change zero-offset    in = in-MIN;        // Scale between 0.0 and 1.0    float temp = (float)in/((float)MAX-(float)MIN);        return temp*127; }

{ https://github.com/LeifBloomquist/MotionMIDI }[MIT License]

Future Plans

• Look into commercial and hobbyist sensors that could be employed (magnetic, inertial).

• Use the sensor inputs to generate notes, directly or as seeds to a fractal generation algorithm.

• Work on more pieces, involve and get feedback from the dance community (i.e. Coexisdance)

• Small production run?

• Explore and have fun!

Acknowledgements

Special thanks to:

• Seth Hardy (Site3 coLaboratory) for XBee radio programming help.

• Gauri Vanarase for previous dance history input.

Related work

• Imogen Heap and “The Gloves” { www.imogenheap.com, theglovesproject.com }

• Loretta Faveri and SonicWear { www.sonicwear.ca } Commercialization of OCADU project similar to ours – in discussions about collaboration.

• Kinectar { www.kinectar.org } Microsoft XBOX Kinect with MIDI devices.

• The LEAP Motion { www.leapmotion.com } Handsfree 3D input device

• Many music-related apps in development (coming July 2013)

• Many more!

Vignette #1: sumi-e

• Using the Image-Line (FL Studio) “Harmor” generator VST with a synthesized Gong preset.

• X-Axis is mapped to the instrument pitch

Vignette #2: sweet synth strings

• Using the FL Studio “Plucked!” virtual strings synthesizer.

• Y-Axis CC is mapped to arpeggiation rate

• X-Axis CC is mapped to tonal color

Vignette #3: oontz

(The world’s shortest electronic body music [EBM] piece)

• Force magnitude CC is mapped to the cutoff frequency for the low-pass filter on the main synth line

Source: glamslamentertainments.com

Any questions?

Performance

Biography – Tricia Postle

Tricia Postle is the artistic director of Majlis Art Garden, a multidisciplinary seasonal space in Queen West presenting poetry, music, dance and storytelling.

This summer Majlis will host a salon-style evening on the intersection of music and technology. For further information please visit majlisarts.com .

Tricia is a lyric mezzo and poet, and has been known to play the hurdy-gurdy, psaltry, and qanun. She has recently started setting contemporary poetry as art song. She holds a BA in Medieval Studies and Music from the University of Toronto.

Biography – Leif Bloomquist

Leif Bloomquist has been creating computerized sounds since the days of the Commodore 64. Trained in clarinet and percussion, he now composes using sequencing software and homebuilt hardware.

His music can be heard in environments such as gothic nightclubs, CBC Radio 3, ambient festivals, and churches. He has released five albums to date through his Schema Factor and Interweaver projects.  For further information please visit www.schemafactor.com .

When not creating experimental music, Leif is a senior engineer at MDA, an aerospace company best known for their work on the Canadarm. He holds a BASc in Systems Design Engineering from the University of Waterloo.