Upload
nau
View
35
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Breath as an Embodied Connection for Performer-System Collaborative Interaction. Greg Corness Thecla Schiphorst Simon Fraser University. Overview. Background + Literature Review Initial Inspiration Key Interaction paradigms The Ariel System Methodology - PowerPoint PPT Presentation
Citation preview
1
Breath as an Embodied Connection for Performer-System
Collaborative Interaction
Greg Corness Thecla Schiphorst
Simon Fraser University
2
3
Overview Background + Literature Review
Initial Inspiration Key Interaction paradigms
The Ariel System Methodology
Studio Session (Performers improvising with Ariel) Interview technique
Procedures for Data Analysis The Three Primary Results Conclusion + Contribution
“The holy grail for me as a director is to produce a dramatic relationship between performer and
media, to grant media real agency, casting them in a role on par with the live performers” [Saltz, D.
“Live Media: Interactive Technology and Theatre”]
4
5
Background – Literature ReviewPerformer- System Interaction: the interaction
between a Performer and a Computer SystemPerformer-Performer Interaction: the interaction
between Two Human Performers
6
Background (Performer-System)Traditional ‘Sense- Respond’ paradigm tends to
focuses on after-the-event interaction.
Rowe, R. (1999). The Aesthetics of Interactive Music Systems.
Winkler, Todd. (1995). Making Motion Musical: Gesture Mapping Strategies for Interactive Computer Music.
7
Background (Performer-System)Traditional Sense- Respond paradigm tends to
focuses on After-the-Event interaction.Multi-modal base interaction considers the
system’s ability to track and anticipate the performer by sensing more channels of information.
Camurri, Antonio et. al. EyesWeb: Toward Gesture and Affect Recognition in
Interactive Dance and Music Systems.
8
Background (Performer-System)Traditional Sense- Respond paradigm tends to
focuses on After-the-Event interactionMulti-modal base interaction considers the
system’s ability to track and anticipate the performer by sensing more channels of information
The performer’s ability to anticipate the system is rarely discussed.
9
Background (Performer-Performer)Performance practiceResearch in neuro-scienceSocial Cues in Embodied Conversational Agent
10
Background (Performer-Performer)Performance practice
Agency, Trust, Collaboration, Negotiation Intuition related to Intention
Research in neuro-scienceSocial Cues in Embodied Conversational Agent
Lockford, L., & Pelias, R. J. (2004). Bodily Poeticizing in Theatrical Improvisation: a Typology of Performative Knowledge.
Bogart, A., & Landau, T. (2005). The Viewpoints Book: a Practical Guide to Viewpoints and Composition
11
Background (Performer-Performer)Performance practiceResearch in neuro-science
Mirror neurons Cognitive latency
Social Cues in Embodied Conversational Agent
Gallese, V. et. al. (2007). Intentional Attunement: mirror neurons and the neural underpinnings of interpersonal relations.
Wagner, D. M. (2003). The Illusion of Conscious Will
12
Background (Performer-Performer)Performance practiceResearch in neuro-scienceSocial Cues in Embodied Conversational Agent
Generating facial expressions Adding Breath Sounds
Foster, M. E. (2007). Enhancing Human-Computer Interaction with Embodied Conversational Agents
Whalen, D. H. (1995). The Effect of Breath Sounds on the Perception of Synthetic Speech
How can the interaction between a movement based performer and an autonomous generative
music system use breath to leverage the performer’s sense of intuition?
13
14
Click icon to add picture
Research Design
-Performers invited to improvise with the system over several sessions.-System designed to act as an autonomous agent generating music in the improvisational sessions. -System provided two basic conditions 1) presenting a preparation breath before it acts, 2) act without any breath.
15
The Ariel SystemDesigned to be
responsive
Designed to be Autonomous
Incorporates a model of embodied/social cues for projecting intention.
16
The Ariel SystemDesigned to be
responsive
Designed to be Autonomous
Incorporates a model of embodied/social cues for projecting intention.
Responds to performer’s actions through a Computer Vision Tracking Responding to motion Responding to shapes
17
Shape tracking Motion tracking
18
The Ariel SystemDesigned to be
responsive
Designed to be Autonomous
Incorporates a model of embodied/social cues for projecting intention.
Responds to performer’s actions through a Computer Vision Tracking Responding to motion Responding to shapes
Combine external and internal influences on the system’s response using weighted random processes
19
20
The Ariel SystemDesigned to be
responsive
Designed to be Autonomous
Incorporates a model of embodied/social cues for projecting intention.
Responds to performer’s actions through a Computer Vision Tracking Responding to motion Responding to shapes
Combine external and internal influences on the system’s response using weighted random processes
Incorporated a simulated breathing behaviour
21
Basis for modeling relationship between the Preparation Breath and following gesture
BreathBite Tail
Phrase
Loudness
22
The Ariel SystemThe system was designed and tested over three pilot
studies.
“It feels like ... It feels musical, as in…what feels good...it feels like it makes sense” (with breath)
“… it feels understandable ‘you know why there was silence” (with breath)
“… it felt like more background to me this time…like …it was more like a track of music playing..” (without breath)
“…(it) felt more chaotic” (without breath)
23
Studio SessionsSystem switched between presenting/not
presenting breath to provided comparison experiences for performers.
Structured improvisations guided performers to avoiding habitual approaches in interacting with a computer system.
Interviewing technique facilitating thick description of experience
24
Interview TechniqueBased on Interview of Explicitation
(Petitmengin)Re-living in an “inner way”Guiding performer’s attention of the experiencePutting into words
Petitmengin, C. (2006). Describing one’s subjective experience in the second Person:
an Interview Method for the Science of Consciousness.
25
Interview TechniqueRe-living in an “inner way”
Metaphor of video to encourage reflection Focus on selected Sensory Triggers found by
performer Sensitive to Body and verbal cues of Re-living
Guiding performer’s attention of the experience
Putting into words
26
Interview TechniqueRe-living in an “inner way”Guiding performer’s attention of the
experience Stabilizing Attention Move from representation to experience using how
questions Directing attention to dimensions of experience
Putting into words
27
Interview TechniqueRe-living in an “inner way”
Guiding performer’s attention of the experience
Putting into words Open use of language and gestures
28
Data AnalysisSeries of procedures
Close reading Open coding (developing themes) Construction of models
synchronic (independent of time) diachronic (time dependent)
Construction of a General model
Petitmengin, C. (2006). Describing one’s subjective experience in the second Person: an Interview Method for the Science of Consciousness.
29
Affordance from the Simulated Breath
Related observations of experience
1 Perceiving System Intention Conscious perception of a signalEmbodied understanding of gesture quality
2 Coordinating with quality of action
Increased ability to connect with new and complex material
3 Understanding and connecting with others.
Attention to presence and social expectations
30
Close Reading - (Perceiving Intention)The simulated breath is perceived as a cue The volume of the breath affects the
performer’s perception of the system’s intention.
31
Perception of the breath as a cue included: A conscious acknowledgment of the breath as a signal An embodied understanding of the upcoming gesture
“I find it was easier to expect. Although it (the music) was still really fast and sporadic at least you get a sense of ‘ it’s coming…. it’s coming… it’s coming” (system with breath).
Close Reading - (Perceiving Intention)
32
The volume of the breath affected the perception of the system’s intention as unnatural or demanding.
“When the breath is loud it sounds like an action being taken, but when it is soft it sounds like a physical cue as if the flute is in the room and getting ready to act.”.
“ … so it was no-longer a command or anything like that, it was like a cue… so it cues you that the music is starting but it doesn’t impede in a way…it doesn’t command you to follow…”
Close Reading - (Perceiving Intention)
33
Affordance from the Simulated Breath
Related observations of experience
1 Perceiving System Intention Conscious perception of a signalEmbodied understanding of gesture quality
2 Coordinating with quality of action
Increased ability to connect with new and complex material
3 Understanding and connecting with others.
Attention to presence and social expectations
34
Open Coding – (Quality of Action)Separating Participant’s CommentsPerformer-Performer communication
Belief/ AssumptionsDescription of experience
Approaches to Improvisation
Belief/AssumptionsDescription of experience
Experience with the Ariel System
Belief/ AssumptionsDescription of experience
35
36
“we were starting to just take silences
“it felt wrong to make big motions on complete silence
“breath was the safest time to move because you know the music is going to start
“there was more freedom with no breath…”
37
Open Coding - (Quality of Action)
38
Model of Expected Reciprocation“… it felt like more background to me this time…like …it was more like a track of music playing..” ( no Breath)
“I was surprised at how responsive…it felt like there was somebody…like give and take” (full Breath)
39
Model of Expected Reciprocation“the flute breaths and we say go ahead and sometimes it doesn’t and we are like ‘you let us down’ (with breath)
“I found I was responding to multiple things at once so picking up elements of what M and S and the music were doing. I also felt I could pay attention to effort and energy rather then precise movement” (with breath)
40
Micro and Macro-actions Macro-actions refers to large actions undertaken
with conscious intentionality . micro-actions refer to small actions performed
with little awareness or conscious intention Addresses the the affect of performer’s
attention Perceiving the system as one of the partners Social and artistic expectation
41
Stages in Performers’ Dynamic knowledge
Pre-improvisation Own expectations, training affecting interpretation of rules, tasks and environment
Start of Improvisation Sense presence, attention, in the room. Assigning of roles of elements still affected by previous experience
Reflection Perceive actions and states of self and othersPerceiving connection/meaning with actions
Construct New Knowledge
Develop variations of significant gesturesDevelop new connection with performersShift attention to perceived relevant sensations
Test Act in the moment based on new knowledge
42
Pre-Knowledge/
Start of Improvisati
on
Reflection
New Knowledg
e
Test
43
Stages in Performers’ Dynamic knowledge
Reflection Perceive actions and states of self and othersPerceiving connection/meaning with actions
Construct New Knowledge
Develop variations of significant gesturesDevelop new connection with performersShift attention to perceived relevant sensations
“…you can anticipate what is coming next-even though it wasn’t patterned.” (with breath) “My body know the length of the gesture by learning the breathing of the system”
44
Affordance from the Simulated Breath
Related observations of experience
1 Perceiving System Intention Conscious perception of a signalEmbodied understanding of gesture quality
2 Coordinating with quality of action
Increased ability to connect with new and complex material
3 Understanding and connecting with others.
Attention to presence and social expectations
45
46
47
48
49
Conclusion
How can the interaction between a movement based performer and an autonomous generative
music system use breath to leverage the performer’s sense of intuition?
50
51
Conclusion (Contributions)Designed Implemented and tested an
interactive autonomous system which uses breath as a social cue to leverage performer’s intuition.
Developed a model of interaction between Performers and autonomous computer system that addresses social cues for intuition and anticipation.
Developed a Methodology for accessing, collecting and evaluating performer experience data in performance
52
Conclusion (Contributions)Designed, Implemented and tested an interactive
autonomous system which uses breath as a social cue Demonstrated the use/ possibilities of social cues as a
parameter in Performer-System interaction Demonstrated Breath as a mechanism for the
embodied knowing of the system’s intentions New Knowledge of interaction between Performers
and autonomous computer systemDeveloped a Methodology for accessing, collecting
and evaluating performer experience data in performance
53
Conclusion (Contributions) Implemented a model of social cues in interactive
performanceDeveloped a model of interaction between
Performers and autonomous computer system Illustrated the importance of social cues in the
experience of interaction Identified three primary affordances of a simulated
breath in relation to the performer’s experience interacting with a system
Developed a Methodology for accessing, collecting and evaluating performer experience data in performance
54
Conclusion (Contributions) Implemented a model of social cues in
interactive performanceNew Knowledge of interaction between
Performers and autonomous computer systemDeveloped a Methodology for accessing,
collecting and evaluating performer experience data introduces a new approach to studying interactive
performance paradigms. Addressing identified issues in collecting thick
description data on performer’s experience
Conclusion (Limitations)
Current model for breathing behaviour quite course
Model acknowledges only a single social cueProviding time for getting to know the system
55
56
Conclusion (Future Work) Improving the breathing model Incorporation intuition into the Ariel system Investigating other body intention cues such as
vision and kinaesthetic Explore the effect of attention and association
Thank you
57