Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
As it is: PEGASYS and Presence
Sarah Moss
Introduction
This chapter discusses an interactive art system produced in the context of the
author’s research into the nature of presence and designing for creative engagement.
The artwork is known as PEGASYS, an acronym for PresencE-Generating Art
SYStem. PEGASYS provides a time-based interactive cinematic experience that
utilises an eye tracking device to access video-on-demand. The research explores the
implementation of design features identified as necessary for enabling optimal
presence engagement for participants. The features include designing a system that
stimulates participants creative engagement, produces the sensation of spatial
presence and enhances the experience of ecological validity. PEGASYS also explores
the use of interactive narratives and addresses the use of colour, motion and audio as
key engagement factors. The art system was designed to consider the participant’s
personal interaction with, and reaction to, the system’s response time.
Researchers of presence aim to support the development of a comprehensive
definition of presence with tangible measures that assist in determining ample levels
of presence for the achievement of goals. ‘Media Presence’ and ‘Inner Presence’ are
two components of presence that are constantly under evaluation in the search for a
standard reliable and valid measure (Coelho et al., 2006). A study was conducted to
evaluate and compare participant’s experience of an art system designed to adhere to
these two aspects of the experience of presence. The art system is composed of two
main parts: a technological component (Media Presence) and a psychological
experience (Inner Presence). Combining research into current presence theories with
research about interactive art systems I then drew upon two models of creativity: a
Model of Creative Engagement (Edmonds et al., 2006) and a Model of Engagement
(Bilda et al., 2008). These models and how they were implemented into the design
framework are discussed in more detail later in the chapter.
PEGASYS: a Presence based Art System
PEGASYS consists of ninety minutes of high definition video (SONY HDV 1080i).
Shot from fourteen fields of view, with audio recorded using a binaural sound capture
device, the art system facilitates a 360 degree interactive cinematic experience for
participants (Moss and Edmonds, 2009). The biotechnological component of the art
system is an eye tracking device (Tobii X120-203-73700036). Networked into the
system by Oleg Sakharov, it provides on demand access to the audio and video data
clips. As the participant navigates the interface via eye-tracking, they pan left and
right choosing to interact between various levels of stimuli or ‘stages’ as they are
referred to. A dynamic database, designed by Alastair Weakley, contains interlinking
video clips that enable participants to traverse through a location named, Crystal
Pool, in Sydney’s Royal National Park, Australia.
Figure 1. PEGASYS: Stage One, Stage Two and Stage Three (Sarah Moss, 2009)
During evaluation, participants were observed to become immersed in the narratives
employed within PEGASYS. These narratives provided a variety of alternative
scenarios for participants to explore. The interview data revealed that the narratives
produced three very diverse phases of interaction. Stage One (five minute duration)
presents the natural landscape with natural sounds, Stage Two (eight minute
duration) introduces an avatar and various audio components, and Stage Three (ten
minute duration) provides features that require a higher level of participation and
engagement (see Figure 1).
The camera techniques employed, such as ‘direct address’I, assisted participants to
develop relationships with the screen-based characters or avatars, influencing the
experience of ‘being together with another’ a signifier of a presence-based
engagement. Participants expressed interest in sharing the digital environment with
others, indicating that mutal understanding and awareness created moments of
intimacy and sensory awareness. Three stages of interaction are accessible: the
participants made creative choices about where they wished to be located within the
system and defined for themselves the content of the engagement through individual
narrative selection. As participants move between the stages the stimuli changes the
experience as additional interactive elements emerge.
Participants were engaged in the experience as they observed and interacted with
elements such water, wind and shadow moving throughout the natural location. In
order to achieve the best effects, the video reproduction of the environment had to
best represent that environment, and to that end, a significant emphasis was placed
on this aspect during post-production. In order that participants could experience the
verisilimitude of a ‘natural’ environment, the cinematic footage used in PEGASYS
was shot in real-time and played back on demand. The art system aimed to provide
not only entertaining experiences, but a place for relaxation, personal reflection and
exploration of the artist’s concepts of enabling experiences of sensory illusion. These
experiences were observed during a month long evaluation period in Beta_Space at
the Powerhouse Museum, Sydney (Beta_Space, 2011).
Design Through Theory
The PEGASYS art system was designed in order to test participants’ experiences of
the degree of immersion. Three issues were considered from a physiological or a
psychological perspective (Media and Inner Presence): content in respect of Media
Presence, form in respect of Media Presence and participant engagement as a
component of the experience of Inner Presence. Riva’s three layers of presence (Riva
et al., 2004) were also considered in the design of PEGASYS as a theory-based
prototype. The layers include: self versus non-self (Proto Presence), self versus
present external world (Core Presence) and self relative to present external world
(Extended Presence).
Existing research on presence predicts that participants can be drawn into a state of
relaxation and restoration through a presence-based experience (Sponselee et al.,
2004, Witmer and Singer, 1998). The experience of PEGASYS itself was, therefore,
designed to relax the participant and facilitate an engagement with an art system that
accesses their conscious and subconscious minds. In this way, PEGASYS was
designed, through attention to form, content and participant, to create a peaceful,
serene and playful experience in an otherwise over-crowded noisy day. In the
evaluation study, the art system was found to produce a meditative, explorative and
creative environment that drew participants into a state of happiness via play and
engagement. The immersive environment explores possibilities for developing
interactive systems that bring the ‘peace of mind’ sensations usually generated by
exposure to natural habitats. In the city, PEGASYS provides a lunch break without
concrete. PEGASYS was thus designed to be experienced as a de-stress technique,
reducing anxiety and enhancing tranquillity in a relaxed yet playfully challenging
engagement. This research therefore extended conclusions about obtaining and
maintaining a state of harmony in a low stress mediated environment: the full extent
of the outcomes may be found in Moss (2011). Keeping all this in mind, PEGASYS
was then designed to adhere to a range of presence signifiers that stem from previous
research into presence.
The Art Systems Seven Design Features
The art system design features described below are ranked in relation to the
practitioner’s process. This involved devising a taxonomy of interactive art systems,
identifying previous research into presence-based systems and allowing for the
practitioner’s twenty years of experience working on large-scale projects in the
creative industries: TV, Theatre and Film. Seven features were identified for
consideration. These features include designing a system to: stimulate participants’
creative engagement, produce the sensation of spatial presence, enhance the
experience of ecological validity, explore the use of narratives, examine the use of
colour and motion, consider the audio, and account for the nature of the participants’
interaction with the system response time.
‘Engagement’ is the first feature to consider in respect of participants’ experiencing
psychological immersion. Engagement includes the participants’ ability to get
involved, facilitating a psychological immersion that can be a two-way
communication with another person or with an artificial entity e.g. computer system.
Engagement is an ‘Inner Presence’ component: however, it is also associated with
participants’ experiences of both the content and the form of the system. The creative
engagement experience differs from participant to participant. This is because
engagement with an interactive art system involves not only the act of looking, but
necessarily involves the participant taking an active role and engaging with the
system physically. Creative engagement invites the spectator to become a participant
within the system, facilitating a unique engagement involving human and computer.
Subjective personal human qualities, such as motivation, emotions and the ability to
suspend disbelief become intermingled with concepts of beauty, fun and pleasure.
Therefore, aesthetic appreciation plays a key role as does the cognitive state of the
participant. In order to engage the participant, two engagement models were
employed in the implementation of the art system design.
A Model of Engagement (Bilda et al., 2008) addresses engagement from an audience
perspective, thus providing guidelines for artists who wish to create interactive
systems with the most engaging possible outcomes.
The Creative Engagement Model proposes a method for designing engagement for
interactive art systems in terms of three classifications: attracters which draw the
participant into the engagement, sustainers that maintain the interest and hold the
participant’s attention, and relaters, which work to captivate and engage the long-
term interest of the participant ( Edmonds et al., 2006). Table 1 illustrates how the
design of PEGASYS combined The Model of Engagement interaction modes and
phases with The Creative Engagement Model’s attractors, sustainers and relaters. A
creative engagement was thus designed in respect of the art system’s six other design
features.
The second design feature is the experience of ‘spatial presence’. Spatial presence is
commonly referred to as physical presence and/or perceptual immersion (ISPR,
2000). Spatial presence transmits a sense of physical space, transporting the
participant into the landscape producing a ‘sense of being there’. i.e.; virtual reality
systems (Schubert et al., 1999). Lessiter states that spatial presence is also referred to
as ‘physical space’ which is proposed to closely connect engagement and naturalness
as key binding elements. Together these factors work with the individual’s personal
evaluation based on appeal and believability of both the displayed environment and
its content (Lessiter et al., 2001). It has been suggested that a central determining
factor for creating a sense of presence lies within the place itself, i.e. the virtual
environment or other location. Previous research into presence suggests that these
places are different from physical, and even mental spaces, as the space of presence is
viewed not as a component but as a defining dimension of presence itself (Spagnolli
and Gamberini, 2004). Spatial presence refers to the physical bodily orientation in
the world and requires appropriate updating of displays through tracking devices.
In presence terms, ‘ecological validity’ is akin to sensory presence. Sensory presence
is experienced through the system’s ability to employ the participant in a tactile
engagement and is gained through the systems ability to produce a sense of
naturalness. To work best, it requires high quality images and an efficent playback
system (Smyth et al., 2006). The pictorial realism and the visual depth are important
factors in the ability of a displayed environment to create a sense of depth. By
addressing ecological validity as a design feature it was possible to create and
maintain a sense of ‘naturalness’ within the art system’s content. Participants in this
system should feel that the environment is natural, believable and part of the real
world. Auditory cues are also associated with an increase in sensory presence
engagement.
The participants’ experiences of the ‘narratives’ as a design feature, has been
discussed in respect of the three stages of interaction (see Table 1). These stages
increase in media content or activity, each stage providing a different view of the
same location with additional interactivity embedded. In this way, each stage was
measured for presence engagement attributes independently through an interview
process that addressed participants’ experiences of both the visual and aural content.
By providing the participant with a contrast of media, comparisons were drawn about
participants’ experiences of the art system. During the evaluation process, a majority
of participants indicated that they enjoyed Stage Two the most. Study results
confirmed that Stage Two, containing one character who dominates the narrative
with a blend of natural sounds, voice and music combined, contained more emotional
dialogue and built in narratives, thereby enhancing the experience of the art system
for many (see Figure 3).
In designing the art system, it was important to consider the impact on the
participant of the use of colour. Since the system was built using real-time video clips,
the colour balance of each clip (fourteen clips in total) was edited to enhance the
natural environment. Where video ‘stitching’ occurs, the colour differences become
more apparent. Hence, it was important to capture the correct daylight colours with
the cameras using the built-in white balance feature.
Motion is also a key factor in the interaction process as participants navigate the
video clips through the eye tracking device by gazing left to right and vice versa. Thus,
the system’s content, through the system’s form, is in constant motion. Negative
effects such as feeling dizzy, disorientation and motion sickness can hinder the
participant’s experience and have an impact upon their engagement; this means that
testing the developing system with users regularly was essential to determining the
ultimate speed of the system.
In keeping with the theme of producing a relaxed experience for participants, the
‘audio’ plays an extended purpose, a duality for creating a heightened awareness of
oneself as well as sustaining the environment in which the film shoot took place. It
has been noted that a very specific form of music with a particular rhythm can induce
a relaxed state in the body; these types of music have been shown to create a music-
induced relaxation but nevertheless, leave the mind alert and able to concentrate.
Physiologists found the rhythms of the body: heartbeat, brain waves, and so forth,
tend to synchronise themselves to the beat of music. Classical music with a very slow,
stately, restful rhythm induces a calm and restful bodily experience (Ostrander and
Schroeder, 1979). The use of a continum of sound facilitates an environment
conducive to reduce stress and anxiety, assists in creating for the listener profound
states of deep mediation and relaxation, increases self awareness and has been
proven to improve mental and emotional well being. These are qualities which were
sought after in the design of PEGASYS. Therapists, new age healing centres and
health practitioners implement this type of musical score precisely to encourage long-
term improvement in mental and emotional well being (Mcleish, 1985). Composer
Marianthe Loucataris produced the score for PEGASYS. Features designed to
enhance the audio experience included atmospheric sounds, music, voice and sound
effects. Some of these sounds were embedded within the media and others were
designed to be retrieved as interactive components. During the interview process
positive comments were made about the use of sounds as a system feature to enhance
the experience of presence engagement. The implementation of sound cues as
attractors, sustainers and relaters increased the possibilities for engagement and
complimented the exhibition environment through a surround-sound score, while
the participant wore headphones (see Table 1).
The system response time is the time taken between the participant’s action and the
perceived effects of those actions upon the displayed environment (Floridi, 2005).
Latency of visual feedback has been seen to detract from presence engagement when
significant intervals of time pass between the action and the re-action. Therefore, the
form of the technology should facilitate a natural flow between the participant and
the interface. The PEGASYS participants experienced a hands-free engagement
because the eye tracking device activated the video-on-demand. Thus, they were
much more likely to become ‘located’ within the content and therefore, less likely to
be distracted by other worldly events; in other words, the state of presence was more
likely to occur. Furthermore, this art system was designed in order to replicate a slow
engagement. This means that the system’s interactivity was slowed down
deliberately, but dynamically, in order to create an experience which forced
participants to examine the details of each image as they were experiencing it. In this
way, the art system facilitated the original intention of the work, which was, to draw
the participant into a reflective ‘place’ that encouraged immersion. PEGASYS
facilitated rest and relaxation as participants engaged with a natural bushland
environment.
Overall, the participants ability to pay attention was found to be vitally important. If
participants were equally involved in the content (23/25 people agreed that the
content appealed to them) and the form of the system (21/25 people agreed that they
could interact with the art system) then it is an indicator that participants were
paying attention. During evaluation participants indicated that the systems features
were found to assist in the generation and experience of presence-based engagement.
A Model of Engagement PEGASYS The Creative Engagement Model
Interaction
Modes
Interaction
Phases
Attractors Sustainers Relaters
Unintended Adaptation STEP 1
Calibration
of the eye
tracker
Introduction to
the operating
system
Interface
design, simple
and efffective
Deliberate Anticipation
Learning
STEP 2
Stage One
Attempts are
made to
operate the eye
tracking
interface
360 degree
view
Diagetic sound
score
Intended /
In control
Anticipation
Deeper
Understanding
Learning
STEP 3
Stage Two
New content
alters the pace
of interaction
The sounds of a
girl humming
Introduction of
a musical score
Guided
interaction
Intended /
Uncertain
Adaptation
Deeper
Understanding
STEP 4
Stage Three
13 avatars
increase the
interaction
engagement
Additional fast
paced visual
media
More choices of
stimuli
Dynamic
sound content
introduced
Unexpected Adaptation STEP FIVE
Stage Three
Interactivity
increases as
participants
adapt their
behaviour
Multi viewing
options
Dynamic
sound content
explored as a
feature
Table 1 PEGASYS as a Dynamic-Interactive Art System
Figure 2. View of PEGASYS under evaluation in Beta_Space (2009)
Exhibiting and Evaluating PEGASYS
Evaluation of PEGASYS was conducted with fifty participants at the Creativity and
Cognition Studios, Interaction Lab and in Beta_Space at the Powerhouse Museum in
Sydney. In Beta_Space, twenty-five participants’ interactive experiences were
evaluated for presence attributes. Each participant was seated in the space as in
Figure 2. Here you can see the eye-tracking device is situated approximately 2.6
metres away from the display environment positioned between the screen and the
chair and directly in front of the seated participant. In order to establish contact with
the built in infrared camera lens embedded within the device, the participant had to
manually adjust the height and positioning of the chair to meet their own personal
requirements. Two tables and a chair were arranged diagonally in the bottom left
corner of the room with access to power. The artist researcher was able to observe the
participants with discretion. Video documentation was captured from a camera
situated directly behind the participant. Two external sound systems were in
operation. The first sound system created the atmospheric sounds and operated
through a DVD player positioned backstage, relaying into two speakers downstage of
the display environment. The DVD player looped a ten-minute audio track
throughout the day. The second sound system was operated via a CD player and mini
amplifier. The CD player was accessible from the operating desk allowing the
evaluator the opportunity to raise and lower the volume at will. Three small speakers
were rigged on the rear wall of the space directly behind the chair. Each participant
wore headphones.
Figure 3. PEGASYS: Stage Two, Clip Three. (Sarah Moss 2009)
PEGASYS and Presence
As we have seen, I designed PEGASYS in accordance with (a) seven presence-
orientated design features, (b) two models for creative engagement, (c) consideration
of participants’ experiences of low-stress environments as healing environments and
(d) elements of human consciousness requiring hightened attention, creativity,
mindfullness and engagement which are required in order to enhance the sensation
of a presence-based engagement.
Designed to provide deep immersive engagement PEGASYS initiated new engaging
experiences for participants with digital networked systems.
“Like a form of visual poetry. At times I was able to feel part of it or in shared control
but mostly I let it wash over me I think. Very tricky combination of sitting still –
feeling a little strapped in but more or less a sense of ‘letting go’ was the joy of the
experience”. Participant 16
An analysis of participant experiences, based on comments made during the
interview, observation and the results of the questionnaire (ITC-SOPI) indicated that
a majority of the participants were very engaged in their experience. Eye tracking
technology had a significant impact on participants’ experiences of the art system.
Participants interacted with the system through a series of repeated patterns or
random movements according to individual choice. The majority of participants
(21/25) agreed they enjoyed themselves, and 18/25 felt involved. 19/25 participants
agreed that they felt they weren’t just watching something and felt as though they
were participating in the displayed environment.
The results also indicated that the degree of interactivity made possible through eye
tracking was a successful design feature. The sensation of immersion was made
possible to some degree, as most of the participants (23/25) expressed that they had
a strong sense of involvement and felt themselves drawn into the experience. 21/25
completely lost track of time, the average duration being 35 minutes. Overall, there
were adequate presence signifiers acknowledged to ascertain that participants
experienced the sensation of presence engagement whilst interacting with the art
system. Participants experienced a new form of communication, and learnt how to
navigate, control and interact with, an audio-visual system through the use of their
eye movements only.
Most participants engaged with PEGASYS in a fluid playful manner; however, as
previously mentioned, a slow system response time was implemented as a system
feature and was observed to hinder some participants’ experiences. What I witnessed
occurring then was ‘absence’, as the participant tried to gain control and failed to do
so, they were drawn to engage with the technology in a new way. By observing the
participants’ eye movements using a track status window, it became apparent that
there had to be a genuine willingness on the part of the participant to interact and
engage with eye tracking technology, particulary when it is programmed to behave
dynamically.
“Yes, parts of it were relaxing. Parts of it I think I was trying really hard to get it to
work, and trying hard not to move and I think that… that I tend to hold everything
from the shoulders up to the chin. I do this (demonstrates). But that’s just me. But yes
it was actually really nice, and I didn’t mind that it wasn’t working… it didn’t frustrate
me. I just went oh well I just have to look at what’s actually happening now rather
than move it, ...because you do tend to go, oh this is really exciting, I can move it with
my eyes and you just want to keep moving around. So when it wasn’t working…I
actually… it didn’t frustrate me at all. It was actually… it meant that I saw things that I
might not have seen.” Participant #17
PEGASYS provides participants with a near realistic sense that they are somewhere
else, located within the media content, transported by the system’s form. Through
practice-based research it was possible for me to develop an interactive art system
that contributes new knowledge about aspects of presence engagement and
interactivity. This information can now inform designers and creative practitioners
about designing art systems through a process that combined theory with creative
practice. By uniting current theories on presence, interactive art systems and models
for creative engagement I produced, exhibited and evaluated a presence-generating
art system that utilises eye tracking in an interactive cinematic experience.
I ‘direct address’ : a technique where the actor on the screen addresses the viewer by looking directly into the camera.