Embed Size (px)
Citation preview
This may be the author’s version of a work that was submitted/acceptedfor publication in the following source:
Altizer Jr, Roger, Handman, Eric, Bayles, Greg, Jackman, Jeff, Cheng,Kun, Ritchie, Sydnie, Newell, Trevor, & Wright, Cheryl(2018)Choreografish: Co-designing a choreography-based therapeutic virtual re-ality system with youth who have autism spectrum advantages.In Proceedings of the 2018 Annual Symposium on Computer-Human In-teraction in Play Companion Extended Abstracts.Association for Computing Machinery, United States of America, pp. 381-389.
This file was downloaded from: https://eprints.qut.edu.au/124674/
c© Consult author(s) regarding copyright matters
This work is covered by copyright. Unless the document is being made available under aCreative Commons Licence, you must assume that re-use is limited to personal use andthat permission from the copyright owner must be obtained for all other uses. If the docu-ment is available under a Creative Commons License (or other specified license) then referto the Licence for details of permitted re-use. It is a condition of access that users recog-nise and abide by the legal requirements associated with these rights. If you believe thatthis work infringes copyright please provide details by email to [email protected]
Notice: Please note that this document may not be the Version of Record(i.e. published version) of the work. Author manuscript versions (as Sub-mitted for peer review or as Accepted for publication after peer review) canbe identified by an absence of publisher branding and/or typeset appear-ance. If there is any doubt, please refer to the published source.
https://doi.org/10.1145/3270316.3271541
Choreografish: Co-Designing A Choreography-Based Therapeutic Virtual Reality System With Youth Who Have Autism Spectrum Advantages
Abstract Choreografish is a virtual reality, therapeutic arts engagement leveraging participatory research and design to collaborate with young adults with autism spectrum disorder (ASD). The research team was motivated by the social anxiety some with ASD have, and the attendant difficulties accessing art forms that may actually play well to Autism Spectrum Advantages (ASA). This project was co-designed with youth with ASA to explore the use of VR and choreographic thinking to empower users and designers to engage with the arts and self-manage anxiety. This paper describes the project, and gives a brief design history of Choreografish.
CSS Concepts • General and reference → Cross-computing tools and techniques; Design • Software and its engineering → Software creation and management; Design
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. [email protected]. CHI PLAY '18 Extended Abstracts, October 28–31, 2018, Melbourne, VIC, Australia © 2018 Copyright is held by the owner/author(s). ACM ISBN 978-1-4503-5968-9/18/10.https://doi.org/10.1145/3270316.3271541
Roger Altizer, Jr. University of Utah, Salt Lake City, UT 84102, USA [email protected] Eric Handman University of Utah, Salt Lake City, UT 84102, USA [email protected] Greg Bayles University of Utah, Salt Lake City, UT 84102, USA [email protected] Jeff Jackman University of Utah, Salt Lake City, UT 84102, USA [email protected]
Kun Cheng University of Utah, Salt Lake City, UT 84102, USA [email protected] Sydnie Ritchie University of Utah, Salt Lake City, UT 84102, USA [email protected] Trevor Newell University of Utah, Salt Lake City, UT 84102, USA [email protected] Cheryl Wright University of Utah, Salt Lake City, UT 84102, USA [email protected]
Author Keywords Virtual reality; game design; games for health; autism spectrum disorder; ASD; anxiety; choreography; choreographic thinking; co-design; participatory design; dance; pattern thinking; neurodiversity
Introduction Dance is a social art form. The fact that it is often practiced with others can be a barrier to entry for people with autism spectrum disorder (ASD). Choreography, however, focuses on manipulating patterns to music, a process that has been shown to be soothing for many people with ASD [18]. This was what motivated the development of Choreografish, a game prototype which explores how we can bring people with ASD into the choreographic experience and if doing so could have positive psychological and social impacts on them. We are an interdisciplinary team of University faculty (representing humanities/video games, dance, and social science) working with young adults with autism through a community partnership/technology education program called NeuroVersity to develop a prototype of an underwater, virtual reality, choreographic thinking game for the HTC Vive. The prototype, Choreografish, engages complex motion pattern dynamics and movement design, while also lowering the barrier to a creative arts experience for individuals with ASD. In short, users get to choreograph the motion of a school of fish to any music they choose. The central idea is that experiencing choreography in an environment they designed will help them learn to both self-soothe and reduce anxiety as well as promote arts access.
Partnering with students with ASD on the design and user experience has been essential to our game
development. We have conducted focus groups and individual interviews with students with ASD for feedback regarding the user experience and design. While we know that Choreografish can enable more people to participate in a creative arts experience, our ongoing research explores to what effect, if any, playing at choreography in virtual reality has on young adults with autism from their perspective. Our goal in this participant-oriented research (POR) is to implement the feedback we have received from these students to iterate on our existing design. These students with ASD have been co-designers and evaluators in the research and this paper describes this process. This research approach helps to displace deficit approaches to disability by affirming the value of differential abilities and neurodiversity. We prefer to think of ASD using the net acronym coined by Dr. Altizer: ASA, or Autism Spectrum Advantage, as we have witnessed in our participant designers.
Background ASD is a life-long condition characterized by impairments in social interaction and communication. Individuals with ASD also have restrictive, repetitive behaviors [2]. Yet many with ASD have strengths in technology, attention to detail, intense interests, and advanced visual processing skills [4]. Despite the strengths and differential abilities of ASD, most ASD research focuses on early intervention with the goal of diminishing impairments, rather than acknowledging that it is a syndrome with differential strengths and abilities [5].
Colorado State University professor and autism spokesperson, Temple Grandin, coined the term “pattern thinkers” for the talents of a category of
Partnered Organizations:
NeuroVersity: A social enterprise that works to help youth with autism spectrum disorder (ASD) embrace their identity as skilled technology enthusiasts with leadership potential.
The GApp Lab: The Therapeutic Games and Apps lab is a serious games research group that currently funds over 30 graduate students who collaborate with faculty all over campus to explore the production of games and apps as research at the University of Utah.
Marriot Center for Dance: A center at the University of Utah dedicated to the physical, intellectual and creative training of emerging dance artists and to inspiring, empowering and connecting people through dance.
persons with ASD [8]. For individuals with ASD, creating patterns establishes order, and exercising order, producing a calmer state. Our approach to reframing disability and identifying some of the strengths associated with ASD is in providing creative opportunities that use these patterning strategies for artistic expression and creative empowerment.
Research has shown that by focusing on strengths, individuals show higher levels of positive behaviors, greater happiness and overall well-being [7]. The current project conceives game design as a vehicle of empowerment and skills development for youth with autism. “[G]ame design is one way that the arts and humanities continue to manifest. In a world of non-linear communication modes of self-expression take on new forms. Educational models need to consider how to provide meaningful creative and interpretive skills that embrace interactive[…]technologies.” [26]
We identified VR games as suitable for achieving the goals of this research in part for their emphasis on strength and abilities. Using game design to improve quality of life, educate, encourage positive behavior, and facilitate healthcare needs is a practice that has been refined over several decades. Research has pointed out many reason for games’ potential in these areas of application[12,14].
Of interest to this research is that games have an ability to engage and empower users by providing a context in which they have agency and which adapts to individualized needs, abilities, and preferences [19]. Games are well suited for maintaining players’ attention and interest by providing progressively more difficult challenges and goals; in a well-crafted game
experience, players receive continuous feedback so they know how to tackle a challenge and are given rewards and positive reinforcement when they perform well [17]. These characteristics have obvious benefits in fields such as education and healthcare [14].
Choreografish is designed to exist in between these categories, aiming to provide young people with ASD an environment that both facilitates a mentally healthy state of mind through meditative and playful interaction, as well as allows them to engage with the arts through a personalized choreographic practice grounded in producing order from chaos. Of particular importance is that the environment provides a platform in which the young adult is in control and can exercise playful agency. Choreografish does not aim to present users with specific challenges. It instead emphasizes open interpretation, creative participation, and playful and artful interactions with the game content. It leans heavily on emergent gameplay [11]. It does however strive to make it clear how the environment will react to the player’s interactions. These components of Choreografish comprise a context that empowers the user with playful, interactive agency and creative participation.
Research Approach: Participant Oriented Research/Design (POR) Participatory-oriented research and design are committed to an inclusive process with individuals whose real-life, meaningful experiences are critical to examining research issues [3,10,15,22]. POR is an orientation to research and game development that “focuses on relationships between academic and community partners, with principles of co-learning, mutual benefits, and long-term commitment, and
Observations on Emergent and Modeled Gameplay
Our intention with Choreografish is that the user would choose their music and choreograph the fish the way they wanted to.
In our initial playtests our co-designers with ASD would adjust all of the settings and manipulate the location, speed, and other boid behaviors.
In one session we had one of our PI’s, a professor of dance, demonstrate Choreografish. His use changed the way participants saw the software and they, afterwards, would use far more exaggerated motions and dance-like patterns.
These observation have led us to experiment with the use of dancers modeling play as a way to invite the user to explore choreographic thinking.
incorporates community theories, participation, and practices into the research efforts” [23:312]. It also shares power of knowledge between the community researchers and game developers [20]. The process enables community co-researchers to take equal ownership of the research and to question traditional approaches and curriculum strategies [9]. The approach also promotes social change strategies developed with researchers and community participants to design practical, beneficial programs and games for primarily underserved groups such as those with autism. They bring their experiences, knowledge and abilities into the research process and provide unique perspectives and insights [11]. The combined views of academic professionals and community research partners are critical to our research.
Inspirations for Design: Choreographic Thinking and Arts Access There are many dance games on the market, yet we have not been able to identify choreography games. This left a space for a game that allows users to produce motion patterns that engage and develop “choreographic thinking.”
The relationships between moving objects and their coordination to music is of prime interest to the choreographer. Choreographic thinking, by extension, is produced when a person creates relations between objects in space and time through movement [13,18,21]. Choreografish enables users to coordinate the actions of external objects to music of their choosing in order to trigger sensations, actions, responses and emotions. It is through synchronizing motor, visual, spatial and auditory inputs that “good Gestalt” of emotional resonance is produced [6].
Choreografish stimulates creativity by engaging users in producing audio/visual feedback loops: the user’s own musicality is channeled through the Vive’s hand controllers that effect the speed and grouping of a school of virtual fish. The swim patterns are affected in turn by the user’s reflexive responses to music. The school’s responsiveness to the user’s actions promotes an empowering sense of spontaneous creativity and incentive for continued play.
One of our research goals is to explore how choreographic thinking may be akin to pattern thinking, albeit with motion. If some people with ASD are pattern thinkers, perhaps setting these impulses to music builds on their inherent ASA strengths and overall enhances their ability to engage with choreography. That is to say that Choreografish may be a tool that extends pattern thinking to new modalities, exploring the strength of people on the spectrum in a new domain through virtual reality.
We believe this may be underexplored due to the social barriers people with ASD experience in accessing the arts. Choreography in particular often involves people working together in highly social and nuanced ways. This can be difficult for people with ASD to navigate[25]. Grandin also emphasizes the value of sensory deprivation for self-soothing to cope with stress. Sometimes, the social nature of choreography can be a challenge for people with autism, since social anxiety affects some people on the spectrum [24]. By providing a safe, private, virtual environment to engage in a creative process with motion patterns, youth with autism can play at choreography without the sensory and cultural stresses that are obstacles to participating in this kind of arts experience [27]By giving users
Figure 1: Choreografish being demoed at the Sundance Film Festival.
Figure 2: Choreografish podium controls in VR.
Figure 3: School of fish being choreographed to music.
access to an arts experience they may not normally have on their own terms, Choreografish serves as a translational tool. Inductive Participatory Design: Our initial design was informed by a series of “design box” sessions with the research team [1,16]. While we began with multiple pitches in hand we set them aside to take the time to focus on who the audience was, what problem we wanted to solve, how we wanted the game to feel and which technologies we could leverage allowed us to narrow down our design ideas and constraints. Constraints, after all, breed creativity.
Over the course of development, we conducted multiple focus groups to learn more about autism, to ideate around the project, and to test game mechanics during actual development. Sessions included young adults with autism as well as educators and specialists in ASD. We also had regular drop-ins, parents and young adults who were interested in and found Choreografish helpful who visited the lab of their own accord to let their child with ASA experience it. These visits were casual but informed our work. Below are a few exemplars.
Focus Group 1 The first focus group took place on July 15, 2016, with students in the NeuroVersity 3D digital design summer camp in South Salt Lake, Utah. We presented to a cohort of 15-20 Sketchup design students on the autism spectrum and then opened the session up for conversation on potential types of games in which the students might be interested. Students mentioned a number of existing rhythm and/or dance games that they enjoyed playing, and discussion shifted to the idea of creating patterns and controlling stage elements like
lights, fireworks, pyrotechnics, and fountains as a means of expressing musicality. In this discussion of patterns and choreography, the students brought up the idea of controlling schools of fish, an idea that would heavily influence the overall direction of the game. This was also the first mention of using trackers or controllers to track the location of the hands during gameplay, an idea that would find implementation in the project’s later use of the HTC Vive virtual reality system.
Focus Group 2 The second focus group was a play test of the Choreografish prototype at The GApp Lab on October 25, 2016. The project was two months into production, and developers were in need of feedback on controls, overall aesthetic, and movement mechanics. A group of students played the game and were able to identify a number of problems with the existing controls schemes. Their comments especially shaped the game’s movement mechanics, which went from a free-roaming system to a hub-based teleportation system, to reduce vertigo and nausea in the virtual reality setting. This focus group was also influential in shaping the development team’s understand that players would interact with the world in very different ways: some would be more focused on the choreography, others would take interest in exploration and free-form play, and yet others would find ways to push the system or use it in unintended ways. This began conversations about how the development team could introduce game elements to allow for different styles of play and varying levels of physical ability. The students’ input on the high level affordances of the prototype helped narrow the design into a more cohesive intervention.
Choreografish Described The design of Choreografish is the product of initial design box sessions by the research team, focus groups with youth on the spectrum, design box sessions with the student game developers and an iterative development strategy that used multiple playtests with NeuroVersity, graduate dance and game students, as well as the general public from events such as the Sundance Film Festival and smaller, local events.
The current prototype is an underwater VR (HTC Vive) experience where the user stands at a pedestal, chooses a song and controls a school of fish that swims around and in front of them on a set path. By manipulating two sliders they control the speed and formation of the fish. They can also rewind the song, and animation of the fish, to get the choreography they desire.
Focus Group 3 The final focus group was geared toward exploring potential future development and took place on March 15, 2017, after the prototype was complete. Developers met with six students on the spectrum and their parents at the Marriot Center for Dance. Students played through the game and then filled out user surveys addressing control schemes, aesthetic, perceived emotional impact, ease of use, and in-game audiovisual feedback. The development team also conducted semi-structured audio exit interviews with each participant. Findings were positive: players found the game to be compelling and intuitive, and most of the participants reported feeling calmer after playing the game. Additionally, a majority of the feedback on the game itself was oriented around adding more content (more types of fish, more areas to explore, etc.)—meaning the core functionality was compelling enough to motivate people to iteration on how to grow it, rather than how to fix it. In our experience when players ask for more content, as opposed to speaking about how to ‘fix’ a prototype the design is headed in the right direction. Participants experienced few if any bugs. We did, however, require users to wait as set-up took longer for us than expected, as did standard PC audio troubleshooting. We now work that time into our schedule when demoing Choreografish. Other helpful comments included the feedback that the environment was not “conscious” enough of the player’s body or did not respond to anything except the hands. A parent of one of the students also mentioned that adding biofeedback, like heart rate and breath rate, would be compelling. He, like many other parents, was excited to have his son at the university, participating in research and being exposed to college life, and he expressed his hope for the project and belief in its potential impact.
The development team hopes to explore the suggestions brought up by both students and parents in future development of the Choreografish game.
Conclusion and Future Plans Choreografish provides a model for co-design with youth with autism. It also illustrates the potential for interdisciplinary collaborations, in this case artists, social scientists and game designers to develop novel and innovative software solutions to solve complex health concerns. The research team has begun to conduct trials to measure the impact of the game on anxiety for youth with ASD. Initial results are promising, showing reduced anxiety in a yet to be published feasibility study, but more research must be done before a claim of efficacy can be made. The team strongly believes that embodied choreographic thinking in virtual reality may be the key to unlocking the potential of pattern thinking for helping young adults with ASD self-soothe through an immersive, creative engagement with technology. Since chaos causes anxiety in the physical lives of young adults with ASD, we are curious to see if Choreografish has a significant effect on anxiety levels by assisting people to exercise order over virtual chaos.
We intend to further co-develop the project and distribute it to schools and organizations that serve youth with ASD.
ACKNOWLEDGMENTS We would like to thank the University of Utah for its generous funding of the project, as well as the playtesters and volunteers from all three allied organizations.
Distribution Discussion: It is important for projects to move from bench to bedside, and while we have engaged the public in the development of Choreografish we also want to distribute it widely. While we conduct trials, we intend to distribute the prototype through NeuroVersity an organization that runs social programs for ASD youth.
Once complete, Choreografish, as well as a wrap-kit containing the source code and development history will be made available to the public via eChannel, a curated online-repository for digital media other scholarly works. eChannel was developed by The GApp Lab and was funded and maintained by the Eccles Health Sciences Library.
REFERENCES 1. Roger Altizer Jr., José P. Zagal, Erin Johnson, et al.
2017. Design Box Case Study: Facilitating Interdisciplinary Collaboration and Participatory Design in Game Development. Extended Abstracts Publication of the Annual Symposium on Computer-Human Interaction in Play, ACM, 405–412.
2. Center for Disease Control Autism and Developmental Disabilities Monitoring Network. 2010. Prevalence of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2010. Morbidity and Mortality Weekly Report: Surveillance Summaries 63, 2: 1–21.
3. Eva Brandt. 2006. Designing Exploratory Design Games: A Framework for Participation in Participatory Design? Proceedings of the Ninth Conference on Participatory Design: Expanding Boundaries in Design - Volume 1, ACM, 57–66.
4. M. -J Caron, L Mottron, C Rainville, and S Chouinard. 2004. Do high functioning persons with autism present superior spatial abilities? Neuropsychologia 42, 4: 467–481.
5. Hendricks Dawn. 2010. Employment and adults with autism spectrum disorders: Challenges and strategies for success. Journal of Vocational Rehabilitation 2: 125–134.
6. H. J. Eysenck. 1942. The experimental study of the ’good Gestalt’—a new approach. Psychological Review 49, 4: 344–364.
7. F. Gander, R. T. Proyer, W. Ruch, and T. Wyss. 2012. The good character at work: an initial study on the contribution of character strengths in identifying healthy and unhealthy work-related behavior and experience patterns. International Archives of Occupational and Environmental Health 85, 8: 895–904.
8. Temple Grandin. 2009. How does visual thinking work in the mind of a person with autism? A personal account. Philosophical Transactions of the Royal Society of London B: Biological Sciences 364, 1522: 1437–1442.
9. Farrah Jacquez, Elaine Ward, and Molly Goguen. 2016. Collaborative engagement research and implications for institutional change. Publicly engaged scholars: Next generation engagement and the future of higher education: 76–95.
10. Rilla Khaled, Vero Vanden Abeele, Maarten Van Mechelen, and Asimina Vasalou. 2014. Participatory Design for Serious Game Design: Truth and Lies. Proceedings of the First ACM SIGCHI Annual Symposium on Computer-human Interaction in Play, ACM, 457–460.
11. Michael D. Kickmeier-Rust and Dietrich Albert. 2009. Emergent Design: Serendipity in Digital Educational Games. Virtual and Mixed Reality, Springer, Berlin, Heidelberg, 206–215.
12. Thomas W. Malone. 1980. What Makes Things Fun to Learn? Heuristics for Designing Instructional Computer Games. Proceedings of the 3rd ACM SIGSMALL Symposium and the First SIGPC Symposium on Small Systems, ACM, 162–169.
13. Megan Nicely. 2014. On Choreographic Thinking. Dancers’ Group. Retrieved July 11, 2018 from http://dancersgroup.org/2014/03/on-choreographic-thinking/.
14. Rabindra Ratan and Ute Ritterfeld. 2009. Classifying serious games. Serious games: Mechanisms and effects: 10–24.
15. J Robertson and C. Howells. 2008. Computer game design: Opportunities for successful learning. Computers and Education 50, 2: 559–578.
16. Roger Altizer and Jose P. Zagal. 2014. Designing Inside the Box or Pitching Practices in Industry and Education. Proceedings of DiGRA 2014, DiGRA. Retrieved from http://www.digra.org/wpPoster Session: Game Design CHI PLAY'17 Extended Abstracts, Oct. 15–18, 2017, Amsterdam, NL 410 content/uploads/digitallibrary/digra2014_submission_134.pdf
17. Jennifer L. Sabourin and James C. Lester. 2014. Affect and Engagement in Game-BasedLearning Environments. IEEE Transactions on Affective Computing 5, 1: 45–56.
18. Sara M. Scharoun, Nicole J. Reinders, Pamela J. Bryden, and Paula C. Fletcher. 2014. Dance/Movement Therapy as an Intervention for
Children with Autism Spectrum Disorders. American Journal of Dance Therapy 36, 2: 209–228.
19. Valerie Shute, Matthew Ventura, and Fengfeng Ke. 2014. The Power of Play: The Effects of Portal 2 and Lumosity on Cognitive and Noncognitive Skills. Computers & Education 80.
20. Katharina Spiel, Laura Malinverni, Judith Good, and Christopher Frauenberger. 2017. Participatory Evaluation with Autistic Children. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, ACM, 5755–5766.
21. Steven Spier. 2011. William Forsythe and the Practice of Choreography: It Starts From Any Point. Routledge.
22. Timothy K. Stanton. 2008. New times demand new scholarship: Opportunities and challenges for civic engagement at research universities. Education, Citizenship and Social Justice 3, 1: 19–42.
23. Nina B. Wallerstein and Bonnie Duran. 2006. Using community-based participatory research to address health disparities. Health Promotion Practice 7, 3: 312–323.
24. Susan W. White and Roxann Roberson-Nay. 2009. Anxiety, Social Deficits, and Loneliness in Youth with Autism Spectrum Disorders. Journal of Autism and Developmental Disorders 39, 7: 1006–1013.
25. Mary Ellen Hunt. 2012. Autism in the Studio. Dance Teacher. Retrieved July 11, 2018 from
https://www.dance-teacher.com/autism-in-the-studio-2392357299.html.
26. Jordan Shapiro. 2013. How Game-Based Learning Can Save the Humanities. Retrieved July 11, 2018 from https://www.forbes.com/sites/jordanshapiro/2013/02/18/how-game-based-learning-can-save-the-humanities/#695bb2033b88.
27. Christina Devereaux Dance/Movement Therapy and Autism. Psychology Today. Retrieved July 11, 2018 from http://www.psychologytoday.com/blog/meaning-in-motion/201404/dancemovement-therapy-and-autism.
