Using Robots in Autism Therapy: Using Robots in Autism Therapy: A Survey of Ongoing ResearchA Survey of Ongoing Research

Marjorie SkubicAssociate Professor

Electrical and Computer Engineering Dept.

Computer Science Dept. (joint apt.)

OutlineOutline

Motivation – How I got interestedAutistic disordersA survey of the research

– Why robots might help

The field of researchersConclusions

How I got interestedHow I got interested

Research in Human-Robot InteractionLooking for a killer applicationBetter – How can we use robots to help people?Talks at the IEEE RO-MAN 2005 Workshop

Autistic DisordersAutistic Disorders

1 of 300 children diagnosed with autism with rates rising– 1 of 800 children diagnosed with Down syndrome– 1 of 450 children diagnosed with juvenile diabetes– 1 of 333 children will develop cancer by age 20

Diagnosis currently made through behavioral observation– No blood test or genetic screening is available although there is

evidence of a genetic link

Autistic Disorders: CharacteristicsAutistic Disorders: Characteristics

Inability to relate to other peopleLittle use of eye contact with other peopleDifficulty understanding gestures and facial

expressionsDifficulties with verbal & non-verbal

communicationDifficulty understanding other’s intentions,

feelings, and mental states

Why Use Robots?Why Use Robots?

Most children, including children with autism, are attracted to robots.

This natural affinity is exploited, and the robot is used as an interactive toy.

Robots may provide a less threatening environment than interacting with people.– Robots can provide a repetitive and more predictable

environment.– This “safe” environment can gently push a child with

autism towards human interaction.

The Connection to ImitationThe Connection to Imitation

One theory: Autism may be caused by early impairments in imitation and shared attention (Rogers & Pennington, 1991) (Baron-Cohen, 1995)

Imitation is a format of communication, a means to express interest and engage others in interaction (Nadel, 1999)

Idea: Use a doll-like robot to engage children with autism and teach basic imitative interaction skills – From: K. Dautenhahn, and A. Billard, Games Children with Autism Can

Play With Robota, a Humanoid Robotic Doll, Proc. 1st Cambridge Workshop on Universal Access and Assistive Technology, 2002

RobotaRobota

A six-year old autistic boy playing with Robota. He seemed curious about Robota's head movements and so he touches the doll.

From: K. Dautenhahn, and A. Billard, Games Children with Autism Can Play With Robota, a Humanoid Robotic Doll, Proc. 1st Cambridge Workshop on Universal Access and Assistive Technology, 2002

“Robota … allows the child to understand that the doll’s movement originates from his own movement (sense of agency) and is limited to a restricted category of movement (enhances intentional action)”

From: J. Nadel, “Early Imitation and a Sense of Agency,” Proc. 4th Intl. Workshop on Epigenetic Robots, 2004

Imitation Using RobotaImitation Using Robota

An autistic child playing “chasing” games with the mobile robot

From: K. Dautenhahn, and A. Billard, Games Children with Autism Can Play With Robota, a Humanoid Robotic Doll, Proc. 1st Cambridge Workshop on Universal Access and Assistive Technology, 2002

Joint Attention Using RobotaJoint Attention Using Robota

From: B. Robins, P. Dickerson, and K. Dautenhahn, “Robots as Embodied Beings – Interactionally Sensitive Body MovementsIn Interactions Among Autistic Children and a Robot,” Proc. RO-MAN 2005

Robota is controlled via teleoperation by the investigator.

The investigator encourages the children to show each other howthey can interact with the robot.

The robot will not move unless the children show the same movement,i.e., they must work together.

Two autistic children:Note Andy’s gaze atJack.

Andy and Jack touch each other to balance themselves while each raising a leg.

Adam shows no interest in his classmatesand usually tries to avoid the rest of thechildren. But Adam is interested in Robota.

Adam takes Rob’s hand to show him how to interact with Robota.

Interacting with KeeponInteracting with Keepon

From: H. Kozima, C. Nakagawa, and Y. Yasuda, “Interactive Robots for Communication-Care: A Case Study in AutismTherapy,” Proc. RO-MAN 2005

Keepon is controlled via teleoperation.

Views from Keepon’s camera eyes

Attentive action

Emotive actionKeepon's kinematic mechanism. Two gimbals are connected by four wires; the lower gimbal is driven by two motors. Another motor rotates the whole inner-structure; yet another drives the skull downward for bobbing.

Enabling InteractionEnabling Interaction

Joint attention: Sharing the perceptual information

Eye-contact: Referring to each other's mental states

Enables people to exchange intention and emotion toward a target.

Emergence of dyadic interaction. Spontaneous actions to Keepon (left) and actions copied from others (right).

Emergence of triadic interaction. The child discovers excitement in Keepon (left) and then looks at the adult to share the excitement (right).

Using Robots for Autism DiagnosisUsing Robots for Autism Diagnosis

From: B. Scassellati, “Quantitative Metrics of Social Responsefor Autism Diagnosis,” Proc. RO-MAN 2005

ESRA Playtest

Autism Diagnosis MethodsAutism Diagnosis MethodsReaction to the ESRA robot with and

without the face configuration

Can generate facial expressionsusing 5 servo motors

Autism Diagnosis MethodsAutism Diagnosis Methods

Measure listening preferences to speech sounds

At the press of a button,an audio clip is played.The interaction is loggedin non-volatile memory.

Autism Diagnosis MethodsAutism Diagnosis MethodsVocal prosody, i.e., how something is said

Separation of two features used in a Bayesian classifier distinguishes low energy categories (neutral and soothing) from high energy categories (approval, attention, and prohibition).

Features F24 vs. F1Mean pitch * energy vs. mean pitch

Autism Diagnosis MethodsAutism Diagnosis Methods

Position tracking relative to another person

Autism Diagnosis MethodsAutism Diagnosis MethodsGaze direction and focus of attention

Red – adolescents with autismBlue – typical adolescents

For example, F(au)*G(self) indicates a filter trained on an individual with autism and tested on that same individual while F(nc)*G(au) indicates a filter trained on a control individual and tested on an individual with autism. The mean performance of this data (y-axis) is a function of the response percentile of individual pairings. Significant differences (all p<0.01 for a two-tailed t-test) are seen between the following classes: (1) F(nc)*G(self), (2) F(au)*G(self), (3) F(nc)* G(other nc), and (4) the three other conditions.

Linear discriminant analysis of autistic (au) and typical (nc) gaze patterns. Linear filters F(x) are trained to reproduce the gaze pattern G(x) of each individual x and then applied to predict the gaze patterns of any other individual.

University of SherbrookeUniversity of Sherbrooke

Project for engineering students:– Design a robotic toy for an autistic child

Educational value– Real world problem– Students work together in a team– Students must first investigate autistic disorders

University of SherbrookeUniversity of Sherbrooke

Pushing Jumbo around the play area.

Rolling game with Roball.

From: Michaud, F., Théberge-Turmel, C. (2002), "Mobile robotic toys and autism", Socially Intelligent Agents - Creating Relationships with Computers and Robots, Kluwer, pp. 125-132.

University of SherbrookeUniversity of Sherbrooke

Assembling the arms and tail of C-Pac.

Girl showing signs of interest toward Bobus.

From: Michaud, F., Théberge-Turmel, C. (2002), "Mobile robotic toys and autism", Socially Intelligent Agents - Creating Relationships with Computers and Robots, Kluwer, pp. 125-132.

The Field of ResearchersThe Field of Researchers

Francois Michaud– University of Sherbrooke, Canada

Kerstin Dautenhahn & Ben Robbins– University of Hertfordshire, UK

Aude Billard– Swiss Federal Institute of Technology (EPFL)

Jacqueline Nadel– French National Centre of Scientific Research

The Field of ResearchersThe Field of Researchers

Brian Scassellati and Bob Schultz– Yale University

Javier Movellan– University of California – San Diego

Hideki Kozima– National Institute of ICT, Japan

Michio Okada– ATR, Kyoto, Japan

ConclusionsConclusions

The use of robots for autism therapy and diagnosis is just beginning.

There is anecdotal evidence that robot therapy can help children with autism

How can we start here at MU with the new Thompson Family Center for Autism and Neurodevelopmental Disorders?

Maybe the Tiger KittyMaybe the Tiger Kitty

The iCat by Philips Research

AcknowledgementsAcknowledgements

Thanks to Brian Scassellati, Francois Michaud, Ben Robins, and Hideki Kozima for helpful discussions.