Engaging Child-Robot Interaction

in a collaborative taskCristina Zaga

Supervisors: Khiet P. Truong, Manja Lohse, Nicu Sebe

14/10/2014

1

2

3

4

5

6

RQ1

RQ2

RQ3

Research Questions

What kind of interaction style should a robot have to engage

children in a collaborative task?

How does the robot interaction style affect social and task

engagement?

How does the robot interaction style affect the interaction between

the children?

7

Mixed method: concurrent embedded strategy

8

Conceptual Framework…

Not merely a collection of concept, but a construct in which each concept plays and integral role

9

10

The experimentAn exploration:

The main objective of the experiment is to gather preliminary information that will help

to better define problems and suggest further hypotheses.

As such, the experiment could be defined as having an exploratory nature, and it will hopefully concur to provide some useful

indications for future works

11

An overview

12

H1

H2

Hypothesis

There will be a significant difference in the social and task

engagement triggered by the two interaction styles

How does the robot interaction style affect social and task

engagement?

13

The scenario

14

The robot: Nao Robot

Choregraphe

15

Interaction styles

Elements of the

interaction style

Tutor Peer

Instrumental

gestures

indicate, explain simulating actions

Emphatic

Gestures

nodding arms in the air

Proximity standing in front

of the children

sitting next to the

children

Reward nodding, spurring

like speech

raising arms, peer

like exulting

Speech maieutic,

reassuring

friendly

16

Interaction styles: states design

17

An example of script

18

From the states to Choregraphe design

19

20

Video

21

Task Design

22

Task 1: Complete with me

23

Task 2:Let’s make a house

24

Task 3: Putting the pieces back in the square

25

Resulting scenario

26

Methodology: WoZ manager

27

Means of assessment:

Observations and manual coding (video, audio)

Facial expressionsdetection

28

Means of assessment: EDA detection

Q sensor and Q software

29

Means of assessment:Questionnaire

30

Means of assessment:Questionnaire

31

Manipulation check: children

32

Manipulation check: teachers

Two videos with the behaviors flow of the

two conditions

Nao robot performs the behaviors facing the

viewer frontally,

like in an experimental session.

Given that the task is not performed

to help the understanding of the flow,

short captions appear on screen providing

a description of the behavior.

The videos are uploaded on the Internet

using the

platform for video streaming Vimeo.

Two mini survey realized with Google

forms.

33

Experiment

1 School: Het Zeggelt1 Class: N=26

3 days of experiment13 experimental sessions5 researchers involved

34

Experiment set-up

35

Experimental sessions

Following the study design, the couples

were organized to be same sex one (i.e.,

boy-boy, girl-girl).

With these two clusters in mind the C1/C2

dyads were randomly formed, six couples

were assigned to the peer condition (N=12)

and other six to the tutor one (N=12).

60% is male and 40% is female. The age

group of the participants span from six to

nine years, with a slight predominance of

participants aged seven years old (45% ).

154’ minutes of audio/visual and EDA data

were recorded.

24 questionnaires were administrated after

the task sessions.

5 separated files of Kinect

Let’s watch some of the videos!

36

Data analysis: Questionnaire

• N=20

• IBM SPSS and codebook (Mann Whitney

U / Chi Square)

• 3 items IMI Enjoyment Sub-Scale( CA

0.851)

• 3 Sidner engagement items (Fun, CA 0.813)

• 3 Again and Again table items

• 6 FFQ McGill (CA 0.809)

• 1 ISO (Inclusion of the self item)

37

Data analysis: Behavioral Observations

Intercoder agreement: from moderate agreement (Facial expression kappa

0.542, p .000) to good agreement (Gaze kappa 0.730 - p .003, Gesture kappa

0.689 p .000, Talk kappa 0.811, p. 004 collaboration 0.750 p .000 ).

ELAN + SALEM+ SPSS (Test: Man Whitney U / Independent t-test)

Salem thanks to: Hanheide, Lohse, Dierker

38

Data analysis: Annotation scheme

39

Results :Questionnaire

40

Results :QuestionnaireSidner engagement:

Distributions of the Fun scores for tutor condition and peer condition

were not similar, as assessed by visual inspection.

Engagement scores for tutor condition (mean rank = 12.15) and peer condition

(mean rank = 8.5) did not have a significant statistical difference,

U = 66.5, z = -1.265, p = .206.

IMI:

IMI scores for tutor condition (mean rank = 10.30) and peer condition

(mean rank = 10.70) did not have a significant statistical difference,

U = 48.00, z = -154, p = .878

Again Again Table:

A chi-square test for association was conducted between conditions

(robot peer vs. robot tutor)

and willing to do the activity with the robot again. There was not a statistically

significant association between condition and willing

to do the activity with the robot again,

χ 1.053 p=1.00.

41

Results :Questionnaire

FFQ McGill:

scores for tutor condition (mean rank = 11.09) and peer condition

(mean rank = 10.09) did not have a significant statistical difference,

U = 55.0, z = .037 p = .075.

The ISO:

score for tutor condition (mean rank = 10.35) and peer condition (mean rank = 10.65)

did not have a significant statistical difference,

U = 48.50, z = -.119, p = 912.

42

Results :QuestionnaireHave you ever played with a robot?

Do you have a robot?

anomaly: apparently 70% of the children never played with a

robot before the experiment, but 50% account about having a robot.

It is unclear the reason of this result, but a possible explanation

is that the children had difficulties in understanding the question, despite the support

of the questionnaire manager.

What is the name of your robot? ,

Do you usually play with the robot and your friends?,

Given that you don’t have a robot would you like to have one?

56% of the children left them blank or with words of difficult interpretation.

The rest of the children 44% provide the names of the robot and account

for usually playing with a robot and some friends and 33% of them are willing

to have one.

43

Results : cognitive attributes of social –task engagement

Independent T-Test: not a significant difference in the gaze to the robot rate between the

tutor condition (M=5.484,SD=2.111), than in a peer condition (M=6.4002 SD=1.010) ( t=-

.662, df=8 p = .527)

Duration of gaze to robot for tutor condition (mean rank = 384.43) were bot statistically

significantly difference in the tutor condition than in the peer (mean rank = 369.74), U =

68.162, z = 1376 , p = .169)No difference in gaze duration at the robot when the robot was expressing a behavior (U= 30 Z=-.473 p=.636) but gaze to task (U=47 Z= -3977 p. 000)

44

Results : behavioral attributes of social –task engagement

Rate of emphatic gesture (mean rank peer= 4.40, tutor= 6.60, U=10, z=.248, p=.310,

Looking at the overlap with the robot behaviors, 86% of the children are directed to

the robot when it greets them.

I tutor condition 36% of times when Nao explain to the child,10% when the robot

rewards the children.

duration of the talk: (mean rank tutor=113.62, mean rank peer=74,51, U=2706, z=-

4.808 p= .000).

45

Results : affective attributes of social –task engagement

2% of the children laugh with the robot in the tutor condition, such reaction was

triggered by a robot behavior during the explanation of the Tangram puzzles.

Scowl rates (mean ranks peer=5 tutor=8 ,U= 0.000, z= -2694, p= 0.08)

Duration of the smile (mean rank peer=60.22, mean rank tutor=54.16, U=1,760,

z=.981, p=.327). First task: 62% smile peer, smile tutor 46% .

Duration smile when a robot behavior was occurring in the tutor condition (M=11.57,

SD=6829), than in a peer condition (M=4.21 SD=5.299) ( t=6.435, df=118 p = .000)

46

Results :task engagementeffectiveness

47

Results :task engagement efficiency

Performance Duration U = 60, z = -2.178, p = .029

.

48

Results :frequent behaviors

Talk: more in the tutor condition (72% ) than in the peer condition (20%). In

the peer condition 80% of the time the talk occurred during the performance

of the task, whereas in the tutor condition 50% of the talk among the

children occurred during the performance, and 38,30% during the

introduction

Collaboration: A Mann- Whitney U test on the duration of collaboration confirms the the distribution of the duration of collaboration is the same in the conditions. (mean rank= 30.70, mean rank= 32.70, U= 398, z=-.736 p=.462)From a qualitative angle, it can be accounted that despite mostly collaborating, some moment of non-collaboration were happening

Gaze: during peer condition 14% of total the gaze annotated is direct to a child, whereas in the tutor condition 19,34%. The gaze between the children overlap in the annotation, signaling mutual gaze both in the peer condition and in the tutor condition for the 19% of the annotation of gaze to the child and for the 2% of the total gaze annotation (peer condition) and the 3% of the tutor condition.

49

Results :frequent behaviors

Gaze per phase: 56% of the gaze to the child is directed during the introduction of the task, 21% during the performance and for 22% in the conclusion. In the tutor condition 52% of the gaze to the child is directed during the introduction of the task, 31% during the performance and 13% in the conclusion.

Smile: smile, in the peer condition the 32% of total facial expressions annotated is direct to a child, whereas in the tutor condition the 27%. The smile to the child was directed in the 41% in the introduction, the 21% during the performance and in the 38% during the conclusion in the peer condition. In the tutor condition, the smile to the child was directed in the 62% in the introduction, the 31% during the performance and in the 7% during the conclusion.

50

Results :manipulation check

51

Results :manipulation check

main factors to declare that the robot is in the tutor condition:

• the more serious style of communication and reward;

• the more essential style of speaking, with a less pitched voice;

• the way to encourage the children is more similar to the one of a tutor;

• the movements are more predictable;

• the style is friendly, detached as a tutor should be;

Peer condition:

• the speech style is similar and appropriate for a child-peer;

• the expressive gestures are typical of exuberant children;

• the encouragement is very affective drive;

• the fact that the robot sit with the children is indicative of equality and it is typical

during shared tasks at school;

• the movements convey enthusiasm;

52

Discussion:

QuestionnaireThe children seems equally engaged among conditions, namely they

account for high levels of engagement and they are willing to repeat

the experience with the robot once more. (H1 rejected, no significant

difference RQ1 not fulfilled)

Considerations:

• Answer polarization despite tailored tools

• Survey methods maybe premature for the age group

• Cognitive development not adequate for questionnaires

• Tendency to please

• Difficulty in expression

• Interaction Styles too abstracts to understand

53

Discussion:ObservationThe results on the social engagement advance the indication that

the amount and duration of the attributes of engagement are not particularly

affected by the interaction styles (H1 rejected, RQ1).

But some cues like smile to the robot, the talk to the robot and scowl

present significant difference in the duration and rate respectively;

these cues are predominant in the tutor condition,

highlighting a light indication that the tutor style both triggers intensive

affective engagement and, at the same time, negative affective engagement.

The engagement with the task though presents a significant difference between

the duration of the performance, signaling that the children in the tutor condition

Took significantly more to perform the task and the task completion

percentage show that the children were more efficient in the peer condition

(H2 confirmed, RQ2, RQ3).

54

Discussion:Observation

Considerations:

One cue at time;

WoZ responsiveness and sensor;

Rate per child, Overlaps and transition matrix;

Novelty effect;

Personality;

Content analysis (e.g., teasing);

Gender;

55

To Sum Up

Peer Interaction style

elicits more task

engagement

It is not affected by scowl as

the tutor.

Phenomena of teasing are

not found.

Social attributes of

engagement are mainly the

same with the tutor as

collaboration.

56

To Sum Up

The tutor interaction style

elicit less task engagement

It is affected by scowl and

longer smile and talk

probably phenomena of

teasing .

Social attributes of

engagement are mainly the

same with the peer as

collaboration.

The design in general

seems to be the one with

major flows.

57

Lesson LearnedUpdate conceptual framework: the definition of engagement hold for the discrimination of social task engagement, but a better specification of the model to apply on the attributes of engagement might be beneficial in further studies.

Robot Interaction style design: low levels behavior, such gaze, gesture, navigation, speech should be taken more in account and a direct control on them might be beneficial.Besides the design, also the exploration of different types of robot more able to be collaborative in the task might enhance both engagement and the research in engaging robot behaviors.

Survey methods with children: When surveying children, special attention should be paid to questionnaire construction, and questionnaires should be thoroughly pretested. Possible new tools or methodologies could be taken into account

58

59

Thank you!