D6.2 Preliminary Usability Evaluation Reportdownload.microsoft.com/download/7/1/D/71DC7697-47... · D6.2 Preliminary Usability Evaluation Report Contract no AAL-2009-2-068 Start date

[PaeLife: Personal Assistant to Enhance the Social Life of the Seniors, Contract nº AAL-2009-2-068]

1

D6.2 Preliminary Usability Evaluation Report

Contract no AAL-2009-2-068

Start date of project 24/02/2013

Due date of deliverable T30

Completion date of deliverable October 2014

Lead partner for deliverable Université de Technologie

de Troyes (UTT)

Type of version v3.0


2

NATURE OF THE DELIVERABLE

R Report X

P Prototype

D Demonstrator

Project co-funded by the European Commission within the AAL Program, call 2

Dissemination Level

PU Public

PP Restricted to other programme participants (including AALA) X

RE Restricted to a group specified by the consortium (including AALA)

CO Confidential, only for members of the consortium (including AALA)


3

Document History

Issue Date Version Change Made / Reason for this Issue

28/02/2014 1 Preliminary user tests on 1st version of application prototype

07/07/14 2 User tests on the 2nd version of application prototype.

29/07/2014 3 Hungarian User tests on the 2nd version of application prototype.

Document Main Author Karine Lan (UTT)

Document signed off by Myriam Lewkowicz (UTT)


4

Table of contents

INTRODUCTION ..................................................................................................... 6

Motivation ........................................................................................................................................... 6

Preliminary evaluation as part of an iterative design process ........................................................ 7

Goal of the deliverable ........................................................................................................................ 7

USER TESTS CONDUCTED WITH THE FIRST PROTOTYPE ........................................... 8

Presentation of user tests ................................................................................................................... 8

Goals ................................................................................................................................................ 8

Settings ............................................................................................................................................ 9

Qualitative methods ...................................................................................................................... 10

Results ............................................................................................................................................... 10

Usability problems ......................................................................................................................... 10

Usefulness of services ................................................................................................................... 12

Satisfaction for multimodal interaction ........................................................................................ 13

Integration of insights in design .................................................................................................... 15

Conclusion ......................................................................................................................................... 15

TESTS ON THE GESTURES MODALITY .................................................................... 17

Presentation of gestures modality tests ........................................................................................... 17

Goals of the tests ........................................................................................................................... 17

Method .......................................................................................................................................... 18

Presentation of the Gesture Modality Test Prototype .................................................................. 19

Presentation of the participants ................................................................................................... 19

Consent form for tests on gesture modality ................................................................................. 19

Tests protocol ................................................................................................................................ 20

Satisfaction questionnaire ............................................................................................................. 29

Results ............................................................................................................................................... 31

Task completion time .................................................................................................................... 31

Error rate ....................................................................................................................................... 32

User satisfaction ............................................................................................................................ 34

Discussion ...................................................................................................................................... 37

Design implications........................................................................................................................ 38

Conclusion ......................................................................................................................................... 39


5

USABILITY EVALUATION CONDUCTED WITH THE SECOND PROTOTYPE .................. 40

Presentation of usability evaluation sessions ................................................................................... 40

Participants .................................................................................................................................... 40

Protocol ......................................................................................................................................... 42

Results ............................................................................................................................................... 45

Main menu .................................................................................................................................... 45

Vocal Commands ........................................................................................................................... 46

Agenda ........................................................................................................................................... 46

Contacts ......................................................................................................................................... 49

Messaging ...................................................................................................................................... 50

Find my… ....................................................................................................................................... 52

News Reader.................................................................................................................................. 52

Preferences .................................................................................................................................... 52

Themes / New GUI ........................................................................................................................ 52

Font Size ........................................................................................................................................ 53

Help ............................................................................................................................................... 53

Icons .............................................................................................................................................. 53

General recommendations ............................................................................................................ 53

Conclusion ......................................................................................................................................... 55

PLANNING THE FIELD TRIALS ................................................................................ 55

Method .............................................................................................................................................. 55

Roll-out .............................................................................................................................................. 56

REFERENCES ........................................................................................................ 57


6

Introduction The elderly population is growing at the same time as barriers to technology access are decreasing

among the elderly. Technologies, including Ambient Assisted Living technologies, are increasingly

used. More and more consumers and users of technology are becoming part of the category “older

adult”. Aging occurs on many levels – biological, psychological, social. Indeed, elderly people are at

risk of being isolated, thus the call 2 of AAL: Social Interaction. Concerning the interaction with

technologies, aging brings changes in perception, cognition and control of movements (Fisk et al.,

2009). Therefore this change in demographics brings with it important changes in the demands for

products and services adapted to older adults, that fit into a context of use and that meet user

needs. Comfort and ease of use for this population group is a vital element in the design of products

that support active ageing. This is especially the case for applications, like AALFred, which is

developed in the framework of the PaeLife project. PaeLife adopts the view that a technology’s

adoption and success relies on both its usefulness and the usability of its interface.

Motivation

Usability and usefulness are absolutely essential when designing AAL technologies, like social

interaction services. The main reason is that even if they contribute to a better quality of life, their is

not a necessity by itself. Contrary to other AAL technologies – like assistive technologies or medical

monitoring devices – social interaction applications are not mandatory for an autonomous living at

home. Therefore, the added value they bring and their perceived benefit are key elements to their

adoption by the elderly, and therefore to their success. Older adults are often described as being

refractory to technology, which is far from being true. Adults over 65 want to keep up with

technology and take advantage of what a technological world has to offer. Understanding new

technologies makes them feel connected to others and the world in general. When older adults

reject technology, it tends to be due to not perceiving the benefit of the technology, not necessarily

because it is too difficult or time-consuming to learn. The “not worth it” impression is more likely to

be stressed by an unusable interface1 (Pak and McLaughlin, 2011: 4).

A “usable” product is a product that is learnt naturally, that is easy and effective to use, and is not

unpleasant to the user. Usability is a quality attribute that assesses how easy user interfaces

(application, website, homepage, etc.,) are to use. However, an object can only be defined as being

usable related to a precise type of user – who represents the main target whom it addresses – and

related to a specific context (Baccino et al., 2005). Therefore, measuring the usability when designing

for the elderly requires taking into account these parameters, which are:

The context of use;

the type of user.

1 Unusable’ or ‘usable’ are the adjectives of the concept usability


7

Apart from usability, another important quality attribute is utility, which refers to the design's

functionality: Does it do what users need? Usability and utility are equally important and together

determine whether something is useful (Nielsen, 1994). Therefore, concerning products and services

designed for the elderly, the “need of use” – or rather the benefits of use – must be made clear

before older adults will voluntarily adopt technology (Fisk et al., 2009). Improved usability enhances

market penetration, improves quality of life of elder users and, with some classes of products, can

save lives (Fisk et al., 2009). This is the reason why a whole work package (WP6) in the PaeLife

project is dedicated to evaluating the usefulness of the services proposed and the usability of the

interface.

Preliminary evaluation as part of an iterative design process

The usability evaluation that has been achieved in the PaeLife project aimed at producing feedback

and recommendations as part of an iterative design process to ensure that the developed application

meets these requirements.

WP6.1 is the preliminary usability and usefulness evaluation of AALFred prototype. From both

methodological and organisational reasons, it appeared absolutely necessary to create clear bridges

between evaluation and development/implementation tasks. The insights of these user tests (WP6.1)

have been useful to inform the implementation (WP2 to 5). Through this coordination and

collaboration in between WPs, knowledge of the first and second prototype testing has usefully

informed the development, therefore making the most of (i) the prototype as an artefact for

reflective practice and (ii) user participation in the iterative design approach.

Goal of the deliverable This document is the second version of the deliverable “Preliminary Usability Evaluation Report”,

integrating:

- Results of the preliminary user tests that have been made on the first working prototype of

AALFred, and insights which have been integrated in the design of the second version of the

applications.

- Results of tests about the gestures modality.

- Results of usability evaluation conducted on the second version of the application, including

voice modality and recommendations that have been integrated in the final version of the

applications to be evaluated during field trials.

- Organization of the field trials in Hungary, France, Poland and Portugal.


8

User tests conducted with the first prototype As explained above, the focus of these user tests was to examine the perceived usefulness of the

services rather than the usability of the interface. It was a known and agreed upon fact that the

interface of the first version of the prototype – on which these tests were made in France – and the

second version, would be sensibly different. Therefore, even if the tests did not neglect the usability

aspect, the user tests consisted in making the users discover the application and ask questions from a

qualitative perspective, more than measure the performance rate when accomplishing tasks.

In this section, we will present the objectives, settings and methods used, before describing in detail

the insights produced, and finally how they have informed and been integrated in the design of the

second version of the prototype.

Presentation of user tests Following the guidelines for qualitative usability testing (Nielsen, 1993), the number of users aged

60+ who participated in the user tests, was three. The tests were held in an office of the living lab

related to our university (UTT), which had been specially prepared and dedicated to the activity. The

first version of AALFred had been installed and tested on a large vertical touchscreen PC running

under Windows 8 (figure 1) .

Figure 1 : Beginning of user test on large touchscreen PC

Goals

Aiming at this complementarity of techniques, our objectives in doing the user tests were threefold.

First, in trying the services, the users would provide feedback on the perceived utility of the services

proposed. Second, the aim was to gain an understanding of the users’ existing practices concerning

mediated communication and the use of technological devices, so as to question how the services

could support the elders’ social interaction habits, and therefore integrate coherently into their way

of life. The third issue was an intertwined utility/usability question. Indeed, the added value of the

tested system lies in the mixed interaction modalities that will be available in the final product:

touch, speech (input and output), gesture, together with mouse and keyboard. We were very aware

of the fact that touch was the only “natural” modality working correctly. Gesture is still under

development and was not available on the prototype; speech was tentative on this version of the

prototype. However, our concern was to examine how users would manage between these different

modes of interaction, and whether this mixed bunch of modalities was perceived as useful or not.


9

Settings

The user tests take the form of a protocol of predefined basic tasks to be achieved using a working

prototype and are combined with situated interview questions. The user test, made with 3 different

participants, lasted about 2 hours for each of them. Appointments had been set at precise hours,

with one user at a time. The main author of this deliverable was present all through to lead the test:

observe and record users’ actions – through both note taking and video recordings – ask questions,

and eventually solve the bugs and assist the user.

The list of basic tasks to achieve focused on the two most resulted services: email and agenda. First,

the users were required to access the application. For the email tasks, they had to read the incoming

messages, reply to an email, forward one to a contact, and more spontaneously typeset an email to a

person whose address they remembered or had saved on their smartphone or agenda. We were very

concerned about the necessity for the tasks to fit – and to succeed in questioning – these seniors

practices and interests. Also, since some of them were not familiar with touch and even more,

speech, interfaces, some tasks remained purposely vague. “Select the text zone in the way that

seems most appropriate” could be done either by clicking the zone or by saying “Object” or

“Message”. The agenda tasks consisted in rescheduling an appointment (following an email asking

for a report), reading through all the activities planned for the week, plan an activity with the grand-

children, look for a specific information related to school holidays in the agenda.

Though these tasks may appear very basic, due to the novelty of certain interaction modalities for

most users, they allowed interesting insights to emerge. The difficulties met by the users made

possible to identify very important problems about structure of information, positioning of buttons,

mandatory character of certain fields and the lack of intuitiveness of certain paths.

The participants to the tests were two men and one woman: User1, User2 and User3, all aged 60+.

User1 and User2 are husband and wife. Interestingly, the three of them had different levels of

computer literacy and different use of ICT.

User1 owns a basic mobile phone, which he uses a lot, but only for phone calls (no text messages and

no internet). He is not used to computers and hates typing. If he needs to send or reply to an email,

he finds it easier to ask his wife, who is much more efficient and quick than him. He rarely surfs on

the internet.

User2 also owns a mobile phone, but does not use it much. As a pre-retired secretary, she types

quickly, but does not use much the computer to look for information or communicate. She is the one

who sometimes sends photos by email to their two sons. The couple owns a desktop computer with

an internet connection at home.

User3 can be considered a technophile. He owns a smartphone, a tablet and laptop computer. He

uses his smartphone to manage several aspects of this daily life. His agenda is shared with his wife,

and he uses voice technologies to dictate his emails on his smartphone. He downloads many

applications on his smartphone and regularly looks for new ones.

The three of them do not have a Facebook account, and will not create one, for security and privacy

reasons.


10

Qualitative methods

Video ethnography

The main method we used, “videography”, also called video ethnography, shares with activity

analysis in ergonomics an interest with real-time / real-life activities, and focuses on the sequential

analysis of social action. The focus in on investigating human activities such as talk, nonverbal

interactions, the use of artefacts and technologies, that is, interaction between people and

interaction between people and technologies. They aim at identifying routine practices and problems

and the resources for their solution.

Apart from resorting to audio-visual data collection devices, videography implies a focused

ethnography that is short term, data intensive and which requires participant observation. The use of

video contributes to the sociotechnical construction of data and the video ethnographer is fully part

of the situation of action and social interaction that collaboratively emerges. The insights of the

video-recorded user tests have been collected through both fine-grained analyses of action based on

transcripts and user’s answers to the situated interview questions. We were interested in analysing

specific touch actions, or elements of prosody for the speech commands, that is multimodal

interaction with the application’s interface.

Situated interviews

Adapting traditional qualitative interviews, we lead “situated interviews”: the questions asked are

directly linked to the current or ‘just occurred’ actions of users engaged in testing the application.

This type of interview allows the production of reliable feedbacks. Also, questioning the users about

their usual practices – for example, how do they take an appointment in a paper agenda and what

information they consider relevant – allowed interesting and trustworthy insights. They produced

knowledge about:

the utility of services

the acceptability of the type of application and device being tested

some thoughts for improving the place for buttons and the mandatory character of certain

text zones.

Results We explained already that the focus was more on the usefulness than the usability aspect. However,

some usability issues have been found and examined as well. This section will present the “usability

problems” that have been identified (and fixed in the second version), and more interestingly the

knowledge that these tests allowed to produce concerning how the elder testers perceived the

usefulness of the services, and their satisfaction of the multimodal interaction framework.

Usability problems

Text zones – virtual keyboard

The first version of the prototype has been installed and tested on a touchscreen PC. Despite the

perceived practicality for User1, who is a left-hander and someone who does not use the computer

much, he still has difficulty typing. He is constrained by the positioning of the virtual keyboard and

text zones.


11

1. The vertical position of the touchscreen requires to raise the arms – which is not a

comfortable longstanding position, especially for typing

2. The virtual keyboard did not appear automatically when the text zone was touched / clicked.

This compelled the user to go and look for the keyboard. The problem is twofold:

a. The user must know the path, assuming that he is familiar with the windows8

interface (none of the three users could do it without the researcher’s help)

b. It requires that 4 actions are accomplished, before the virtual keyboard appears

I. Access the menu by sliding the main page.

II. Click on « Parametres »

III. Click sur « Keyboard »

IV. Select « Touch keyboard »

Mandatory and preeminent character

For the agenda, in order to take an appointment, the “duration” was a mandatory data to be

entered. Also, the place of the text zone (top left) gave pre-eminence to “duration” rather than

“time” (top right) or “object” (middle top right). The “description” text zone takes about half of the

screen (figure 2).

Figure 2: Screenshot of agenda

This raises several problems at different levels, where it clearly appears that usability and usefulness

issues cannot be examined separately.

1. The mandatory character of “duration” led to the appointment not being taken into account,

and being followed by an error message, though the other data were entered correctly.

Indeed, because all of the three users did not consider this data as being relevant, they did

not enter it. They simply could not figure out where the “error” was, which caused them to

be seriously annoyed.

2. The exact same problem occurred with “description”. Not considered as being relevant, the

users did not enter any description, which led to the same problem and to the same

miscomprehension.


12

Length of speech outputs

Though considered positively as giving a vivid sense of presence (cf. Section ‘Satisfaction of

multimodal interaction’ below), speech outputs should be kept short. The vocal help that is available

obtained the same negative / very skeptical feedback from all of the three users. When looking for

another function, User2 had accidentally clicked on the “help” button. The information is not written,

but vocal, mixing different categories of help information in a same stretch. The user starts to listen

to it, producing several continuers “that’s nice”, “yes”, “good”, before she is heard breathing loudly.

After 40 seconds of uninterrupted help message, she tries to stop the message by clicking on the

same button:

“well, well, well, shut up (laughs as message will not stop), top, (clicks again),

(laughs) te te te shut up! (Laughs) I can’t stop it! (Laughs).”

User3 opens his eyes wide and grins after 13 seconds after the help message has started, and

comments while the message goes on:

“I’m not delivered”

“it speaks by itself”

“I don’t want to hear all that”.

To the question “it is really too long, right?”, he responds

“that’s crazy! One hour and a half, huh? I don’t even see the interest of that

information.”

The fact that the help message comes in one go is, of course, due to a problem on that version of the

prototype, and has been fixed. Otherwise, we will have users expressing “I did not understand

anything”, which is problematic, especially for a “help” function. However, the insights are

interesting in that it confirms the necessity to keep output vocal messages

- short

- to the point

- clear

Usefulness of services

We are very aware that AALFred’s innovation and added value does not reside in the services –

which remain classical and basically useful – but on:

- Integrated Multimodal system - speech, touch and gesture (also mouse and keyboard)

- Availability of speech technologies in five languages, adapted to the specifics of ageing voices

- Ease-of-use of the Graphical User Interface, also adapted to the specificities of ageing

- Unified multimodal communication system

- Unified integration of the different modules


13

However, the user tests confirmed that the proposed services could successfully enhance the

mediated social interaction of elderly people at home.

Video conference service

The users perceived the benefit of the mediated communication allowed by the system. User1 and

User2 had never used videoconference applications and did not have it on their home computer,

though they had heard about Skype. After having tried one such application by talking to a colleague

of ours, User1 says:

“Easy like that, I think some elderly people who are against eh… With that,

they could get to it, and they would stay in touch. "Yeah I saw no one today"

[said with a complaining voice]. Well yeah, you take your thing, here you are:

you call! huh? That is good!”.

By “they could get to it”, the user refers to seniors of his entourage who are refractory to

technologies, and who can get to using computers. He puts forward the ease-of-use of the

application, which is a main advantage for computer beginners, and its usefulness to stay in touch,

especially when isolated people complain about being lonely. Therefore, this videoconference

application was considered as a convenient and easy means to maintain contact, as it gives a vivid

sense of presence, through both vocal and visual elements.

Indeed, videoconference applications allow for a type of heavyweight contact, that the elders value.

This preference of elderly for focused interaction of phone calls or face-to-face encounters was

confirmed in our interviews, and also from the user studies made as part of another similar project

(Alaoui et al., 2014). Talking is considered by our informants as being much more “natural” than

typing, whether on a keyboard, a tablet or a smartphone2.

Social media service

While demonstrating the social media service, we also questioned their practices of sharing photos

and the fears refraining them from having a Facebook account, so that we can refine the design of

content sharing service. The aim was to understand very clearly what the users’ value about sharing

photos, so that the services proposed correspond to users’ social habits, values and way of life. A

Secure Content Sharing system would palliate the negative aspects of existing solutions like

- Facebook: privacy and security issues

- Sending photos by email: time and effort consuming

Satisfaction for multimodal interaction

‘Naturalness’ of speech

The fact that speech technologies were proposed was very much appreciated. User2 was not at all

familiar with speech technologies. User1 had used dictation software, but it was a few years ago for

2 Concerning the ‘naturalness’ of speech, the fact that speech technologies were proposed was very much

appreciated.


14

work, when the technology was not very well developed, but nevertheless, had found it practical to

save time (since he does not like typing). User3 is very familiar with speech recognition, and uses it a

lot on his smartphone. He dictates his emails and firmly asserts that dictating is the best way to

“write” on a smartphone, which is too small for comfortable typing. He is convinced of the utility and

convenience of such a technology, that he has already adopted. However, as explained earlier, the

speech technology on the prototype is not fully developed and was not working very efficiently

during the tests. This has been explained to the participants, and has been taken into account. But

still, the feedbacks collected from the users give us hints about what they consider useful and what

aspects require improvements.

‘Mix’ between two modalities

Users expressed their satisfaction of being able to use the voice commands. The test with User2

shows that the use of both voice and touch modalities is easy. When the voice command does not

work, she rapidly understands that it has not been successful and easily shifts to touch. The several

feedbacks from the system, whether vocal or displayed on the screen, play an important role in the

user’s understanding of the situation. The availability of the feedbacks ensures their comprehension

by the user and a successful interaction. Therefore, there are several insights that can be useful to

inform improvements in design.

1. The interest of proposing a multimodal system is confirmed. The user’s actions, where she

adapts to the contingencies very fluently, confirms the scenario hypothesis where different

modalities would fit different situations in the home, depending on the activities the user is

engaged in.

2. The delay in obtaining feedback from the system leads to confusions about whether or not

the action had been taken into account by the system. A more rapid feedback would

prevent miscomprehensions and a more efficient interaction with the device.

Speech output – sense of vivid presence

The system we have tested allows both speech input (vocal commands and speech dictation in the

final product) and speech output (vocal feedback and Text-to-Speech in the final product). The vocal

welcoming message when one gets to the main page of the application is “Welcome to your personal

assistant. How can I help you?” This welcoming message is perceived as “good”, though,

interestingly, the user laughs when it is produced once more: “thanks bla bla bla. Thanks bla bla bla”.

The user’s perception of it is:

“Yes, after some time, we get it, of course. But for the people who are alone,

that’s a voice. Well, I think so. I know too many elderly people who are

isolated and lonely. Well, they would be here, they could call their

grandchildren, they could call their friends, don’t know. And then, when they

open the computer, well there is a question, well there’s someone with me (.)

For me, I know that I like it when I open my computer. It doesn’t say hello

every day but well… [Laughs].”

However, though it gives a sense of vivid presence, which could be appreciated by lonely elders,

these messages should be kept short.


15

Touch – virtual keyboard

User1 has written an email, using the Virtual keyboard on the touchscreen PC. Despite his difficulties

in typing, before sending his email, he gives useful feedback about his appreciation of the touch

modality: “Easy”, “Nice tool”, “Better than physical keyboard”. As a left-hander, he finds the virtual

keyboard more practical than the physical one, especially since he does not use the computer much

(figure 3).

Figure 3 : User1 typing email using virtual keyboard

Integration of insights in design

These preliminary user tests were analysed, in the context of a close coordination between the

evaluation and the development teams. Videography being data intensive and detailed-focused,

exploring the full potentiality of these data was time-consuming. However, the insights that

videography allows appeared as the most relevant and the most helpful at this level of progression.

The insights produced from the qualitative analyses were formalized and translated in

recommendations for design. The iterative improvements have been integrated into the next version

of the prototype. Now, when creating an appointment in the calendar the “name” and “day” are the

first fields in the list to fill and the only ones that are mandatory. Nevertheless, all the other fields are

kept for the users to fill if they wish so.

For the problem of the speech outputs, these have been shortened and made more specific. The help

is still there, but it is better localized and contextualized instead of being played all at once.

Moreover, the users have now the option of disabling or enabling the speech output according to

their needs.

The virtual keyboard will be completely redesigned according to the elderly needs. During this

process all the input collected in the user tests will be taken into account.

Conclusion These tests’ sessions allowed us, first, to demonstrate the interest and added value of proposing an

integrated multimodal system, especially voice and touch. It was interesting to see that the elderly

users we have met do not have any problem for shifting from one modality to the other. These tests

also showed the utility of videoconference systems and of speech outputs, if they remain short.

Though the 3 users had different degrees of competence in using technology, interestingly, the three

of them were very satisfied with the more “natural” modes of interaction. Also, the difficulties met

by the users allowed the identification of very important problems about structure of information,


16

positioning of buttons, mandatory character of certain fields and the lack of intuitiveness of certain

paths.

In the iterative approach we adopted, the issues were identified early and improvement is

continuous. The empirical details of ‘what users do with the AALFred’ are captured, analyzed and

translated in recommendations. Thus, the problems in this first prototype version were considered as

opportunities to identify what can be improved very early in the process, thus adding value to design.

The insights of the evaluation of the first version of the prototype in WP6.1 – in terms of usability of

the interface and usefulness of services have successfully informed the development of the second

version of the prototype. We believe that the iterative design process which PaeLife has adopted will

succeed in significantly improving the usability and usefulness of AALFred and follow the guidelines

for designing for older adults, so that it fits the needs and context of use of the elderly people.


17

Tests on the gestures modality

Presentation of gestures modality tests Over the last decade, gestural interfaces have earned increasing interest, both in the commercial

industry as well as in research. This type of interface gained popularity in the video game context

where, usually, users move their body that acts as a controller to play video games. However, due to

the broad availability and low cost of gesture recognition hardware, several applications are being

developed out of the gaming context.

Since people express themselves and interact in everyday social life through gestures, gestural

interfaces are considered very natural and easy to use (Correia et al. 2013). Therefore, body gesture

interfaces provide easier technological interactions for groups that, until now, have shown some

resistance in adopting technology, such as the older adults. However, seniors have particular physical

characteristics that can be a hindrance when using gestural interfaces. Research shows that ageing

brings along a significant decline in cognitive, perceptual and motor abilities. Motor issues of older

adults include slower motion, less strength, less fine motor control and decreased range of motion

and grip force (Nichols et al. 2006). Therefore, interactions should be designed carefully in order to

avoid fatigue, exhaustion and fine motor control. On the other hand, since some degree of physical

activity is required to interact with gestural interfaces, it is likely to positively impact the health of the

senior users, even if the intensity of physical activity is low (Saposnik et al. 2010).

Current literature focuses mainly on gesture interactions for average adults or, when it focuses on

older adults, it is usually in the gaming context. Therefore, seniors’ performance and acceptance

towards body gesture interfaces are not well understood, particularly considering their specific needs

and abilities out of the gaming context. In this study, we aim to understand how older adults can

benefit from gesture based interactions, in terms of suitability and acceptance, when interacting with

technological interfaces in general.

In order to evaluate how seniors adapt to gestural interfaces, we focused on two types of tasks

required to interact on most technological interfaces: navigation and selection. For each task, we

developed and evaluated two alternative gestures. Regarding navigation, we defined a Swipe gesture

and a Grab and Drag gesture. For the selection task, we developed a Point and Push gesture and a

Point and Hold gesture.

An experimental evaluation with the target user group was performed, where we measured both

performance and acceptance of the defined gestures.

Goals of the tests

This user study aimed to answer three main research questions:

1. Are gestural interfaces adequate for older adults, in order to interact with general

technological interfaces?

2. Which type of gesture allows for fastest navigation and selection with the lowest error

rate?


18

3. Do older users enjoy using gestural interfaces, finding it easy to use? Which gestures do

older users prefer?

Method

In order to understand if gestural interfaces are suited for seniors when interacting with a general

technological interface, we focused on two types of tasks: navigation and selection. For each task, we

evaluated two alternative gestures. We designed simple one hand gestures, thus avoiding problems

that may arise with bimanual interactions (Nichols et al. 2006). For all the defined gestures, it is only

required that seniors move their dominant hand above the hip and in front of their body for a short

period of time. Therefore, all the gestures are relatively simple and physically easy to achieve.

Regarding navigation, we evaluated Swipe and Grab and Drag gestures. To perform a Swipe,

users should drag either hand in the air and perform a horizontal motion to the desired direction.

A Swipe gesture is only considered when users horizontally move their hand for at least 30cm.

The vertical motion of the hand should not exceed 10cm, or the gesture is not considered a

horizontal Swipe. The time interval of the gesture should be between 0.25 and 1.5 seconds.

Regarding the Grab and Drag gesture, we used the implementation of Microsoft’s Kinect SKD. To

perform the Grab and Drag, users should raise either hand so that a hand cursor appears on

screen. The hand should be open, and the palm should be facing the Kinect sensor. Then, users

can close their hand to “grab” the content and then they can drag the hand in the desired

direction to scroll. To scroll more, users have to open their hand to “release”, so they can Grab

and Drag again. This alternative may require more movements and coordination than the Swipe

gesture, but we expect users to have more control on the navigation process. The Swipe gesture

strives for simplicity.

For the selection task, we developed Point and Push and Point and Hold gestures. For both

gestures, users should raise either hand towards the screen so a hand cursor appears. Then, to

perform a selection through the push gesture, users should move their hand in the direction of

the screen, as if they were touching the target. For this gesture, we also used the

implementation in Microsoft’s Kinect SDK. Regarding the Point and Hold gesture, users should

keep the hand cursor over a target for 1.5 seconds to select it. The interface gives feedback

about the selection state of the target by progressively filling its background with a lighter color,

like a sandglass. When the target is completely filled, it is selected. We expect the Point and Push

gesture to be more precise, since it will not restrict the time users have to aim. The Point and

Hold is simpler, as the users only have to keep pointing for a while to perform a selection.

Prior to performing the user tests, the developed gestures were evaluated by a physical

therapist, in order to access their suitability when taking into account seniors’ physical

limitations. The physical therapist concluded that the defined gestures posed no danger of

overexertion or lesion on older people. Our test prototype was implemented as a Windows

Presentation Foundation application, and the gesture tracking was developed using Kinect for

Windows SDK. To test the navigation gestures, we had a list of horizontally scrollable numbers.

For the selection gestures, a different number of targets were displayed on the screen, and users

were asked to select a particular target from the set.


19

Presentation of the Gesture Modality Test Prototype

The Gesture Modality Test Prototype was developed in order to perform the tests. The application

automatically collected and stored relevant data for posterior processing, while the monitor

collected user comments and feedback. Finally, all the gathered data has been analyzed in order to

improve the gesture modality.

The application had two distinct executables:

GestureModalityUserTests.exe –performs the user tests. The application is sequentially

testing the different features provided by the gesture modality framework.

TrainGestures.exe –allows the elderly to train each evaluated gesture for a while, before

performing the user tests.

We are now going to describe first how the tests took place, starting with the informed consent that

was signed by the participants, and the protocol that was followed in Hungary and France. Then, the

satisfaction questionnaire will be described and consolidated results will be given.

Presentation of the participants

20 participants were recruited in France and in Hungary. Due to technical problems, only 19 tests

were possible to analyse in France. So, in total, 39 tests were performed and analysed.

Most of the participants were aged over 60 years old. All participants had some experience with

computers, being that four of them (one in France and three in Hungary) did not own a computer at

home. However, none of them had prior experience with gestural interfaces. Most users had some

sort of physical movement limitations, such as slight rheumatism, tendinitis, osteoarthritis,

ankylosing spondylitis, but nothing particularly severe. These conditions did not prevent them from

using gestural interfaces. All precautions were taken to let them rest if they felt tired or had aching

articulations.

Consent form for tests on gesture modality

First of all thank you for participating in this study. Your collaboration is essential to the success of

our project.

This study falls within the PAELIFE project, whose aim is to study new ways of interacting with

devices and ICTs, using multimodal interfaces (which include various modes of interaction such as

speech, touch and gesture). The project is mainly focused on facilitating access to different services

concerning the elderly population. Examples include social media services, like Facebook and

YouTube, and other entertainment, organizational and communication services, such calendar, email

and audio / video conferencing. In the end we intend to develop a working prototype to test the

results of the study.

During this session we will ask you to perform some simple tasks with which you should be

comfortable, consisting of simple body gestures. At the end of the session we will gather your

comments and observations.


20

The application we ask you to try stores information about skeleton positioning while interacting and

a low resolution image of your face. All data collected is confidential and accessible only to people

involved in this study.

We request that you fill in the following data (the data provided is strictly confidential and is

intended only for future contact and for statistical analysis):

• Age: __________________________________________________________________

• Computer experience: ____________________________________________________

• Physical movement health problems: ________________________________________

[ ] Check the box if you want to receive updates on this study in the future? (If yes, please provide us

your email: __________________________________________)

I have read and understood the objectives of this session, participating willingly in it.

Signature: __________________________________________________________________

The researcher: _______________________________________________________________

Tests protocol

Starting with the TrainGestures application

Before performing the actual user tests, the older users were asked to train each gesture for a

maximum of 2 minutes. If the user was already comfortable performing the gesture before this time

interval, the monitor is free to skip to the next gesture (figure 4).

Figure 4: TrainGesture Main Interface


21

The radio buttons in the upper section of the application’s window allow the monitor to choose

which gesture is going to be trained. The main section represents the area where users interact by

performing the respective gesture. There were 4 gestures to be trained:

Navigation: these gestures allow users to scroll information.

o Swipes: To perform a swipe, the user must move either hand horizontally to the left

or right, depending on the side he wants to scroll. The faster the swipe, the more the

list scrolls.

o Grab and drag: To navigate using this gesture, the user should raise either hand

towards the screen. After a cursor appears on screen, he can perform a grab gesture

(by closing the hand). Then he can drag the hand to the desired side to scroll the list

of numbers.

Selection: these gestures allow users to select a particular target in a set.

o Hover: Users should raise either hand towards the screen so a cursor appears. By

keeping the hand cursor over a button for some time, that target is selected.

o Touch: As happens before, users should raise either hand towards the screen so a

hand cursor appears. To select a target, users should perform a press gesture

towards the screen. The cursor hand will become blue as the user moves their hand

towards the screen.

After the user is comfortable performing every gesture or the time for each gesture has run out, the

user tests started.

Using the GestureModalityUserTests application

The GestureModalityUserTests runs in a sequence of states that cannot be skipped or canceled. At

each step, a feature will be tested.

In the following diagram (figure 5), the possible states of the application are illustrated, as well as the

actions needed to go to the next state. Each user should go through a whole cycle of the application,

fulfilling all the steps at each state.

Figure 5: possible states of the GestureModalityUserTests application

Welcome

Face Recognition

Navigation Gestures

Selection Gestures

Attention Tracking

End


22

Through all the states, it is possible to see the current application state in the window’s title, as well

as the number of the log where the information is currently being stored on. As we will see in

PaeLife’s Gesture Modality Test Prototype - Monitor’s Checklist, this information is crucial to keep the

Monitor’s notes and user comments in synchrony with the automatically generated log files. In the

next sections we will depict each of the possible application states.

Welcome State

In this state, the user is required to take a seat in front of the Kinect’s captured area. When a user is

detected, the application automatically switches to the next state.

Face Recognition State


23

At this state, the user recognition module will be tested. A separator appears so the monitor can

explain the user what is going to be tested. After pressing the “Next” button, the application goes to

the actual Face Recognition state.

The monitor should begin by adjusting the elevation angle of the Kinect sensor. This can be done by

adjusting the slider at right side of the application’s window, labeled ELEVATION. The whole body of

the user should captured by Kinect.

Then, the Monitor should perform the following steps. Note that the application only allows

performing each necessary command in a predetermined order, so there is no need for the Monitor

to memorize these steps.

1. Ask the user look at the screen.

2. Click the Recognize button. This will try to recognize the user before he/she is in the

database. A message box will appear displaying information about the recognition. If any

error occurs, please read the message, correct it, and then click the recognize button again.

When everything went fine, click OK in the message box to dismiss it and go to the next step.

3. Click the save button. A message box will appear displaying information about the saving

process. If any error occurs, please read the message, correct it, and then click the save

button again. Press OK to dismiss it and proceed to the next step.

4. Click the recognize button again. The system will try to recognize the user and then will

display the user recognized. A message box will appear displaying information about the

recognition. If any error occurs, please read the message, correct it, and then click the

recognize button again. When everything went fine, please click OK in the message box to

dismiss it and go to the next application state.

There is no need to take notes on this state. After these steps were performed successfully, the

application will automatically proceed to the next state.


24

Navigation Gestures State

There are two navigation gestures that will be tested: the swipe gesture and the grab and drag

gesture. The monitor should take notes of the user comments and difficulties during this phase. E.g.:

the user performed a swipe but the application did not recognize it; the user performed a swipe to

the left but it was recognized as a swipe to the right; the grab and drag gesture required too much

coordination.

In order to avoid any sequential bias, the application randomizes the order of these two gestures.

This way, some users will perform the swipes first, while others will perform the grab and drag.

If the user is going to perform the swipes first, the following separator appears:

These separators can be used to take notes regarding previous interactions. It also serves as a

remainder for the monitor to explain the feature that is about to be tested. After pressing the “Next”

button, the application goes to the actual navigation by swipes.


25

In this state, the user is asked to scroll the list to a predetermined number that is displayed on the

screen. After the user scrolls to the required number and stays there for a couple of seconds, the

application automatically shows a new target.

A swipe gesture occurs when the user drags his hand in the air, performing a horizontal motion to

the left or to the right. The user is free to choose the hand he prefers to perform the required swipes.

After all the required targets are acquired by performing swipes, the application automatically

switches to the next state, which will be the Navigation by grab and drag (if it wasn’t performed yet)

or the Selection state.

Regarding grab and drag gesture test, before performing the actual test, the following separator

appears:

As explained, these separators can be used to take notes regarding previous interactions. It also

serves as a remainder for the monitor to explain the feature that is about to be tested. After pressing

the “Next” button, the application goes to the actual navigation by grab and drag.


26

The objective in this interaction is very similar to the one depicted for the swipe gestures, but the

user has to perform a grab and drag gesture to scroll. To do this, the user should raise their hand to

the screen until a cursor appears. Then, he can “grab” the list (by closing his hand) and then dragging

to the desired side. In order to scroll more, the user has to “release” (open their hand again) so he

can grab again later. The user is free to choose either hand to perform the gesture.

After all the targets are acquired, the application automatically switches to the next state, which will

be the Navigation by swipes (if it wasn’t performed yet) or the Selection state.

Selection Gestures State

There are two gestures that will be tested to perform selections on screen: the hover gesture and the

touch gesture. The monitor should take notes of the user comments and difficulties during this

phase. E.g.: it was difficult to keep the hand still over a target; users found difficult to perform a

touch gesture without sliding away from the desired target.

As happens in the Navigation state, in order to avoid any bias related to the sequence of the

performed gestures, the application randomizes the order of these two gestures. This way, some

users will perform the hover gesture first, while others will perform the touch gesture.

If the user is going to perform the hover gesture first, the following separator appears:

As already explained, these separators can be used to take notes regarding previous interactions.

After pressing the “Next” button, the application goes to the actual target selection by hovering.


27

The application will start by asking to select a random target in a grid of 2 targets, then in a grid of 4,

then in grid of 8, and finally in a grid of 16 targets. This procedure is repeated for 3 times. On each

time, the user is asked to select a particular target. When the user selects it, the application

automatically moves to the next target selection task.

In order to select, users should raise either hand towards the screen so a cursor appears. By keeping

the hand cursor over a button for some time, an animation is displayed and when it finishes, that

target is selected.

After all the selections are performed, the application automatically switches to the next state, which

will be the Selection by touch (if it wasn’t performed yet) or the Attention Tracking state.

Regarding touch to select gesture test, before performing the actual test, the following separator

appears:

After the monitor has taken all the notes regarding previous interactions, he can press the “Next”

button. Then the application goes to the actual target selection by hovering.


28

The task to perform is similar to the hover to select task, but in order to select targets users must do

a touch gesture. Therefore, in order to select, users should raise either hand towards the screen so a

hand cursor appears. To select a target, perform a press gesture towards the screen. The hand will

become blue as the user moves their hand towards the screen.

After all the selections are performed, the application automatically switches to the next state, which

will be the Selection by hover state (if it wasn’t performed yet) or the Attention Tracking state.

Attention Tracking State

Before performing the actual attention tracking test, the following separator appears:

At this state, the user attention module will be tested. The user is requested to look at the screen

and then look away two times. For a better user experience, please turn up the volume of your

computer, so the user can easily realize if the video is playing or not.


29

Please take note of the user comments and difficulties during this phase. E.g.: the user was looking at

the screen but the application did not detect it.

End State

The user successfully completed all the required tasks. The reference number generated in the log

files can be seen in the top right corner of the window. You should add the user’s number to the

comments sheet you’ve been taking notes on. You can now close the application or press the restart

button if you are going to perform another test. Note that if you are going to use the TrainGestures

application, this application must be close in order to free Kinect resources.

Satisfaction questionnaire

Navigation gestures

Regarding the swipe gesture recognition:

1. Do you think the gesture is easy to perform?


30

Very difficult Very easy

2. Was it tiring to your arm?

Very tiring Not tiring

3. Was the system accurate in recognizing your gestures?

Not accurate Very accurate

Regarding the grab and drag gesture recognition:







Selection gestures

Regarding the hover gesture recognition:







Regarding the touch gesture recognition:







31


Attention Tracking

Regarding the attention tracking:

7. Was the system accurate in recognizing if you were paying attention?


Other comments:

__________________________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

_________________________________________________________________________________

Results In order to understand how senior users interact and adapt to gestural interfaces, we quantitatively

analyzed the time required to complete the tasks we proposed, as well as the number of errors that

participants made while performing those tasks. We also qualitatively analyzed the questionnaires’

answers and the users’ comments.

We must note that the results we are presenting are not completely uniform, as we slightly changed

the way our application gave feedback to users in Swipe tasks. In the first 5 user tests, no visual

feedback was given about the state of the gesture recognition in Swipe tasks. However, participants

reported that they missed visual feedback, which is present for the Grab and Drag gesture. So, after

the 5th user, we decided to incorporate a simple feedback mechanism. It consists in showing a hand

icon on the bottom-right corner or bottom-left corner of the screen (depending on the hand used)

when users raise their hand above the hip, thus indicating that the system is ready to detect Swipes.

The hand icon is static and does not replicate users’ movements; it is just meant to be a simple

indicator of the system’s gesture recognition state. After integrating this simple feedback

mechanism, the users commented that this type of feedback was very useful. It did not change,

however, the performance nor the number of errors, as the means are similar before and after the

improvement. Moreover, the scores on the satisfaction questionnaire are similar before and after

the feedback was introduced, so we consider the results comparable.

Task completion time

The boxplot in Figure 6 illustrates the time required, in seconds, to perform the proposed tasks,

grouped by gesture. Regarding navigation tasks, users completed them faster when using the Swipe

gesture. Indeed, a paired t test revealed that the differences are statistically significant (p<0.005).

This occurred mainly because Swipes allow to scroll bigger distances faster. Moreover, most senior

users found the Grab and Drag gesture to be more complex and harder to perform than the Swipe


32

gesture. Some participants reported that they needed to be very focused in order to coordinate the

motions required to perform the Grab and Drag gesture. Indeed, some participants (one who

recently had a stroke for example) were having so many difficulties performing this gesture, we had

to end this task before the user completed it. On the other hand, some users preferred the Grab and

Drag gesture because it allowed a finer control, especially for small distances. For the Swipe gesture,

only one senior user had major problems using it. The way this user interacted led the system to

recognize Swipes in the opposite direction than the user intended, which created a very frustrating

experience in this case. Otherwise, most users found the swipe as being more natural, ease of use.

Regarding selection tasks, the Point and Hold (Hover) gesture allowed for a better performance when

compared to the Point and Push (Touch) alternative. A paired t test showed that this difference is

statistically significant (p=0.009). Since both gestures require pointing at the screen, we can conclude

that users took more than 1.5 seconds to perform the push gesture – the time required in Point and

Hold to perform a selection. Almost all users performed the selection tasks without difficulties,

finding the gestures simple and easy to perform. The exceptions were mainly the same users unable

to perform the Grab and Drag gesture. They did not manage to conclude all the Point and Push

gesture tasks, in which case we had to terminate the test before the user finished it. In Hungary users

could usually select a random target without any problem in a grid of 4 targets.

Figure 6: Time required to complete the proposed tasks (in seconds).

Error rate

Regarding navigation tasks, we considered and classified errors into three categories: Direction, No

Output and Precision errors. Direction errors occur when users are asked to navigate in one direction

but end up scrolling in the opposite direction. This can happen when participants did not fully

understand or did not perform the gesture correctly, or when the system fails to recognize or

recognizes wrongly the users’ movements. No Output errors are considered when users move their

hand with the intention of navigating, but no actual scrolling occurs. This may occur when the

gesture is not wide or fast enough or when the Kinect failed to precisely recognize the motions of the

user. Finally, Precision errors happen when users are scrolling in one direction to get to a particular

number but, due to lack of precision of the gesture, pass it by. In this case, users have to perform

another gesture to acquire the desired number.


33

Regarding selection tasks, we considered an error when users selected a different target from the

one they were required to select. Figure 7 summarizes the number of errors that users made when

performing the proposed tasks. As we can see, the total number of errors in both navigation tasks is

similar. Indeed, a paired t test showed that there are no statistically significant differences (p=0.18).

However, the types of errors users committed on alternative navigation gestures are different (Figure

7). We must note that not all errors were due to users’ fault, but because the technology still lacks

accuracy in some cases. Nevertheless, since this type of technology is still being improved, it is

expected to be more accurate in the near future. By far the most common type of error was

Direction, and users made more of these errors using the Grab and Drag gesture. When performing

this gesture, direction errors occurred mostly because of lack of coordination. In order to perform

consecutive scrolls in the same direction using this gesture, users have to close their hand to grab,

drag the hand to the desired direction, then open the hand and bring it back again to repeat this

motion. However, some users would forget to open the hand between these steps, which made

them scroll in the wrong direction, to the point where they started. This error occurred more

frequently when users had to scroll consecutive times in the same direction, and also in the

beginning of the test, when users did not have so much experience. Regarding the Swipe gesture,

direction errors occurred mainly because sometimes it is difficult to algorithmically interpret the

intention of the user. E.g., if users wanted to perform a Swipe to the left, they would raise their hand

and, in order to have more amplitude of movement to perform the gesture, they would first move

their hand to the right. In this case, if the movement to the right was wide enough, the system would

erroneously recognize this as a Swipe to the right. To segment and recognize the intentions of the

user in a continuous space of gestures is a complex challenge, magnified by the fact that each user

has his/her own way of interacting.

Figure 7: Number of errors that users made while performing the proposed tasks.

Regarding the No Output errors, both navigation gestures had similar results (Figure 7). These errors

occurred when users’ movements were so slight that the system did not recognize them as

intentions to scroll. Finally, regarding precision errors, the Swipe gesture had 50% more than the

Grab and Drag (Figure 7). This is mainly because when using the Grab and Drag gesture, users get

instant feedback and direct mapping of hand movements to scrolling. However, for the Swipe


34

gesture, users have to perform the whole gesture first for it to be detected by the system, and then

only see the results after the scroll. Users do not have instant feedback while performing the Swipe

gesture, which does not allow a precision as good as on the Grab and Drag gesture. This data

corroborates with users’ comments about their perception of control with the Grab and Drag

gesture.

In Figure 8 we can see that for selection tasks both gestures achieved very similar results. However,

when users were performing the Point and Hold gesture, most of the errors occurred when there

were only 2 or 4 targets on screen (83% of errors). This happened because the users would start

pointing at an undesired target, but they did not have enough time (selection is effective in 1.5

seconds) to adjust the hand position to the desired target, and then an erroneous target selection

would occur. On the other hand, most errors that were made on the Point and Push gesture occurred

when there were 8 and 16 targets on screen (60% of errors). In this case, the reason was the lack of

precision users had when performing the push gesture. Indeed, users had no trouble pre-selecting

the desired target by putting the hand cursor above it, but when they were performing the push

gesture they would slightly move their hand and would accidently select another target.

Figure 8: Percentage of errors, grouped by type of error that participants committed in navigation tasks.

User satisfaction

At the end of the user study, participants were asked to answer a satisfaction questionnaire

regarding the easiness of performing the gestures, whether it was tiring, and the accuracy of the

gesture detection. A 5 point Likert scale was used, with the higher score being the better. Figure 9

shows a boxplot of the results of the satisfaction questionnaire. Regarding navigation tasks, we found

that both gestures achieved similar satisfaction results. Indeed, a Wilcoxon signed-rank test showed

no statistically significant differences between the Swipe and the Grab and Drag gestures, for every

measured metric. Participants were divided between these two gestures: some would prefer the

Swipes and others the Grab and Drag. For selection tasks, a Wilcoxon signed-rank test showed that

there were statistical significant differences, being the Point and Hold gesture easier to perform,

although with lower confidence (Z=-1.813, p=0.07). For the tiring and accuracy measures, no

statistically significant differences were found.


35

Figure 9: Results of the satisfaction questionnaire.

Besides the satisfaction questionnaire, we performed an informal interview and also gathered

participants’ comments while interacting. Regarding the navigation gestures, we have seen that both

alternatives achieved similar results in the satisfaction questionnaire. However, this occurred

because some participants preferred the Swipe, while other preferred the Grab and Drag, which tied

the satisfaction scores. Thus, it is relevant to analyze the most frequent comments made by

participants. They reported that the Swipe is easier to learn and execute than the Grab and Drag

gesture. Therefore, Swipe was considered a more natural gesture. Some participants found the Grab

and Drag too complex and demanding in terms of coordination, considering it a gesture that is not

usually performed in everyday life and thus harder to master. Some participants also linked the

difficulty of performing navigation gestures with the lack of practice. However, they were optimistic

that if they had more time to practice, they would get used to it and would be able to use gestures

more proficiently.

Regarding the precision of the gestures for the navigation tasks, users reported that the Swipe did

not allow for very precise scrolling, particularly when users wanted to scroll very little. Indeed,

participants who were able to perform the Grab and Drag gesture usually preferred this alternative

over the Swipe, since it allowed for more control and precision. However, some users reported

discomfort while performing the Grab and Drag gesture, stating that since the hand palm needs to be

facing the television screen, it is an uncomfortable position. Besides this, seniors did not regard

either navigation gestures as tiring, except for a couple of participants who suffered from arthritis

and mobility and balance issues.


36

Figure 10: Swipe navigation task test in Hungary

The selection tasks were performed more easily when compared to the navigation tasks. We are

aware that this is also probably due to the fact that they were performed after the navigation

gestures, leaving time to the users to get used to the gesture based interaction. For selection tasks,

most users preferred the Point and Hold gesture. They reported this gesture to be very simple and

easy to perform. In France even when the targets got smaller, users reported that it was easy to aim

and select the desired target, while in Hungary users complained that from more than 4 grids it was

sometimes hard to select the target one. Participants also enjoyed the Point and Push gesture, but

reported that it was a bit more tiring to the arm. Some users started to perform the gesture with the

arm already stretched. In this case, there was no room for the arm to stretch more and perform the

“push” part of the gesture, resulting in users stretching the whole upper body painfully. Some users

had no problem pre-selecting, i.e. getting hand over target, but when performing the push would

lose precision and press on another target or even out of the screen.

Figure 11: Point and Hold (Hover) selection task test in France


37

Discussion

After analyzing all data, both quantitative and qualitative, we are now able to answer the research

questions proposed at the beginning of this user study.

1. Are gestural interfaces adequate for older adults, in order to interact with general technological

interfaces? Despite not having any previous experience with gestural interfaces, most senior users

performed all the proposed tasks without major problems. Only just a few users was not able to

complete all the tasks in the Grab and Drag and the Push gestures, but all of them, especially a lady in

France had expressed her motivation in participating in tests of “natural interaction” precisely

because she had a stroke and had difficulties in understanding and concentrating. All the other

participants, even the ones suffering from other health problems, such as arthritis and mobility and

balance issues, had no major hindrances performing the whole user test. Moreover, previous studies

found that even these low intensity exercises positively impact the health of the older people [16].

Therefore, in short, gestural interfaces proved to be a solid alternative for older people to interact

with simple technological interfaces.

2. Which type of gesture allows for fastest navigation and selection with the lowest error rate?

Regarding navigation, the Swipe gesture outperformed the Grab and Drag gesture in terms of speed.

This result occurred because the Swipe gesture allowed users to scroll bigger distances faster and

also because this simpler gesture is easier to learn and perform. In terms of number of errors, both

alternatives achieved similar results. However, participants committed more Direction errors on the

Grab and Drag gesture and more Precision errors on the Swipe gesture. For selection tasks, the Point

and Hold gesture allows for faster selections than the Point and Push. Both gestures are similar in

terms of error rate, although the Point and Hold allows for greater precision even when there are

more targets on screen.

3. Do older users enjoy using gestural interfaces, finding it easy to use? Which gestures do older users

prefer? Most seniors adapted very well to gestural interfaces. They found them easy to use and

enjoyed using them. All the developed gestures achieved good rates on our satisfaction

questionnaire in terms of easiness to use, whether it was tiring, and the accuracy of the gesture

recognition. In terms of preference for navigation gestures, some older adults preferred the Swipe,

while others preferred the Grab and Drag, which resulted in a draw satisfaction score. Regarding the

selection gestures, participants agreed that the Point and Hold is easier to perform than the Point

and Push.

This user study also allowed us to have some insights that may improve our original solution. One of

the problems of the Swipe gesture was recognizing the direction the user intended to scroll, as some

users had a way of interacting that led the system in generating a Swipe in the opposite direction. A

possible solution to this problem is to only allow each hand to Swipe to a particular direction.

However, despite certainly reducing number of errors, this solution limits the number of possible

interaction scenarios as it requires the user to have both hands free in order to navigate in both

directions. Moreover, most users declare they rarely use their non-dominant hand, and people

suffering from one arm may not feel comfortable with being compelled to use both hands. Regarding

the time and distance thresholds we imposed for the Swipe gesture, we found that they were not

perfect for all users. For some users, their Swipe motion was so wide that our gesture recognizer


38

would detect 2 consecutive Swipes in the same direction. For other users, their gestures were so

slight that the system was not recognizing them. Therefore, despite having defined reasonable

thresholds that allowed all users to adapt and perform all the proposed tasks, we conclude that each

user has his/her own particular way of interacting.

We also observed that the Swipe gesture allowed users to fulfill the navigation tasks faster than the

Grab and Drag gesture. This is mainly because Swipes are simpler to perform, and allow to scroll

bigger distances faster. However, the Grab and Drag gesture allowed for more control and precision

while navigating. Therefore, for technological interfaces that require much precision, the Grab and

Drag gesture may be a better alternative. Nevertheless, we must stress that some of our senior

participants had difficulties in coordinating and performing this gesture, so it may not be the best

choice for this particular age group. Regarding the Point and Hold gesture, an improvement we could

make is to increase the time required to hold over a target to select it. When performing selection

tasks, participants erroneously selected some targets because they did not have had the time to

adjust to the right target.

Transversal to all gestures, for we noticed that users had a tendency to perform better in the second

gesture that was tested. This probably occurred due to the increased experience with gestural

interfaces and also to the reduction of stress associated with a test environment, which is usually

higher at the beginning. This was also corroborated by the participants’ comments, as they said that

they would certainly perform better if they had more time to practice. We also noticed that the

detection performed by Kinect is not perfectly accurate. Sometimes, the Kinect hardware cannot

accurately detect users’ positions and movements. We particularly noted that the detection loses

precision when the user is sitting, as the body is constantly in contact with the supporting object,

which makes user identification less precise. Therefore, some of the errors that occurred were not

because users wrongly performed the gestures, but because the gesture detection mechanisms are

still not perfect. Finally, we must also note that the conclusions we present on this paper may not

apply to other cultures, as very different cultural backgrounds, either on different meanings of

gestures as well as familiarity with technology, may induce differences on the final results.

Design implications

From our results, we derive the following design implications for gestural interfaces:

Keep the defined gestures as basic as possible. Gestures that may look simple for the average adult,

such as Grab and Drag, may prove to be complex coordination challenges for older adults. Our

gestures that were composed by two distinct steps (Grab and Drag, Point and Push), demanded more

concentration from seniors, which led to a reduction in performance. The simpler the gesture, the

easier it is to learn, which also increases motivation to keep using it.

Develop gestures that allow to be used by any hand. In this study we tried to simulate a real life

scenario where gestural interfaces bring value, such as a living room. In this scenario, users may not

have their dominant hand free. Therefore, and related to the previous design implication, the

gesture must be simple enough to be used by the non-dominant hand. All participants have only

used their dominant hand to perform the Grab and Drag gesture, but some seniors used both hands


39

to perform the Swipe gesture in both directions. The Swipe, by being simple enough to be performed

by any hand, allows for greater freedom in interactions.

Give visual feedback of the state of the gesture recognition. In our first implementation of the Swipe

gesture there was no visual feedback simply because, for this gesture, there is no direct mapping

from hand movements to the elements on screen. However, senior participants felt lost when no

visual cues were given, wondering on when they could perform the gesture. Participants felt more

confident performing Swipes when visual feedback was given, even if it was as minimal as simply

displaying an icon on screen when the user’s hand was raised.

Allow personalization and adaptation. Each user has his/her own particularities in the way he/she

moves, both in speed and distance. This makes static thresholds not optimal for all population.

Gesture recognition is a great challenge per se, but the optimal solution involves adapting these

thresholds to each user, preferably automatically. Otherwise, manual personalization should be

available.

Conclusion In this study, we showed that gesture interactions are an appropriate way for older adults to control

a general technological interface. Our results showed that older people enjoyed using gestural

interfaces, finding most of the evaluated gestures easy to learn and use. We found that gestures that

are simple to perform by the average adult, may be a challenge to some older adults. Indeed, the

simpler the gestures, the better performance participants had in completing the tasks we proposed.

The Swipe gesture, used for navigation, was simpler and therefore allowed users to complete the

proposed tasks faster. The other navigation gesture, the Grab and Drag, did not have such a good

performance, but allowed users to have more precision and control over the navigation process.

Regarding selection tasks, the gesture with most success was Point and Hold, as it allowed for

accurate and fast selections. Older users highly rated all gestures in the satisfaction questionnaire,

which means that this type of interaction was widely accepted. In general, the senior participants

showed a positive attitude towards gesture-based interactions.


40

Usability evaluation conducted with the second prototype

Presentation of usability evaluation sessions In these tests, the users try to complete typical tasks. Doing so, the user identifies and describes

usability problems while using the system that is being tested, that is, he/she comments his/her

actions and what is seen on the screen, in real-time. A researcher-observer is present and prompts

the user to continue talking, watches, listens and takes notes. The objective is to gather both

qualitative data from feedbacks and analysis of the activity performed, and quantitative data on

participants' performance: time on task, error rates etc. The goal is to identify usability problems

precisely and to fix them at this iteration. The identified problems are then formalized in a list, and

include a detailed description of each usability problem. The insights inform the third version of the

application, which will be tested in real-life situations at elderly users’ homes during a one-month

period for France (November) and have been tested during a three-month period for Hungary and

Poland (results reported in D6.3).

Participants

In France

The tests were held end of April - beginning of May. Apart from evaluating the usefulness of the

functionalities and the usability of the current interface (essentially paths and place of buttons),

users were also asked to choose the new graphical design of AALFred in between 3 styles.

Following Nielsen's recommendations, the tests involved 6 participants. Three of them (MCD, JLD, JA)

had already tested the 1st version, so that they could comment the improvements; the other three

had participated in the gesture modality tests.

MCD – already tester of V1 of AALFred (July 2013). Unfortunately, “test” per se, based on

predefined tasks’ protocol, could not be done because of bug and new version of AALFred

being implemented. More of a usefulness interview with what could be done.

JLD – already tester of V1 of AALFred (July 2013). Does not use the computer much, and

focuses on the necessity for these kinds of tools to be efficient. He is a very good subject to

test the colour contrast and size of fonts for GUIs: he is one of the rare persons in France

who recovered from Birdshot (which otherwise leads to blindness), but has aftermaths.

JA – already tester of V1 of AALFred (Oct 2013). Uses the computer quite a lot. Most of all,

manages a lot of things from his smartphone – agenda, emails (vocally dictates messages) –

and always shows how his phone allows him to do things efficiently and easily.

JCB – Kinect tester who appreciated gesture modality a lot. Uses the computer quite a lot,

and uses emails “to save time” (he is the president of an association and manages a lot of

member information sharing).

JA – Kinect tester who appreciated gesture modality a lot. She is a former IT developer (in

management tools) who retired 10 years ago. Has 2 computers at home, a desktop in her

office, and a laptop which she likes to carry where she wants in her home. Does not like the

tablet, that, so far, she has not adopted.

CS – “new user” to AALFred, who had participated in the voice data collection. She is a

retired speech therapist. She has a computer, which she uses mainly for email, as well as an


41

Ipad. But according to her, she uses the computer only to do basic things, and appreciated

much more “pen and paper, and books”.

In Hungary

The Second Prototype was tested in June with four users. During the tests the same protocol like in

France was followed with small differences based on the status of the development and the timing of

the evaluations. These differences were that the:

1) voice commands could not be tested because the Hungarian TTS and ASR was not ready at that

time;

2) evaluation of the new graphical interface design (Themes point) was done previously and

separately in May. At that time 10 users were asked to vote for the graphical design of AALFred.

Users involved in testing the second prototype are the ones who will most probably take part in the

field trials:

HGY– already tester of Kinect, gesture modality. Uses computer quite a lot. She is not used to

use touchscreen but very opened for new technologies. She is having dialysis from April,

2014. Emails could be not sent during the test.

SZA – The Hungarian female voice of the AALFred, already tester of Kinect, gesture modality.

Uses computer, emails, applications every day. Uses smart phone to manage many things -

agenda, emails, weather forecast. She is in her last years of her active working period.

HCS - The Hungarian male voice of the ALLFred. Does not use the computer much, and

focuses on the necessity for these kinds of tools to be efficient. He is very good to test

usability and utility of the AALFred because he works with elderlies as a social worker. He will

be retired within a few years.

KI – already tester of Kinect, enjoyed gesture modality a lot, wanted to do the test twice.

Uses the computer quite a lot to manage a lot of things like emails, agenda, keeping contact

with his friends through social network (Facebook). He is a main organizer of several cultural

programs, radio talks. Appreciated that he could test the AALFred.

In Poland

In Poland tests of the second version of AALFred started in July and were held till the end of

September. There were three end users testing the application. We used the same protocol as in

France and Hungary but we had to adapt it to the limitations and the development status of the

Polish version. The following functionalities abovementioned in the protocol were causing main

problems during the first month of testing :

Very limited speech recognition

Contiunuous errors occuring while trying to log into the microsoft account (limited access to

agenda and messages)

Polish testers :


42

TK – recently retired gym coach, still active (part time job as a city guide). Uses computer,

smartphone and a tablet on regular basis. A Kinect owner (plays sport games). Very

interested in Weather App and ‘Find my’ functionality. Due to technical problems and bugs

was unable to test agenda and messages.

ER – retired graphic designer, has part time jobs at home. Uses computer and a smartphone

on a daily basis. Wears eye-glasses and was very interested in speech modality. Took part in

speech collecting.

EC – president of the senior women’s society, very active, perfectly familiar with new

technologies and keen on testing. Uses computer, tablet, smartphone on regular basis.

Already tested Kinect during some other trials carried by her NGO organization. She was

exited of the idea of hving all her contacts and communication services (mails, FB, skype) in

one place.

Protocol

We put here the protocol we gave the participants. The same list of predefined tasks has been used

in the 3 countries, translated in the local language.

You will test a prototype application that integrates different services and features. You are kindly

asked to follow the procedure below and perform all the tasks in the listed order.

Currently, the application is installed on a computer connected to a Smart TV. You will help create a

future version that will control the application, an innovative way, by:

- Remote gestures.

- Voice.

- Touch (with a tablet).

For this test, we are going to use:

- The keyboard and mouse, mainly.

- The voice command for certain actions (see list on the last page).

Main menu

Discover the different modules and services offered by AALFred, browsing on the main menu. At any

time, you can go back to the main page – find out how.

Vocal commands (was not used and tested in Hungary)

We will test together certain vocal commands

Contacts

Droite (x 2)

Gauche (x 2)

Arrière

Messagerie

Descendre / Vers le bas

Monter / Vers le haut


43

Arrière

Préférences

Arrière

Agenda

Till now, you have been using a paper-based calendar or another type of digital agenda (figure 10).

Now that you have adopted AALFred, you want to reschedule some of your activities.

Figure 10: "paper-based" agenda

I. Re-schedule your appointments and activities for the week of May 5 on the agenda.

II. Browse your activities for the week of May 19

III. Go directly to the July 27, and note down a picnic with your friends at East Lake.

Contacts

Some of your friends are saved in your contacts list on AALFred application. You want to confirm to

some of them, by email, the picnic of July 27. For example, you communicate a lot with Ian McKellen

by email. Select other contacts to whom it is possible to send this email.

Messaging

I. Open "Messaging".

II. Check whether your email has been correctly sent to your contacts concerning the picnic.

III. In the inbox, read your new messages.

IV. Answer to the first message asking you to confirm that you arrived well.

V. Click or say "Back" to return to main menu


44

Find my…

I. Open the "Find my" module and give your opinion about the proposed services offered. What are

these services?

II. What more do you think would make these services really useful?

News Reader

Discover the "News Reader" and give us your feedback about this module. What does it remind you

of? Where is the innovation according to you?

Preferences

AALFred is intended to be a "virtual butler," a personal companion to you. Learn how you can

customize AALFred, exploring each category.

Themes (was tested in Hungary separately in May)

AALFred’s graphical interface will be improved once more. You are asked to look at the different

screen captures, make comments, and answer the questionnaire to evaluate the 3 styles that are

considered for the new interface.

List of vocal commands (was not done in Hungary and Poland)

What is your opinion about the vocabulary used? Do you usually use other words? Which ones? Can you give an example of application?

Vocal Command Action

Fin

Aller à la fin de la liste des modules

revenir au dernier module

dernier module

aller au dernier module

revenir à la dernière application

dernière application

aller à la dernière application

moins d'options

Cacher la barre d’options montrer moins d'options

me montrer moins d'options

plus d'options

Montrer la barre d’options montrer plus d'options

me montrer plus d'options

revenir en arrière revenir en arrière

Arrière

vers le bas directions


45

descendre

Gauche

aller à gauche

aller au côté gauche

Droite

aller à droite

aller au côté droit

vers le haut

monter

préférences

Aller au module de préférences préférences utilisateur

aller à mes préférences

contacts

Aller au module de contacts aller à mes contacts

afficher tous les contacts

me montrer tous les contacts

messagerie

Aller au module de messagerie aller à mes messages

montrer mes messages Table 1: List of French vocal commands tested in France

Results Following the user tests made in France, Poland and Hungary, we were able to understand in finer

detail the usability and usefulness issues, and formulate recommendations. The description of

problems and the recommendations are organized per module, and general recommendations are

listed at the end. These have been formalized both in a word document and reported on the bug

tracking tool GitHub.

Main menu

Horizontal dropdown menu was not understood at first sight by JCB. But later, commented

that a menu with smaller icons, with no / little hierarchization in between modules is not

desirable. MCD considered the 3 main items in this order: Contacts – Agenda – Messaging ;

and explains that to take appointments or to send messages, it all starts with the people.

o RECOMMENDATION: Keep proposed new GUI with 3 main items. Possibly allow

personalized organization of 3 main items

Navigation. When coming back to main menu, JCB comes at left hand side beginning of

menu. This lead to user thinking that a module was present twice.


46

o RECOMMENDATION : Come back in main menu at level where one has clicked (if

mouse click or gesture used to navigate)

Subtitles of modules icons considered useful for some users, useless for others. From

observations made during user tests with V2, users do not read the current subtitles. So,

even if some declare that it would be useful, when the subtitles are present, they do not read

it.

Main menu button from modules much appreciated to save time (even if clicking on “more

options” needed first). Is keyboard shortcut considered ?

Theme of the Main menu is appreciated by the Hungarian users. Colours are harmonic, font-

type is legible.

o RECOMMENDATION: a bit bigger font-size

Icons of the Main menu was understood by everyone, SZA noticed that the arrows are

different than on the other pages.

o RECOMMENDATION: integration of the icons used on several pages are needed.

Vocal Commands

The commands were not functioning very well. Of the ones supposed to work, about 2/3 actually

worked out, even lesser number during tests. The feedback was then a limited usefulness from user’s

point of view. JCB says he would prefer gesture (which he tested and liked a lot), which would better

fit his way of life and different situations at home : has a wife, and often has friends at home, and

wouldn’t “stop talking to people to talk to my computer”.

Unnaturalness – because vocal commands were not working very well, CS (former speech therapist)

finds it very unnatural. She noticed that she really had to articulate and slow down, and her

comment is that “I am used to making pronunciation efforts, but it would defeat the purpose if

people have to concentrate and learn the pronunciation to make it work”.

In Hungary vocal commands were not tested because the Hungarian ASR was not ready at that time.

Agenda

These remarks were already pointed out from V1 user tests:

End Time is (nearly) unanimously considered irrelevant by users. Location and Description, is

eventually relevant for some.

Description usefulness limited to being a “reminder” (JCB suggestion: for an “association

meeting” (Subject), note list of documents to bring in (Description). Or may be useful for very

ageing people with cognitive impairments to act as reminder: Ex. Bring social security and

insurance cards (Description) for “Doctor visit” (Subject). The importance of repetitive

reminder was mentioned by HGY for instance reminding elderlies to take their pills on time.

Reminder was also recommended by the other Hungarian HCS user. TL has currently one line

to note appointments in her paper agenda, but if has possibility to write description in

AALFred, would come to use it. JA usually includes short description on the 1 line available.

However, even for those who put descriptions (minority of users) not so much space needed


47

though. CS would use only “description”, which corresponds better to her current use of

paper agenda. But, when she understands that “Subject” must be filled in order to see the

appointments ‘at a glance’ on the week view, sees the interest of filling in “Subject”.

Also, mandatory nature of “description” is not very visible or understandable for some users.

When error occurs, red asterisk is not very visible, and “mandatory field” appears on lower

part of page (not displayed). Thus, users cannot make sense of why error occurred, since

“description” is irrelevant for them. “Subject” is also mandatory, and if not filled, if asterisk

not seen and “back” clicked (what happened for CS, KI), everything is lost.

SZA was suggesting to change the order of the Agenda to have the following: Subject, Time,

location and then the optional description. For HGY the text of the interface was a bit hard to

read.

o RECOMMENDATION :

Remove mandatory nature of “description”

Reduce size of “description” and prioritize “Subject” by giving more screen

space

Generally increase the size of the textboxes and also the font-size

Change the order of the elements

Remove “end time” / at least modify the ‘by default’ 30 minutes duration,

which creates useless errors

Thus, give preeminence on what is essential : 2 elements – Subject + Hour

(Notification considered as useful), eventually “location”, which then could

appear on same page without scrolling

the Hungarian GUI interface elements should be integrated

Also, the first two users did not scroll down the page, and it was only 3rd user who noticed

“location”. So, they were too many elements on the page, with need to scroll. It therefore

confirms the need to restrict to the fundamental.

New appointment. When clicking on a date where no appointment planned, “no

appointment” appears. JCB, JLD, KI clicked on it and expecting it to work.

“+” icon is not clear, nor intuitive (for new messages neither). Would save time to, instead of

clicking on “+”. For KI it was hard to save the appointment because the icon for saving was a

bit strange: Now it is a pipe, which for him doesn't have any connection to saving.

o RECOMMENDATION:

make “no appointment” clickable to get to “new appointment” (instead of

necessarily going to “+” button). Simplification by direct access

changing the save button: having a save button instead of a pipe

Erase / cancel appointment. Currently, to cancel, one must click on date, then right click on

each appointment to pre-select, then click on “more options”, and click on “erase”. TL, HGY

could not find how to erase and was quite frustrated: “it’s better not to have any agenda at


48

all than have an agenda like this! Cancel an appointment is something you do all the time”.

JA could not find it either, and found it absurd.

o RECOMMENDATION:

Selection (for opening) with 2 clicks instead of one ;

Pre-selection with 1 left click (instead of right click),

Avoid right click a maximum, since users will be mainly using the

tablet, where right click is not very intuitive. Also, what right click

being considered: maintaining or sliding down?

Add the erase button much more clearly, below “+” (new appointment)

(same for messaging)

Modify date of appointment – Currently; we can change the details inside the appointment.

But to change the date, one must erase one (not easy!) and then create a new appointment,

inputting all the information again.

o RECOMMENDATION: Simplify date modification

Appointment overlap – JLD wondered what would happen if he planned 2 appointments

during same time lapse. Appointment 1: 8h00 – 17h00. Appointment 2: 10h-10h30. JA and

TL, SZA, KI wondered also about this.

No indication when taking appointment 2 that had already something planned. What he

does in his paper agenda is draw a vertical line from 8 to 17h, so that, at one glance, he

knows that he cannot take another appointment, unless specific personal arrangement. It

would also be useful to have indication on week’s view that there are several appointments

for one day. Currently, a scrolling display of one appointment at a time, that is not quick and

not very visible.

o RECOMMENDATION:

Little notification of previous appointment, and display “at a glance” on

week’s view of several appointments (Ex : display the first 3 appointments,

and then display “3+” which would imply clicking).

Include start hour next to “Subject” on week’s view, so that what appears is

not only “Doctor”, but “Doctor – 10am”

Space would be available if date number (currently very big) reduced

Nationalization of relevant data: what users look for is something that is

“closest as possible to our current paper agenda”. 8th of May is a public

holiday in France, and this info would have been desirable from user’s

perspective.

Display of numbers is problematic and does not correspond to (French) users habits. For

hours, they expect “08”, not “8”. And confusion occurs for the months: “07 juil.” Led user

wondering why he could not have “27” (day he had already selected, but which was masked

by month menu), and only “07” as day (JLD). JA pointed out that confusion stressed by “05”

display, would probably be less with “5”.

o RECOMMENDATION:


49

Harmonize the numbers display across the European countries (if possible),

or make it specific for each country.

Select one or the other: Either put the number “7” or month “July”, not both

Scrolling menu for day and month selection – arrow considered too small; lift not visible

enough; colour to indicate pre-selection too light.

Next / Previous day/week – navigating more rapidly is wished for. When opening a day, e.g

Monday, one has to go back to week view to open Tuesday.

o RECOMMENDATION: include arrows in day view (like arrows for “next / previous

week” of week view).

Recurrent appointment – Is it possible to plan a recurrent appointment, e.g person who

broke his leg and having rehab session each Thursday at 10am during 8 weeks?

Can the location be given/transferred to the Find My...?

Contacts

Important categories. All users consider the 2 most important categories as i) Family, ii)

Friends. Knowing that they are retired, « co-workers » not useful category anymore. They all

wonder what comes under « community ». Also, JLD used no category but only alphabetical

order (which can be considered in some way)

o RECOMMENDATION: only 3 default categories (which would appear on same page

without need to scroll) and possibility of sub categories

1. Family

a. Close family (children, grand-children)

b. Extended family (cousins, nephews)

2. Friends

a. Close friends

b. Buddies

3. Other contacts

In fact, if users all agree upon the utmost importance of “family” and “friends”, some would organize

current “other contacts” as “acquaintance” – people you know but who are not friends, or really

“other contacts”, e.g the plumber.

o RECOMMENDATION:

Allow for personalization and modification of categories (should not be

fixed).

Allow search by alphabetical order

Contacts // Messaging. Since it is possible to send messages from “Contacts”, JCB could not

understand why it was necessary to go “Messaging” to check for “sent messages”. TL finds it

a pity that the sorting out is not consistent – for her, it is necessary to “in messaging, to sort

out by type of message, in contacts to sort out all the messages sent to this specific contact

(makes the parallel with her current use of Mozilla Thunderbird). All agree that a

confirmation of sent message is absolutely necessary.

o RECOMMENDATION: Include:

check messages (received and sent) for each contact,


50

a confirmation message

Also, when checking for “sent messages” (email sent to several contacts), no list of contact

visible, but an automatic scrolling of contacts’ pictures. Too long, and not clear enough

o RECOMMENDATION: allow for a quick and efficient list of contacts who have

received the message

Positioning of buttons considered problematic and lead to errors. The text that appears for

both “edit user’s information” (currently open envelope with @) and “send an email” is

“email”. Several users clicked on “contact email”, which is situated on the right of the

contact’s picture – a pathway that appears to them much more logic. Also, in the window

that pops up, you can “modify” or “cancel”. Users cannot readily understand that “cancel”

means for the action, not “cancel the contact from my list”

o RECOMMENDATION: reverse order / prioritization of “send an email” (below contact

picture) and “edit user’s information” buttons

Type of account – currently when looking at contact list, or category groups, it is not possible

to know who can be contacted by email / FB

o RECOMMENDATION: include small icon so that this (indeed, relevant!) information

can be made visible at a glance

Adding a FB username - SZA found out that currently it is not possible not add someone's FB

username. AALFred forces the user to have one for him/herself and then to log in.

o RECOMMENDATION:

user can add the FB username of her/his contact without having a FB

account

Messaging

Sending a message

For HGY the messaging was not working, which was because of a setting of her email account.

o RECOMMENDATION:

users should be informed about the necessary settings

Writing a message: KI also mentioned that for him the possibility to save the unfinished

email as a draft version would be very useful.

o RECOMMENDATION:

be able to save the draft version of the email

Read v/s unread messages do not appear clearly.

o RECOMMENDATION: Distinguish read v/s unread. Ex :

Bold convention for unread messages, like any email box

Colour change, which might be more appealing for seniors

Closed / open envelope (based on current email type of message icon)


51

Inbox / Sent do not appear clearly with dark / white background. Coupled to lack of bold

indication for unread message, TL says “now, I don’t know where I am, I can’t see anything”

o RECOMMENDATION: Distinguish much more clearly than only colour background

(convention that is not standardized)

Contacts // Messaging. Since it is possible to send messages from “Contacts”, JCB could not

understand why it was necessary to go “Messaging” to check for “sent messages”. Several

actions needed

a. RECOMMENDATION: Include check messages (received and sent) for each contact

Also, when checking for “sent messages” (email sent to several contacts), no list of contact

visible, but an automatic scrolling of contacts’ pictures. Too long, and not clear enough

b. RECOMMENDATION: allow for a quick and efficient list of contacts who have

received the message

Group message – if message needs to be sent to whole group, especially to specific temp

ones (thus necessity to be able to personalize the groups), allow for sending of email from

group, rather than selecting each contact individually.

Email icon – is an open envelope. Coupled to the problem of unread messages not being

distinguished, confusion for user who saw email icon as “read message” since envelope was

open (JLD, JA, HCS, KI).

c. RECOMMENDATION: consider changing the icon.

Answer button – not present on message page. Users look for quite some time at how to

“reply”, some do not succeed in finding and need to be told so that test can continue

(currently: “more options”, “reply” – 2 actions and big distance to travel)

d. RECOMMENDATION: include answer button that is visible and readily available on

same page, at current positioning of type of message icon (Ex: email icon is big and

useless)

Also, seeing at a glance in list of incoming messages the ones which have been replied to

would be considered as useful

Email icon size – when opening an incoming mail, too visible while being useless

e. RECOMMENDATION: reduce size and/or replace it with Answer button

Default message – when answering an email, “Bonjour” appears by default, while the

original message has disappeared. Explore further if this is desirable… or not.

Voice command for email icon + number. Both are extremely confusing. Also, must clarify to

what the number corresponds: countdown, with limited number of email selection with

voice command?

More options. “Reply” is an extremely basic action that should not be in “more options”. I

remember having myself looked for how to delete a message… User SZA, JLD: “we are

already used to emails, so we expect to find the same things that are working”

f. RECOMMENDATION: adopt basic email interface instead of uselessly complexifying

basic actions

Erase message is not easy to find. Like agenda, pre-selection is done by right click, then

“more options”, and then “erase”


52

g. RECOMMENDATION: Add the erase button much more clearly, below “+” (new

message)

Horizontal menu for messaging tried in preferences and checked: not appreciated. Confirms

usability guidelines to avoid horizontal scrolling.

Classify messages – JA suggested the possibility for emails to be classified –like professional

email boxes

Find my…

Users consider that « Find my” would be most useful

o when travelling (because one knows about these services near one’s home)

o on Sundays and at night, to look for open pharmacy

o with, not only the map, but also “how to get there”, especially if driving in a city

centre

o for JA, with other services than the 3 existing ones, i.e restaurants, cinemas, parkings

etc

o RECOMMENDATION:

change icons. Post office looks too much like “email”, and “$” sign badly

perceived for Europeans.

would be nice to see the type of the bank

the PoI appearance sometimes not seeable, should be changed.

the light background of the PoI should be changed.

would be nice if users can search for the name of the hospital for example

because users were saying that many times they don't know the address but

know the name of the hospital.

News Reader

The reader was creating bugs, therefore not available for testing per se. However, perceived

usefulness if RSS Feeds could really be personalized in order to only have access to news one is really

interested in.

Users very much interested in the TTS – to allow them to do other things at the same time “as if we

were listening to the radio”, and very practical if speed can be personalized (preferences) easily.

Preferences

Personnalization of AALFred considered very useful. Only HGY mentioned that it would be nice if the

first setting option would be the setting of the user account. Other than this no suggestions to

improve.

Themes / New GUI

Cf. questionnaires and notes specifically produced for design selection. To sum up :

Hierarchization of modules icons: the 3 proposed ones are the ones which are the essential

ones for the users.

o RECOMMENDATION: keep the horizontal dropdown menu (like current V2)


53

Preference: depends on user and on personal perception of colours. No style considered

perfect, improvements based on “mixing good practices from different styles” suggested,

focusing on

o Readability

o Visibility

o Good colour contrast

o Aesthetics

Font Size

SIZE for readability is really an issue. User test conditions: user sitting at 2,50 from 55” 1080p TV

screen, i.e “normal” expected living room conditions of use for AALFred.

Several texts considered much too small for readability. Ex:

o Find my… - text for name of pharmacy / ATM

o Navigating buttons – much too small to read when mouse hovers overs

o Subtitles in Preferences

o Error messages

RECOMMENDATION: systematically check size of text

Help

Make sure that when “help” button is visible, that it actually works. It gives a very bad impression

when user has difficulties, clicks on “help” because precisely he needs it, that nothing appears.

Icons

Icons must be harmonized. Same button should be used for the same function .

General recommendations

Font size

We must not forget

Who is AALFred user – a person aged 60+.

Our “targets”, even if they wear glasses, have eyesight limitations for most of them.

The conditions of use (figure 11):

The computer is linked to a large screen TV. Typically, when users arrive for the tests and sit

in the living room, if they are not already wearing their glasses, they leave their glasses in

their bag. This means that they wear them for reading but not for watching TV. Therefore, in

this “TV-like” conditions, the font size is essential.

Basic GUI recommendations when designing for older people


54

Figure 11:

Right picture – user sitting normally as error message appears; Left picture – user bending over, shrining, trying to read the error message

RECOMMENDATION: systematically check all font size for all types of text

Integrated messaging

Users do not understand the added value of having an integrated messaging since

Messaging is not “well” linked to “Contacts”. It clearly appears that, what they value, is not

“the message”, but “the person”.

None of the 6 persons having tested AALFred V2 (or any person met for the Kinect tests - 20)

have a Facebook account. It appears that the “cons” are too important as compared to the

“pros”, and this generation of users would not use FB. When they use Skype, it is mainly for

the videoconference. Therefore, it is important to consider a way to put forward the email,

and lessen somehow the social media, especially if developers meet difficulties in integrating

Skype.

RECOMMENDATION: make as many bridges as possible between Contacts and Messaging

Buttons’ position

For the 3 most important modules of AALFred, “agenda”, “contacts” and “messaging”, the tests

revealed that the most basic actions were difficult to accomplish because the buttons for these

actions were difficult to find. For the users (who are used to sending emails), this is just not

acceptable.

o RECOMMENDATION:

o make sure the basic actions, like “reply” are accessible

Icons

With the new design, the icons will probably be automatically changed. But the next user tests must

cater for testing with users the meaning of icons. Ex : the most problematic ones are :

The open envelope, which means “email type of message” – where users understand “read

message”

The “speakers + number” for voice command of message, which is really hard to understand

systematically check: all the buttons should have the save icon if their function is the same


55

Error messages

Must be made explicit and translated.

Right now, mandatory fields in the agenda lead to errors because the compulsory nature of

filling in “description” just does not make sense for many users. The red asterisk is not visible

enough, and they can “go back” without any other error message, and everything is lost.

Must be visible

Ex : red asterisk is not visible (too small + insufficient contrast), error message window is blue

(blue window over blue interface is barely visible, and therefore not understandable)

RECOMMENDATION: ensure visibility, understandability and clarity of error messages. Define

them carefully based on most recurrent errors (which can be listed in future usability tests)

Conclusion As appears clearly in the section above, the user tests allowed to identify an important number of

problems concerning and rectify them so as to improve the next version of the application. Apart

from gathering feedback about the interface (paths, place of buttons etc.,), these tests allowed a fine

understanding of how the seniors organize their daily life - their use of their paper agenda, or how

they look for a hospital - and how they value social interactions. This understanding, together with

the identification of specific usability problems, allowed us to refine and improve the interface and

the way actions are done in AALFred. Before reporting these recommendations as bugs in GitHub,

there were fruitful discussions, between partners doing the user tests and the developers, about the

most efficient way to formalize the insights and present the recommendations. Apart from a few

very specific problems linked to the difficulty of integration, which will be fixed in the final version,

the problems reported above have been solved successfully.

All the users in the 3 countries liked the idea and the available version of the AALFred very much

even if they reported many necessary changes. They were very much able to see the usefulness of

the application and they are really keen on seeing the updated version based on their

recommendations. All users showed their interests for being part of the field trials and to test the

fully developed AALFred application. They expected a lot from the integrated speech and gesture

modality, which, from their point of view, constituted the added value of AALFred.

Planning the field trials

Method The field trials consist in testing the application in people's homes during a one-month period (cf

D6.3). They are complementary to the experimental user tests, which have been made in a

laboratory setting. The aim of field trials is to gather data "real-life", in situ, as people act ordinarily in

the everyday context of their home. The interest of the approach has been confirmed by our

previous experience for similar projects, where the evaluation in the home has been successful.

Second, there is a growing interest to design assisted living technologies to support independence at


56

home ‘in the wild’, i.e. taking account of how real people live in real homes and in communities

(Wherton et al., 2012). These field trials have allowed, more than any other technique used so far, an

understanding of the daily life practices and the subtle aspects of the home, so that they can be

integrated into the design process.

The methods used are mainly observational methods, combining videography (if it is possible) – of a

more focused and data intensive type – and conventional ethnography. These methods involve an

investigator viewing users as they work in a field study, and taking notes of, or video recording the

activity that takes place. Observation may be either direct, where the investigator is actually present

during the task or indirect, where the task is viewed a posteriori when the participants have recorded

their own activity in the absence of an observer. In both cases, the availability of participants and

time for this longitudinal ethnography allows for the observation to be ‘iterative’ also. One such

protocol is the use of “self-confrontation interviews”, where users are asked to comment their own

past actions while they are viewing them. Data elicited in this manner are likely to have greater

“ecological validity”, having the advantage of staying much closer to the actual events than questions

removed from the activity of interest.

4-5 participants have been recruited in France, Hungary, Poland and Portugal

France Hungary Poland Portugal

P1 CR HGY TK EC

P2 CS SZA ER DB

P3 TS KI EC JE

P4 CD HCS MC

P5 ML

Roll-out The following hardware and software have been installed in the home of the participants

France Hungary Poland Portugal

Hardware 55" Samsung LED TV, Dell laptop, Kinect for Xbox with an adapter cable

Samsung T700, Kinect for Windows, LCD monitor

Microsoft Tablet Asus laptop Kinect for Xbox Samsung TV

Asus Transformer Book T100

Software required Windows 8.1 operating system and programs ( Microsoft Speech Platform 11 SDK, Microsoft Speech

required Windows 8.1 operating system and programs ( Microsoft Speech Platform 11 SDK, Microsoft Speech




57

Platform 11 Runtime, Kinect 1.6 Runtime, Java JRE 7 or higher) and the current version of AALFred and interaction modalities



Platform 11 Runtime, Java JRE 7 or higher) and the current version of AALFred and interaction modalities

References Alaoui, M., Lewkowicz, M., & Lan Hing Ting, K., (2014). The urge for empirically-informed design of

social oriented AAL applications – The example of 2 AAL projects, in DSAI 2013 Proceedings.

Baccino, T. et al. (2005). Mesure de l’utilisabilité des Interfaces, Hermès Science Publisher: Paris.

Correia, A., Miranda, L., and Hornung, H. "Gesture-Based Interaction in Domotic Envi-ronments: State

of the Art and HCI Framework Inspired by the Diversity." Human-Computer Interaction–INTERACT

2013. Springer Berlin Heidelberg, 2013. 300-317.

Fisk A.D., et al., (2009). Designing for Older Adults: Principles and Creative Human Factors Approaches

(second edition), London and New York, CRC Press

ISO 9241-210:2010, Ergonomie de l'interaction homme-système -- Partie 210: Conception centrée

sur l'opérateur humain pour les systèmes interactifs

Nichols, T. A., Rogers, W. A., Fisk, A. D. (2006). Design for Ageing. In: Salvendy, G. (Edt.). Handbook of

human factors and ergonomics.Hoboken, N.J: John Wiley (pp. 1418-1458).

Nielsen, J. (1993). Iterative User Interface Design, Jakob Nielsen’s Alertbox: November 1, 1993

Nielsen, J. (1994). Usability engineering, Morgan Kaufmann Publishers Inc. San Francisco, CA

Pak R. and McLaughlin, A., (2011). Designing displays for older adults, Boca Raton, London and New

York, CRC Press

Saposnik, G., Mamdani, M., Bayley, M., Thorpe, K. E., Hall, J., Cohen, L. G., & Teasell, R. (2010).

Clinical trial protocols Effectiveness of Virtual Reality Exercises in STroke Re-habilitation (EVREST):

Rationale, Design, and Protocol of a Pilot Randomized Clinical Trial As-sessing the Wii Gaming

System. International Journal Of Stroke, 5(2) (pp. 47-51).

Wherton, J., Sugarhood, P., Procter, R., Rouncefield, M., Dewsbury, G., Hinder, S., and Green-halgh, T.

(2012). Designing assisted living technologies ‘in the wild’: preliminary experiences with cultural

probe methodology. BMC Medical Research Methodology 2012 12:188.

Documents

D6.2 Preliminary Usability Evaluation Reportdownload.microsoft.com/download/7/1/D/71DC7697-47... · D6.2 Preliminary Usability Evaluation Report Contract no AAL-2009-2-068 Start date