FCT4UWhen private mobile displays meet public

situated displays to enhance the user experience

The 10th IEEE International Conference on Ubiquitous Intelligence and Computing

UIC-2013

Pedro AlbuquerqueRui Neves MadeiraNuno Correia

December 20th, 2013Vietri sul Mare, Italy

Outline

● Introduction● Contributions● Related Work● Principles and Goals● FCT4U System● Personalization● User Study and Results● Conclusions and Future Work

Introduction

● We live surrounded by:– Mobile devices

● e.g., smartphones, tablets, notebooks

– Large situated displays● e.g., TVs, computer, screens, advertising and information

displays

● They should be used to improve the user experience and to develop and improve interaction mechanisms

Introduction

● There is a growing body of research around the area of public displays– It remains a complex and system-specific effort

– Strong focus on advertising content

● There is an opportunity to leverage the ubiquity of personal devices and public displays– Provide useful and personalized information to users

– It is important to add real-time interactivity to increase the system usefulness

Contributions

● FCT4U– (Semi-)public situated display based system

– Provides useful and contextual information to people nearby in an university campus:

● e.g., students, teachers, researchers, etc.

● Personalization approach to enhance the user experience

● Lab user study to assess and validate the system's purpose and interaction modes

Related Work

● Benefits of large displays (e.g., peripheral awareness, productivity, immersiveness)

● How people interact with public displays● Interaction techniques, input sources and ways to abstract them

– Bluetooth scanning has been used to sense user presence and to allow some degree of interaction

● Low entry barrier because it is very common

– Other mentioned input sources include SMS and e-mail

● There has also been some focus on group-based personalization and content broadcasting based on user preferences

Principles & Goals

● Situated displays can be:– Public, shared, interactive or proactive/personalized

● FCT4U combines all of these categories● It aims to:

– Provide information to users in the surrounding space while sharing the public display between up to 3 users

– Expand information, by using mobile and public displays simultaneously

– Promote sharing and interaction among users (e.g., short messages and content sharing)

FCT4U System

● It stands for the faculty's name + for you● The system is composed of:

– Mobile application that runs on smartphones and tablets● Android (version 2.3.3 and up) was chosen as the mobile

platform due to its current market share

– Application running on a situated display● JavaFX-based application to support multiple platforms

– The applications share the same web-based UI● Reusability, portability and adaptability to different form factors

FCT4U System

● Bluetooth was used to sense users' presence– BlueCove (JSR-82 implementation) is used to implement a presence

sensing routine

– There were attempts to use the RSSI to determine the approximate distance from the display

● Communication between the public display and mobile devices is done through the WebSocket protocol– It allows bi-directional communication, while bypassing common

network restrictions

– It supports end-to-end encryption when run over TLS

– It can be used by pure web applications running on modern browsers

Architecture

P2MUCA means Personalization Platform for Multimodal Ubiquitous Computing Applications

P2MUCA OAuth 2.0

Test Intallation

User InterfaceMobile Application

Public Display Application

Widgets

Widgets

Widgets

Mobile Application

● It extends the public display– Allowing users to view information while away from

the public display

– Providing a safe place to show privacy sensitive information and to set preferences

– Working as an input device

● It is used to associate the mobile device's Bluetooth MAC address with a certain user

Messaging & Facebook

(Optional) Location Tracking

● It allows us to automatically determine where someone lives, works and lunches– Density Clustering (DBSCAN algorithm)

● Geo-fencing of public transport hubs is used to determine the preferred transportation mode

Personalization

● P2MUCA– Major focus of my MSc dissertation and also part of

Rui's PhD

– Uses a generic personalization model● It can be configured through an XML configuration file or

a web-based GUI

– It is exposed through an OAuth 2.0 protected HTTP API

Personalization Model

● Resources represent interaction streams, preferences, options or contextual information

● They can be aggregated into parameters using formulas– e.g., 0.2*messagesRead+0.3*messagesReceived+0.5*messagesSent

● Personalizations define multiple options from the combination of multiple parameters

● The framework also supports context segmentation

User Study

● A user study was conducted at CITI– The main focus was to evaluate if the idea of

system like FCT4U was well accepted

– Comparing different system interaction modes was also a major concern

– Personalization evaluation will be the focus of a longer study with more users (in January)

User Study

● The public display application was deployed on a 46” touch-screen● Multiple Android devices were used for the mobile application● Participants went trough 3 scenarios

– Passive, mobile application interaction and touch-screen interaction

– A questionnaire was answered after each scenario

● 9 test subjects who work at CITI– Aged between 26 to 39 years (average of 30.5)

– 2 females

– 8 PhD students and a post-PhD researcher

– 2/3 have mobile devices have Android 2.3.3+ devices

User Study Results

● 7 out of 9 users gave positive scores to the system's usefulness (median of 6)

● They also thought that system could save them time and that it would be even better if was already tailored for their needs

● Passive Mode– Answers were mostly positive

● System responsiveness was considered adequate by most users● The information provided by the passive mode was considered just above

adequate (median of 5)

– The first widgets were the ones seen that users saw (i.e., Map, Weather, Lunch Menus)

User Study Results

● Mobile Application Interaction– It gave users control of what is shown on screen

– It was considered useful and easy to use (answers with medians of 6)

– Answers related with ease of use, user effort and recovering from mistakes got medians of 5 or 6

– But, in general, they considered the application pleasant to use and quick to master (medians of 6 and 7)

User Study Results

● Mobile Application Specific Features– Regarding the short messages services the

reactions were mixed

– Users also did not consider the system very collaborative, even though some recognize its potential to evolve in that direction

– Those are two areas to improve in the future

User Study Results

● Touch Screen Interaction– It was not as well accepted as the mobile

application, getting slightly lower scores on the same questions

● It may be due to people not being used to use such a large touch-screen in an upright position

● Maybe they did not see an advantage over the mobile application

● Nevertheless, 7 out of 9 users considered this mode easy to use

User Study Results

● Users preferred to use the public display alone, but they were able to cope with sharing it

● Overall system UI was rated with a a mean of 5.33– There were no negative scores, but most (6 out of

9) gave neutral or just slightly positive scores● This showed us that there is still room for improvement

– Other more specific metrics had higher scores

User Study Results

● Widgets– Favorite widget: Lunch Menus

– High rated widgets: Map and Weather

– Lower rated widgets: Messages and News

● Users preferred the passive mode– All scenarios had similar scores, but most users said

that they would use the passive mode together with one of the other active interaction modes

– Personalization should increase their satisfaction

User Study Results

● Privacy Sensitive features– 7 out of 9 users were apprehensive about

● Location tracking● Bluetooth visibility

– Users want control over the information they share● But are interested in smart features● A compromise must be found

Conclusions and Future Work

● This paper and presentation followed the development of FCT4U:– A (semi-)public situated display based system that proactively shows

information to nearby users in an university campus, while providing interactivity through a mobile application

– FCT4U's system architecture may be used on other similar systems

– The findings from the user study gave us some insight about the user receptiveness to this type of systems

● A long-term user study with personalization will be conducted to help fine tune the personalization model configuration

● In general, we will continue to add features, improve existing ones and explore new ideas

Thank you for your attention

??Contact: pe.santos@campus.fct.unl.pt